CRTP base for single-input multi-axis reduction ops (sum, mean, prod, max, min, …).
Concrete reductions inherit as
class SumBackward : public ReduceKernel<SumBackward> and supply
(a) a static constexpr OpSchema schema_v1, (b) one of the kernel
hooks recognised by HasReduceDispatch /
HasReduceGpuKernel (or a plain cpu_kernel static), and
(c) a grad_formula(grad_out) -> Storage that converts the upstream
gradient into the input gradient. The base owns axis normalisation,
output-shape computation, GPU/CPU dispatch, and the extra
reduction-state plumbing that the backward formula needs.
Unlike UnaryKernel, the forward output has a different
shape than the input, so the backward pass must broadcast the
gradient from the reduced shape back to the full input extent. That
broadcast is encoded once in apply_for_graph (graph-mode) and
is the responsibility of grad_formula in storage-mode.
Math
scale_op is identity for sum, for mean,
for prod, an argmax/argmin indicator for max/min.
Template Parameters
Derived : class
CRTP self-type. Must expose schema_v1, a kernel hook
satisfying one of the reduction concepts (or cpu_kernel), and
grad_formula. May override scale_graph_grad for graph-mode
scaling (mean, prod, …).
See Also
UnaryKernel, BinaryKernel, NaryKernel —
sibling CRTP bases for shape-preserving ops.
SumBackward, MeanBackward, ProdBackward,
MaxBackward, MinBackward — canonical consumers.
Attributes
kSavesInputbooltrue by default — most reductions need the input activation during backward. SumBackward overrides this to false.kSavesOutputboolfalse by default. ProdBackward / MaxBackward / MinBackward override to true so they can route gradient through the saved output (or its argmax indicator).kHasGradientbooltrue. All canonical reductions participate in autograd.reduce_axes_std::vector<int>forward for use in apply_for_graph and in the derived grad_formula.keepdims_boolfull_input_shape_ShapeNotes
Dispatch order. forward chooses the kernel path with
if constexpr:
HasReduceDispatch— callDerived::dispatch(backend, storage, shape, axes, keepdims, dtype).- GPU +
HasReduceGpuKernel— callDerived::gpu_kernel. - CPU — call
Derived::cpu_kernel.
Contiguity. CPU inputs are routed through contiguous_op
before dispatch; GPU inputs are passed through as MLX handles strides
natively.
Static methods
1int forward(const int & a, const int & axes_user, bool keepdims)Forward entry point: run the reduction and wire the backward node.
Normalises axes_user (negative indices → positive, dedup),
computes the output shape (respecting keepdims), enforces CPU
contiguity, dispatches to the most specific kernel hook the
Derived class exposes, and — when GradMode is enabled and
the input requires grad — constructs the backward node with the
extra reduction state (reduce_axes_, keepdims_,
full_input_shape_) populated.
Parameters
aconst std::shared_ptr<TensorImpl>&axes_userconst std::vector<int>&keepdimsbooltrue, every reduced axis appears as a size-1 dim in the output; otherwise it is removed.Returns
std::shared_ptr<TensorImpl>Output tensor with the reduced shape and the AMP-effective dtype (driven by schema_v1.amp_policy).
Raises
LucidErrora is null.NotImplementedDerived exposes neither HasReduceDispatch nor HasReduceGpuKernel.Methods
5Storage-mode backward: delegate the full broadcast-and-scale to Derived::grad_formula.
Unlike UnaryKernel::apply, this method performs no
reduce_grad_to_shape step — the reduction kernel's
grad_formula is responsible for broadcasting the upstream
gradient back to full_input_shape_ using the saved
reduce_axes_ and keepdims_ fields, and for any
reduction-specific scaling.
Parameters
grad_outStorageReturns
std::vector<Storage>Single-element vector containing the input gradient, shaped like full_input_shape_.
Graph-mode backward: rebuild the broadcast in tracked ops.
Re-inserts size-1 axes for any dims that were dropped because
keepdims=False, runs broadcast_to_op to expand the
gradient back to full_input_shape_, then defers to
scale_graph_grad for the reduction-specific factor. The
entire chain participates in the autograd graph so higher-order
gradients work correctly.
Parameters
grad_outconst TensorImplPtr&TensorImpl (i.e. a graph node), not just a Storage.Returns
std::vector<TensorImplPtr>Single-element vector containing the input gradient tensor.
Schema-derived op name, exposed to the IKernel virtual interface for error messages and profiler scopes.
Returns
std::string_viewDerived::schema_v1.name (e.g. "sum", "mean").
Schema-derived node name used by the autograd graph display and create_graph error reports.
Returns
std::stringOwning copy of Derived::schema_v1.name.
Graph-mode gradient scaling hook.
Default implementation passes the gradient through unchanged,
matching the sum rule (every reduced element contributed with
unit weight). Derived classes whose rule has a non-trivial scale
— MeanBackward divides by , ProdBackward
multiplies by , the argmax variants apply an indicator
mask — override this method to inject their factor before the
gradient is returned.
Parameters
gconst TensorImplPtr&Returns
TensorImplPtrThe scaled gradient — either g itself or a fresh tensor produced by a graph-tracked op.