Backward-graph fusion pass that collapses adjacent fusible nodes.
FusionPass is invoked once per backward call, just before the
Engine begins traversing the graph. It scans the topologically-ordered
node list for known multi-node patterns (see FusionPattern) and
rewrites them in place so that subsequent traversal launches fewer
kernels and allocates fewer intermediate gradient buffers.
The pass is stateless across runs except for the stats_ counters,
which are reset on each call to run. No node ownership changes:
raw pointers reference nodes owned by their parent edges' shared_ptr.
See Also
run_fusion_pass — convenience entry point used by
Engine::backward().
Attributes
stats_Statsrun call. Cleared at the entry of each run.Notes
Cost / benefit trade-off. Fusing eliminates intermediate gradient buffers and kernel-launch overhead but tightens the dependency between adjacent ops — the fused node must materialise all saved tensors that either sub-op would have needed independently. For sub-graphs that already share most of their saved state (Linear+activation, Conv+BN) this is a clear win; for unrelated ops it is not.
Detection only. All try_fuse_* helpers currently return true
as placeholders — actual kernel substitution will be layered in per
pattern in subsequent phases. The counters in Stats therefore
reflect detected opportunities rather than executed fusions.
Invariants.
- FusionPass never takes ownership of any
Node. - A successful fusion erases the consumed nodes from the graph vector and leaves the surviving (fused) node in place at the same index.
Constructors
1FusionPass
void FusionPass()Construct a fresh FusionPass with zeroed counters.
Defaulted; the only state is the stats_ Stats member,
whose POD counters are zero-initialised by their in-class default
initialisers. Each run call re-zeroes stats_ at entry,
so re-using a single instance across backward calls is safe.
Methods
6Scan graph for fusible patterns and apply matches in place.
Walks the node list from front to back examining a 2-node window for
Linear+Activation fusions and a 4-node window for Scaled Dot-Product
Attention fusions. When a pattern matches and the corresponding
try_fuse_* helper returns true, the consumed (non-surviving)
nodes are erased from graph and the index counter is rewound so
the merged node is re-examined on the next iteration.
Parameters
graphstd::vector<Node*>&Returns
intTotal number of fusions performed during this call (== Stats::total).
Notes
The stats_ counters are zeroed at function entry, so calling
run repeatedly on the same instance is safe.
Read the per-pattern counters collected by the last run.
Returns
const Stats&Reference to this pass's counters. Stable until the next call to run, which zeroes them.
try_fuse_conv_bn_relu
→boolbool try_fuse_conv_bn_relu(Node * conv, Node * bn, Node * relu)Attempt to fuse a Conv + BatchNorm + ReLU backward triple.
Parameters
Returns
booltrue on success; caller removes bn and relu from the graph and leaves conv as the surviving fused node.
Notes
Placeholder — returns true unconditionally pending the
implementation of the fused Conv+BN+ReLU backward kernel.
Attempt to fuse a sliding window of LayerNorm backward sub-nodes.
Parameters
windowstd::vector<Node*>&Returns
booltrue if the window matched a LayerNorm pattern and was fused. Currently always returns false (LayerNorm fusion not yet implemented).
try_fuse_linear_activation
→boolbool try_fuse_linear_activation(Node * linear_node, Node * act_node, FusionPattern pattern)Attempt to fuse a Linear backward node with an adjacent activation backward node.
Parameters
linear_nodeNode*act_nodeNode*linear_node's output.patternFusionPatternLinearRelu / LinearGelu / LinearSilu). Selects the fused kernel that will eventually replace the pair.Returns
booltrue when the pair was successfully fused (caller removes act_node from the graph), false to skip.
Notes
Current implementation is a placeholder returning true
unconditionally — the actual fused-kernel substitution is staged
for a follow-up phase.
try_fuse_sdpa
→boolbool try_fuse_sdpa(Node * mm1, Node * scale, Node * softmax, Node * mm2)Attempt to fuse a four-node Scaled Dot-Product Attention backward chain.
Parameters
Returns
booltrue on success; caller erases scale, softmax, and mm2 leaving mm1 as the merged SDPA node.
Notes
Placeholder implementation; counts the detection but does not yet substitute a fused attention-backward kernel.