class

BilinearLayerBackward

BilinearLayerBackward

source edit

Autograd node for the bilinear (rank-3-weight) layer.

Implements the bilinear form

y_{n, o} = \sum_{p=1}^{D_1} \sum_{q=1}^{D_2} x_{1, n, p} \, W_{o, p, q} \, x_{2, n, q} + b_o

over arbitrary leading batch dimensions, equivalent to the contraction y = einsum("...p,opq,...q->...o", x1, W, x2) + b. Both inputs and the weight are saved so that the backward can compute

\nabla_{x_1, n, p} = \sum_{o, q} W_{o, p, q} \, x_{2, n, q} \, g_{n, o}, \quad \nabla_{x_2, n, q} = \sum_{o, p} W_{o, p, q} \, x_{1, n, p} \, g_{n, o}, \quad \nabla_{W, o, p, q} = \sum_{n} x_{1, n, p} \, x_{2, n, q} \, g_{n, o}.

The bias gradient is simply grad_out reduced over all batch axes.

Math

y = x_1 \,W\, x_2^\top + b, \quad W \in \mathbb{R}^{D_{\text{out}} \times D_1 \times D_2}, \; b \in \mathbb{R}^{D_{\text{out}}}.

Attributes

schema_v1OpSchema

"bilinear_layer", AmpPolicy::Promote.

orig_x1_shape_Shape

Saved x1 shape for backward reconstruction.

orig_x2_shape_Shape

Saved x2 shape for backward reconstruction.

Notes

All leading "batch" dimensions of x1 and x2 must match exactly. The bias is detected as absent (no bias term) when the saved bias shape recorded in input_shapes_[3] is empty.

Static methods

static

forward

→TensorImplPtr

int forward(const int & x1, const int & x2, const int & weight, const int & bias)

source edit

Compute the bilinear layer with autograd wiring.

Parameters

x1TensorImplPtr

Tensor whose last dimension equals

D_1

x2TensorImplPtr

Tensor whose last dimension equals

D_2

and whose leading shape matches that of x1.

weightTensorImplPtr

Weight tensor of shape (D_out, D_1, D_2).

biasTensorImplPtr

Optional bias vector of shape (D_out,). Pass nullptr to disable the additive bias.

Returns

TensorImplPtr

Output tensor of shape (*, D_out) where * denotes the shared leading shape of x1 / x2.

Methods

apply

int apply(int grad_out)

source edit

Backward pass: gradients w.r.t. x1, x2, weight, and (optionally) bias.

class

BilinearLayerBackward

BilinearLayerBackward

source edit

Autograd node for the bilinear (rank-3-weight) layer.

Implements the bilinear form

y_{n, o} = \sum_{p=1}^{D_1} \sum_{q=1}^{D_2} x_{1, n, p} \, W_{o, p, q} \, x_{2, n, q} + b_o

over arbitrary leading batch dimensions, equivalent to the contraction y = einsum("...p,opq,...q->...o", x1, W, x2) + b. Both inputs and the weight are saved so that the backward can compute

\nabla_{x_1, n, p} = \sum_{o, q} W_{o, p, q} \, x_{2, n, q} \, g_{n, o}, \quad \nabla_{x_2, n, q} = \sum_{o, p} W_{o, p, q} \, x_{1, n, p} \, g_{n, o}, \quad \nabla_{W, o, p, q} = \sum_{n} x_{1, n, p} \, x_{2, n, q} \, g_{n, o}.

The bias gradient is simply grad_out reduced over all batch axes.

Math

y = x_1 \,W\, x_2^\top + b, \quad W \in \mathbb{R}^{D_{\text{out}} \times D_1 \times D_2}, \; b \in \mathbb{R}^{D_{\text{out}}}.

Attributes

schema_v1OpSchema

"bilinear_layer", AmpPolicy::Promote.

orig_x1_shape_Shape

Saved x1 shape for backward reconstruction.

orig_x2_shape_Shape

Saved x2 shape for backward reconstruction.

Notes

All leading "batch" dimensions of x1 and x2 must match exactly. The bias is detected as absent (no bias term) when the saved bias shape recorded in input_shapes_[3] is empty.

Static methods

static

forward

→TensorImplPtr

int forward(const int & x1, const int & x2, const int & weight, const int & bias)

source edit

Compute the bilinear layer with autograd wiring.

Parameters

x1TensorImplPtr

Tensor whose last dimension equals

D_1

x2TensorImplPtr

Tensor whose last dimension equals

D_2

and whose leading shape matches that of x1.

weightTensorImplPtr

Weight tensor of shape (D_out, D_1, D_2).

biasTensorImplPtr

Optional bias vector of shape (D_out,). Pass nullptr to disable the additive bias.

Returns

TensorImplPtr

Output tensor of shape (*, D_out) where * denotes the shared leading shape of x1 / x2.

Methods

apply

int apply(int grad_out)

source edit

Backward pass: gradients w.r.t. x1, x2, weight, and (optionally) bias.