class

CrossEntropyBackward

CrossEntropyBackward

source edit

Python wrappers

pylucid.nn.functional.cross_entropyfunction

Autograd node for multi-class cross-entropy loss (fused log-softmax + NLL).

For a batch of $N$ samples with logits $x \in \mathbb{R}^{N \times C}$ and hard integer targets $y \in \{0, \ldots, C-1\}^{N}$ the loss is

\mathcal{L} = -\frac{1}{N'} \sum_{n: y_n \neq \text{ignore}} w_{y_n} \log p_{n, y_n}, \quad p_{n, c} = \mathrm{softmax}(x_n)_c

where $N'$ counts samples that are not masked by ignore_index and $w$ is the optional per-class weight vector. The backend computes the softmax probabilities internally using the standard log-sum-exp stabilisation $p_{n, c} = e^{x_{n, c} - \max_c x_{n, c}} / Z_n$ and returns them in saved_softmax_ for the backward pass, which emits $\nabla_x = (p - \mathbf{1}_{c = y}) \cdot w_y / N'$ .

Math

\mathcal{L}_{\text{ce}} = -\frac{1}{N'} \sum_{n: y_n \neq \text{ignore}} w_{y_n} \log \frac{\exp(x_{n, y_n})}{\sum_{c=1}^{C} \exp(x_{n, c})}

with the leading $1/N'$ factor present only for Reduction::Mean.

References

Goodfellow, Bengio, Courville, "Deep Learning" §6.2.2 (2016).

Attributes

schema_v1OpSchema

"cross_entropy", AmpPolicy::ForceFP32.

reduction_Reduction

Reduction mode.

eps_double

Numerical clamp passed to the backend (used for label-smoothing paths).

ignore_index_int

Class index whose samples are excluded from both the loss numerator and the denominator

N'

. Default -100 (matches the public API).

has_weight_bool

true when saved_weight_ holds a non-trivial per-class weight.

orig_input_shape_Shape

Full logits shape (N, C, ...) for backward reconstruction.

saved_softmax_Storage

Per-sample softmax probabilities returned by the backend.

saved_target_Storage

Integer class labels.

saved_weight_Storage

Optional per-class weight vector (length

C

saved_valid_count_Storage

Scalar count of non-ignored samples used for the Mean denominator.

Notes

Only the input logits are wired as a saved autograd edge; the target, weight and softmax tensors are stored in the dedicated Storage fields because they are not differentiable inputs.

Static methods

static

forward

→TensorImplPtr

int forward(const int & input, const int & target, const int & weight_or_null, Reduction reduction, double eps, int ignore_index)

source edit

Compute cross-entropy loss with autograd wiring.

Parameters

inputTensorImplPtr

Logits of shape (N, C) or (N, C, d_1, ...).

targetTensorImplPtr

Integer class labels of shape (N,) or (N, d_1, ...).

weight_or_nullTensorImplPtr

Optional per-class weight vector of length

C

. Pass nullptr to disable class weighting.

reductionReduction

Mean, Sum, or None.

epsdouble

Numerical clamp passed to the backend.

ignore_indexint

Class index excluded from the loss / gradient.

Returns

TensorImplPtr

Scalar (Mean / Sum) or per-sample (None) loss tensor.

Methods

apply

int apply(int grad_out)

source edit

Backward pass: gradient w.r.t. logits input.

class

CrossEntropyBackward

CrossEntropyBackward

source edit

Python wrappers

pylucid.nn.functional.cross_entropyfunction

Autograd node for multi-class cross-entropy loss (fused log-softmax + NLL).

For a batch of $N$ samples with logits $x \in \mathbb{R}^{N \times C}$ and hard integer targets $y \in \{0, \ldots, C-1\}^{N}$ the loss is

\mathcal{L} = -\frac{1}{N'} \sum_{n: y_n \neq \text{ignore}} w_{y_n} \log p_{n, y_n}, \quad p_{n, c} = \mathrm{softmax}(x_n)_c

Math

\mathcal{L}_{\text{ce}} = -\frac{1}{N'} \sum_{n: y_n \neq \text{ignore}} w_{y_n} \log \frac{\exp(x_{n, y_n})}{\sum_{c=1}^{C} \exp(x_{n, c})}

with the leading $1/N'$ factor present only for Reduction::Mean.

References

Goodfellow, Bengio, Courville, "Deep Learning" §6.2.2 (2016).

Attributes

schema_v1OpSchema

"cross_entropy", AmpPolicy::ForceFP32.

reduction_Reduction

Reduction mode.

eps_double

Numerical clamp passed to the backend (used for label-smoothing paths).

ignore_index_int

Class index whose samples are excluded from both the loss numerator and the denominator

N'

. Default -100 (matches the public API).

has_weight_bool

true when saved_weight_ holds a non-trivial per-class weight.

orig_input_shape_Shape

Full logits shape (N, C, ...) for backward reconstruction.

saved_softmax_Storage

Per-sample softmax probabilities returned by the backend.

saved_target_Storage

Integer class labels.

saved_weight_Storage

Optional per-class weight vector (length

C

saved_valid_count_Storage

Scalar count of non-ignored samples used for the Mean denominator.

Notes

Only the input logits are wired as a saved autograd edge; the target, weight and softmax tensors are stored in the dedicated Storage fields because they are not differentiable inputs.

Static methods

static

forward

→TensorImplPtr

int forward(const int & input, const int & target, const int & weight_or_null, Reduction reduction, double eps, int ignore_index)

source edit

Compute cross-entropy loss with autograd wiring.

Parameters

inputTensorImplPtr

Logits of shape (N, C) or (N, C, d_1, ...).

targetTensorImplPtr

Integer class labels of shape (N,) or (N, d_1, ...).

weight_or_nullTensorImplPtr

Optional per-class weight vector of length

C

. Pass nullptr to disable class weighting.

reductionReduction

Mean, Sum, or None.

epsdouble

Numerical clamp passed to the backend.

ignore_indexint

Class index excluded from the loss / gradient.

Returns

TensorImplPtr

Scalar (Mean / Sum) or per-sample (None) loss tensor.

Methods

apply

int apply(int grad_out)

source edit

Backward pass: gradient w.r.t. logits input.