class

GaussianNLLLoss

extendsModule

GaussianNLLLoss(full: bool = False, eps: float = 1e-06, reduction: str = 'mean')

source

Gaussian negative log-likelihood loss for heteroscedastic regression.

Models the negative log-likelihood of a Gaussian distribution whose mean and variance are both predicted by the network. For predicted mean $\mu$ (x), target $y$ , and predicted variance $\sigma^2$ (var):

\ell(\mu, y, \sigma^2) = \frac{1}{2}\left(\log \sigma^2 + \frac{(y - \mu)^2}{\sigma^2}\right)

When full=True, the constant term $\frac{1}{2}\log(2\pi)$ is also included. The variance is clamped below by eps to prevent division by zero.

Parameters

fullbool= False

If True, adds the

\frac{1}{2}\log(2\pi)

constant. Default False.

epsfloat= 1e-06

Minimum value for the variance. Default 1e-6.

reductionstr= 'mean'

'none' | 'mean' (default) | 'sum'.

Attributes

fullbool

Whether the full Gaussian constant is included.

epsfloat

Variance lower bound.

reductionstr

The reduction mode.

Notes

x (mean) : $(N, *)$ — predicted means.
target : $(N, *)$ — observed values.
var : $(N, *)$ or $(N, 1)$ — predicted variances (must be positive).
Output : scalar for 'mean' / 'sum'; $(N, *)$ for 'none'.

This loss is also called the heteroscedastic regression loss because the noise level is input-dependent (predicted by the model rather than fixed).
Minimising this loss simultaneously trains the model to produce accurate predictions (via the residual term) and well-calibrated uncertainty estimates (via the log-variance term).

Examples

Predicting mean and variance simultaneously:
>>> import lucid
>>> import lucid.nn as nn
>>> criterion = nn.GaussianNLLLoss()
>>> mean = lucid.tensor([1.5, 2.0, 0.5])
>>> var  = lucid.tensor([0.5, 1.0, 0.2])
>>> y    = lucid.tensor([1.0, 2.5, 0.3])
>>> loss = criterion(mean, y, var)
With full Gaussian constant:
>>> import lucid
>>> import lucid.nn as nn
>>> criterion = nn.GaussianNLLLoss(full=True, eps=1e-4)
>>> mean = lucid.tensor([[0.0, 1.0], [2.0, 3.0]])
>>> var  = lucid.tensor([[0.1, 0.2], [0.3, 0.4]])
>>> y    = lucid.tensor([[0.1, 0.8], [2.2, 2.8]])
>>> loss = criterion(mean, y, var)

Methods (3)

dunder

init

→None

__init__(full: bool = False, eps: float = 1e-06, reduction: str = 'mean')

source

Initialise the GaussianNLLLoss module. See the class docstring for parameter semantics.

forward

→Tensor

forward(x: Tensor, target: Tensor, var: Tensor)

source

Compute the loss between predictions and targets.

Parameters

xTensor

Input tensor.

targetTensor

Input tensor.

varTensor

Input tensor.

Returns

Tensor

Scalar loss (or unreduced tensor depending on reduction).

extra_repr

→str

extra_repr()

source

Return a string representation of the layer's configuration.