class

CosineEmbeddingLoss

extendsModule

CosineEmbeddingLoss(margin: float = 0.0, reduction: str = 'mean')

source

Cosine embedding loss for learning similarity/dissimilarity.

Measures whether two inputs are similar or dissimilar using cosine similarity. The label y must be +1 (similar) or -1 (dissimilar):

\ell(x_1, x_2, y) = \begin{cases} 1 - \cos(x_1, x_2) & \text{if } y = +1 \\[4pt] \max\!\bigl(0,\; \cos(x_1, x_2) - \text{margin}\bigr) & \text{if } y = -1 \end{cases}

where $\cos(x_1, x_2) = \dfrac{x_1 \cdot x_2}{\|x_1\| \|x_2\|}$ .

Parameters

marginfloat= 0.0

Minimum cosine similarity below which dissimilar pairs incur zero loss. Must be in

(-1, 1)

. Default 0.0.

reductionstr= 'mean'

'none' | 'mean' (default) | 'sum'.

Attributes

marginfloat

The cosine similarity margin for dissimilar pairs.

reductionstr

The reduction mode.

Notes

x1 : $(N, D)$ .
x2 : $(N, D)$ .
y : $(N,)$ with values in $\{-1, +1\}$ .
Output : scalar for 'mean' / 'sum'; $(N,)$ for 'none'.

Useful when the magnitude of embeddings is not informative — only the angle between them matters (e.g. sentence similarity, image pairing).
Setting margin > 0 means dissimilar pairs are only penalised when their cosine similarity exceeds the margin.

Examples

Paired sentence embeddings:
>>> import lucid
>>> import lucid.nn as nn
>>> criterion = nn.CosineEmbeddingLoss(margin=0.5)
>>> x1 = lucid.tensor([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]])
>>> x2 = lucid.tensor([[0.9, 0.1, 0.0], [0.0, 0.0, 1.0]])
>>> y  = lucid.tensor([1.0, -1.0])
>>> loss = criterion(x1, x2, y)
Zero margin for dissimilar pairs:
>>> import lucid
>>> import lucid.nn as nn
>>> criterion = nn.CosineEmbeddingLoss()
>>> x1 = lucid.tensor([[1.0, 2.0]])
>>> x2 = lucid.tensor([[2.0, 1.0]])
>>> y  = lucid.tensor([1.0])
>>> loss = criterion(x1, x2, y)

Methods (3)

dunder

init

→None

__init__(margin: float = 0.0, reduction: str = 'mean')

source

Initialise the CosineEmbeddingLoss module. See the class docstring for parameter semantics.

forward

→Tensor

forward(x1: Tensor, x2: Tensor, y: Tensor)

source

Compute the loss between predictions and targets.

Parameters

x1Tensor

Input tensor.

x2Tensor

Input tensor.

yTensor

Input tensor.

Returns

Tensor

Scalar loss (or unreduced tensor depending on reduction).

extra_repr

→str

extra_repr()

source

Return a string representation of the layer's configuration.