class

CosineEmbeddingLoss

extendsModule

CosineEmbeddingLoss(margin: float = 0.0, reduction: Reduction = 'mean')

source edit

Cosine embedding loss for learning similarity/dissimilarity.

Measures whether two inputs are similar or dissimilar using cosine similarity. The label y must be +1 (similar) or -1 (dissimilar):

\ell(x_1, x_2, y) = \begin{cases} 1 - \cos(x_1, x_2) & \text{if } y = +1 \\[4pt] \max\!\bigl(0,\; \cos(x_1, x_2) - \text{margin}\bigr) & \text{if } y = -1 \end{cases}

where $\cos(x_1, x_2) = \dfrac{x_1 \cdot x_2}{\|x_1\| \|x_2\|}$ .

Parameters

marginfloat= 0.0

Minimum cosine similarity below which dissimilar pairs incur zero loss. Must be in

(-1, 1)

. Default 0.0.

reductionstr= 'mean'

'none' | 'mean' (default) | 'sum'.

Attributes

marginfloat

The cosine similarity margin for dissimilar pairs.

reductionstr

The reduction mode.

Notes

x1 : $(N, D)$ .
x2 : $(N, D)$ .
y : $(N,)$ with values in $\{-1, +1\}$ .
Output : scalar for 'mean' / 'sum'; $(N,)$ for 'none'.

Useful when the magnitude of embeddings is not informative — only the angle between them matters (e.g. sentence similarity, image pairing).
Setting margin > 0 means dissimilar pairs are only penalised when their cosine similarity exceeds the margin.

Examples

Paired sentence embeddings:
>>> import lucid
>>> import lucid.nn as nn
>>> criterion = nn.CosineEmbeddingLoss(margin=0.5)
>>> x1 = lucid.tensor([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]])
>>> x2 = lucid.tensor([[0.9, 0.1, 0.0], [0.0, 0.0, 1.0]])
>>> y  = lucid.tensor([1.0, -1.0])
>>> loss = criterion(x1, x2, y)
Zero margin for dissimilar pairs:
>>> import lucid
>>> import lucid.nn as nn
>>> criterion = nn.CosineEmbeddingLoss()
>>> x1 = lucid.tensor([[1.0, 2.0]])
>>> x2 = lucid.tensor([[2.0, 1.0]])
>>> y  = lucid.tensor([1.0])
>>> loss = criterion(x1, x2, y)

Used by 1

lucid.nn.modules

Constructors

dunder

init

→None

__init__(margin: float = 0.0, reduction: Reduction = 'mean')

source edit

Initialise the CosineEmbeddingLoss module. See the class docstring for parameter semantics.

Instance methods

extra_repr

→str

extra_repr()

source edit

Return a string representation of the layer's configuration.

forward

→Tensor

forward(x1: Tensor, x2: Tensor, y: Tensor)

source edit

Compute the loss between predictions and targets.

Parameters

x1Tensor

Input tensor.

x2Tensor

Input tensor.

yTensor

Input tensor.

Returns

Tensor

Scalar loss (or unreduced tensor depending on reduction).

Paired sentence embeddings: >>> import lucid >>> import lucid.nn as nn >>> criterion = nn.CosineEmbeddingLoss(margin=0.5) >>> x1 = lucid.tensor([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]]) >>> x2 = lucid.tensor([[0.9, 0.1, 0.0], [0.0, 0.0, 1.0]]) >>> y = lucid.tensor([1.0, -1.0]) >>> loss = criterion(x1, x2, y) Zero margin for dissimilar pairs: >>> import lucid >>> import lucid.nn as nn >>> criterion = nn.CosineEmbeddingLoss() >>> x1 = lucid.tensor([[1.0, 2.0]]) >>> x2 = lucid.tensor([[2.0, 1.0]]) >>> y = lucid.tensor([1.0]) >>> loss = criterion(x1, x2, y)