class

CosineSimilarity

extendsModule

CosineSimilarity(dim: int = 1, eps: float = 1e-08)

source edit

Cosine similarity between two tensors along a specified dimension.

Computes element-wise cosine similarity:

\text{CosineSimilarity}(x_1, x_2) = \frac{x_1 \cdot x_2}{\max\!\bigl(\|x_1\|_2 \cdot \|x_2\|_2,\; \varepsilon\bigr)}

The result is a scalar (or tensor with dim reduced) in the range $[-1, 1]$ , where $1$ indicates identical direction, $0$ orthogonality, and $-1$ opposite direction. The eps floor on the denominator prevents division by zero for zero-norm vectors.

Parameters

dimint= 1

Dimension along which cosine similarity is computed. Default: 1.

epsfloat= 1e-08

Small value

\varepsilon

added to the denominator for numerical stability. Default: 1e-8.

Notes

Input x1: $(N, D)$ or any shape broadcastable to x2.
Input x2: $(N, D)$ or any shape broadcastable to x1.
Output: $(N,)$ — dim is reduced; same batch dimensions as input.

Examples

>>> import lucid
>>> import lucid.nn as nn
>>> m = nn.CosineSimilarity(dim=1)
>>> x1 = lucid.tensor([[1.0, 0.0], [0.0, 1.0]])
>>> x2 = lucid.tensor([[1.0, 0.0], [1.0, 0.0]])
>>> m(x1, x2)
tensor([1., 0.])
>>> # Comparing sentence embeddings along the feature dimension
>>> emb1 = lucid.randn(32, 768)
>>> emb2 = lucid.randn(32, 768)
>>> sim = nn.CosineSimilarity(dim=1)(emb1, emb2)
>>> sim.shape
(32,)

Used by 1

lucid.nn.modules

Constructors

dunder

init

→None

__init__(dim: int = 1, eps: float = 1e-08)

source edit

Initialise the CosineSimilarity module. See the class docstring for parameter semantics.

Instance methods

extra_repr

→str

extra_repr()

source edit

Return a string representation of the layer's configuration.

forward

→Tensor

forward(x1: Tensor, x2: Tensor)

source edit

Apply the activation function element-wise.

Parameters

inputTensor

Input tensor of arbitrary shape.

Returns

Tensor

Output tensor of the same shape as input.

>>> import lucid >>> import lucid.nn as nn >>> m = nn.CosineSimilarity(dim=1) >>> x1 = lucid.tensor([[1.0, 0.0], [0.0, 1.0]]) >>> x2 = lucid.tensor([[1.0, 0.0], [1.0, 0.0]]) >>> m(x1, x2) tensor([1., 0.]) >>> # Comparing sentence embeddings along the feature dimension >>> emb1 = lucid.randn(32, 768) >>> emb2 = lucid.randn(32, 768) >>> sim = nn.CosineSimilarity(dim=1)(emb1, emb2) >>> sim.shape (32,)