class

Sigmoid

extendsModule

Sigmoid()

source

Sigmoid (logistic) activation function.

Applies element-wise:

\sigma(x) = \frac{1}{1 + e^{-x}}

Maps all real inputs to the open interval $(0, 1)$ . Commonly used in the output layer of binary classifiers and as a gating function in recurrent architectures (LSTM, GRU).

Notes

Input: $(*)$ — any shape.
Output: $(*)$ — same shape as input, values in $(0, 1)$ .

Sigmoid saturates for large absolute inputs, causing vanishing gradients in deep networks. For hidden layers, ReLU or GELU are generally preferred.

Examples

>>> import lucid
>>> import lucid.nn as nn
>>> m = nn.Sigmoid()
>>> x = lucid.tensor([-2.0, -1.0, 0.0, 1.0, 2.0])
>>> m(x)
tensor([0.1192, 0.2689, 0.5   , 0.7311, 0.8808])
>>> # Binary classification output layer
>>> x = lucid.randn(16, 1)
>>> probs = m(x)   # probabilities in (0, 1)
>>> probs.shape
(16, 1)

Methods (1)

forward

→Tensor

forward(x: Tensor)

source

Apply the activation function element-wise.

Parameters

inputTensor

Input tensor of arbitrary shape.

Returns

Tensor

Output tensor of the same shape as input.