tanh

→Tensor

tanh(x: Tensor)

source

Hyperbolic tangent activation.

A rescaled, recentred sigmoid that maps the real line to $(-1, 1)$ . Because outputs are zero-centred, tanh is generally preferred over sigmoid for hidden layers in shallow networks and is the canonical squashing function in RNN cells.

Parameters

xTensor

Input tensor of any shape; activation is element-wise.

Returns

Tensor

Activated tensor with the same shape as x, values in $(-1, 1)$ .

Notes

\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}} = 2\sigma(2x) - 1

Derivative $1 - \tanh^2(x)$ , maximum value $1$ at the origin. Saturates for $|x| \gtrsim 3$ ; in deep networks this leads to vanishing gradients, so modern feed-forward stacks prefer ReLU-family activations.

Examples

>>> import lucid
>>> from lucid.nn.functional import tanh
>>> x = lucid.tensor([-2.0, 0.0, 2.0])
>>> tanh(x)
Tensor([-0.9640,  0.0000,  0.9640])