elu

→Tensor

elu(x: Tensor, alpha: float = 1.0, inplace: bool = False)

source

Exponential linear unit activation.

A smooth alternative to relu that saturates to $-\alpha$ on the negative side instead of clipping at zero. The exponential branch pushes the mean activation closer to zero, which speeds up training by reducing internal covariate shift (Clevert et al. 2015).

Parameters

xTensor

Input tensor of any shape; activation is element-wise.

alphafloat= 1.0

Saturation value

\alpha > 0

for negative inputs. Default 1.0.

inplacebool= False

Accepted for API compatibility; currently ignored.

Returns

Tensor

Activated tensor with the same shape as x.

Notes

\text{ELU}(x) = \begin{cases} x & x > 0 \\ \alpha (e^x - 1) & x \le 0 \end{cases}

Continuous and once-differentiable at the origin (derivative is $1$ from above and $\alpha$ from below — equal when $\alpha = 1$ ). Unlike ReLU it produces negative outputs, which helps push activation means toward zero; unlike Leaky ReLU it bounds the negative tail, providing implicit noise robustness.

Examples

>>> import lucid
>>> from lucid.nn.functional import elu
>>> x = lucid.tensor([-2.0, -1.0, 0.0, 1.0])
>>> elu(x)
Tensor([-0.8647, -0.6321,  0.0000,  1.0000])