class

ELU

extendsModule

ELU(alpha: float = 1.0, inplace: bool = False)

source

Exponential Linear Unit activation function.

Applies element-wise:

\text{ELU}(x) = \begin{cases} x & \text{if } x > 0 \\ \alpha \left(e^{x} - 1\right) & \text{otherwise} \end{cases}

ELU smoothly saturates to $-\alpha$ for large negative inputs, producing mean activations closer to zero than ReLU. This self-normalising tendency can accelerate convergence in deep networks.

Parameters

alphafloat= 1.0

Scale

\alpha

for the negative exponential branch. Default: 1.0.

inplacebool= False

If True, modifies the input tensor in-place. Default: False.

Notes

Input: $(*)$ — any shape.
Output: $(*)$ — same shape as input.

Examples

>>> import lucid
>>> import lucid.nn as nn
>>> m = nn.ELU(alpha=1.0)
>>> x = lucid.tensor([-2.0, -1.0, 0.0, 1.0, 2.0])
>>> m(x)
tensor([-0.8647, -0.6321,  0.    ,  1.    ,  2.    ])
>>> # Custom alpha shifts the negative saturation floor
>>> m = nn.ELU(alpha=0.5)
>>> x = lucid.randn(2, 16)
>>> out = m(x)
>>> out.shape
(2, 16)

Methods (3)

dunder

init

→None

__init__(alpha: float = 1.0, inplace: bool = False)

source

Initialise the ELU module. See the class docstring for parameter semantics.

forward

→Tensor

forward(x: Tensor)

source

Apply the activation function element-wise.

Parameters

inputTensor

Input tensor of arbitrary shape.

Returns

Tensor

Output tensor of the same shape as input.

extra_repr

→str

extra_repr()

source

Return a string representation of the layer's configuration.