class

CELU

extendsModule

CELU(alpha: float = 1.0, inplace: bool = False)

source edit

Continuously Differentiable Exponential Linear Unit activation function.

Applies element-wise:

\text{CELU}(x) = \max(0,\, x) + \min\!\left(0,\; \alpha\!\left(e^{x/\alpha} - 1\right)\right)

CELU differs from ELU in that it is continuously differentiable everywhere, including at zero, by ensuring the left and right derivatives match at $x = 0$ . Both branches coincide with an exponential scaled by $\alpha$ , and the output transitions smoothly from the negative saturating region to the linear positive branch.

Parameters

alphafloat= 1.0

Scale

\alpha > 0

controlling the slope of the negative branch and the value to which it saturates. Default: 1.0.

inplacebool= False

Accepted for API compatibility; currently unused. Default: False.

Notes

Input: $(*)$ — any shape.
Output: $(*)$ — same shape as input.

Examples

>>> import lucid
>>> import lucid.nn as nn
>>> m = nn.CELU(alpha=1.0)
>>> x = lucid.tensor([-2.0, -1.0, 0.0, 1.0, 2.0])
>>> m(x)
tensor([-0.8647, -0.6321,  0.    ,  1.    ,  2.    ])
>>> # Smoother negative branch than ELU for gradient-sensitive architectures
>>> m = nn.CELU(alpha=0.5)
>>> x = lucid.randn(4, 128)
>>> out = m(x)
>>> out.shape
(4, 128)

Used by 1

lucid.nn.modules

Constructors

dunder

init

→None

__init__(alpha: float = 1.0, inplace: bool = False)

source edit

Initialise the CELU module. See the class docstring for parameter semantics.

Instance methods

extra_repr

→str

extra_repr()

source edit

Return a string representation of the layer's configuration.

forward

→Tensor

forward(x: Tensor)

source edit

Apply the activation function element-wise.

Parameters

inputTensor

Input tensor of arbitrary shape.

Returns

Tensor

Output tensor of the same shape as input.

>>> import lucid >>> import lucid.nn as nn >>> m = nn.CELU(alpha=1.0) >>> x = lucid.tensor([-2.0, -1.0, 0.0, 1.0, 2.0]) >>> m(x) tensor([-0.8647, -0.6321, 0. , 1. , 2. ]) >>> # Smoother negative branch than ELU for gradient-sensitive architectures >>> m = nn.CELU(alpha=0.5) >>> x = lucid.randn(4, 128) >>> out = m(x) >>> out.shape (4, 128)