prelu

→Tensor

prelu(x: Tensor, weight: Tensor)

source

Parametric Rectified Linear Unit (He et al. 2015).

Variant of leaky_relu whose negative slope is a learnable parameter rather than a fixed hyperparameter. In practice weight may be a scalar (shared across channels) or a per-channel vector, matching the convention of the reference framework's nn.PReLU.

Parameters

xTensor

Input tensor.

weightTensor

Learnable slope

\alpha

. Either a scalar tensor (slope shared across all elements) or a 1-D tensor broadcastable against x along the channel dimension.

Returns

Tensor

Activated tensor with the same shape as x.

Notes

\text{PReLU}(x) = \max(0, x) + \alpha \min(0, x)

Allows the optimiser to discover the optimal negative-side slope per channel; the original paper showed this reaches super-human ImageNet accuracy. When $\alpha = 0$ recovers relu; for a fixed non-zero $\alpha$ see leaky_relu.

Examples

>>> import lucid
>>> from lucid.nn.functional import prelu
>>> x = lucid.tensor([-2.0, -1.0, 0.0, 1.0])
>>> w = lucid.tensor(0.25)
>>> prelu(x, w)
Tensor([-0.5000, -0.2500,  0.0000,  1.0000])