class

SiLU

extendsModule

SiLU(inplace: bool = False)

source

Sigmoid Linear Unit (Swish) activation function.

Applies element-wise:

\text{SiLU}(x) = x \cdot \sigma(x) = \frac{x}{1 + e^{-x}}

where $\sigma$ is the logistic sigmoid function. The gate $\sigma(x)$ smoothly interpolates between 0 (deep negative) and the identity (large positive), producing a non-monotone shape with a small dip below zero near $x \approx -1.3$ .

Parameters

inplacebool= False

Accepted for API compatibility; currently unused. Default: False.

Notes

Input: $(*)$ — any shape.
Output: $(*)$ — same shape as input.

SiLU / Swish is used in EfficientNet, MobileNetV3, and many modern vision backbones as a smooth, self-gated alternative to ReLU. It is differentiable everywhere, including at zero.

Examples

>>> import lucid
>>> import lucid.nn as nn
>>> m = nn.SiLU()
>>> x = lucid.tensor([-2.0, -1.0, 0.0, 1.0, 2.0])
>>> m(x)
tensor([-0.2384, -0.2689,  0.    ,  0.7311,  1.7616])
>>> # Common backbone activation — shape-preserving
>>> x = lucid.randn(8, 256)
>>> out = nn.SiLU()(x)
>>> out.shape
(8, 256)

Methods (2)

dunder

init

→None

__init__(inplace: bool = False)

source

Initialise the SiLU module. See the class docstring for parameter semantics.

forward

→Tensor

forward(x: Tensor)

source

Apply the activation function element-wise.

Parameters

inputTensor

Input tensor of arbitrary shape.

Returns

Tensor

Output tensor of the same shape as input.