class

Hardswish

extendsModule

Hardswish()

source

Hard Swish activation function.

Applies element-wise:

\text{Hardswish}(x) = x \cdot \frac{\text{ReLU6}(x + 3)}{6}

which is equivalent to:

\text{Hardswish}(x) = \begin{cases} 0 & \text{if } x \leq -3 \\ x(x+3)/6 & \text{if } -3 < x < 3 \\ x & \text{if } x \geq 3 \end{cases}

Hard Swish approximates the SiLU (Swish) activation using only integer arithmetic-friendly piecewise-linear operations, making it particularly efficient on hardware without native sigmoid support (e.g. mobile NPUs). It was introduced in MobileNetV3.

Notes

Input: $(*)$ — any shape.
Output: $(*)$ — same shape as input.

Examples

>>> import lucid
>>> import lucid.nn as nn
>>> m = nn.Hardswish()
>>> x = lucid.tensor([-4.0, -1.5, 0.0, 1.5, 4.0])
>>> m(x)
tensor([0.    , -0.375,  0.    ,  1.125,  4.    ])
>>> # Efficient mobile backbone activation
>>> x = lucid.randn(1, 96, 28, 28)
>>> out = m(x)
>>> out.shape
(1, 96, 28, 28)

Methods (1)

forward

→Tensor

forward(x: Tensor)

source

Apply the activation function element-wise.

Parameters

inputTensor

Input tensor of arbitrary shape.

Returns

Tensor

Output tensor of the same shape as input.