class

ReLU6

extendsModule

ReLU6(inplace: bool = False)

source

ReLU6 activation function.

Applies element-wise:

\text{ReLU6}(x) = \min\!\bigl(\max(0,\, x),\, 6\bigr)

A capped variant of ReLU that clamps activations to the range $[0, 6]$ . The hard cap prevents activations from growing unboundedly, which can improve robustness of fixed-point quantisation (8-bit or lower) by keeping the dynamic range bounded. Widely used in mobile architectures such as MobileNetV1 and MobileNetV2.

Parameters

inplacebool= False

Accepted for API compatibility; currently unused. Default: False.

Notes

Input: $(*)$ — any shape.
Output: $(*)$ — same shape as input, values in $[0, 6]$ .

Examples

>>> import lucid
>>> import lucid.nn as nn
>>> m = nn.ReLU6()
>>> x = lucid.tensor([-1.0, 0.0, 3.0, 6.0, 10.0])
>>> m(x)
tensor([0., 0., 3., 6., 6.])
>>> # Quantisation-friendly activation in depthwise-separable convolutions
>>> x = lucid.randn(4, 32, 14, 14)
>>> out = m(x)
>>> out.shape
(4, 32, 14, 14)

Methods (2)

dunder

init

→None

__init__(inplace: bool = False)

source

Initialise the ReLU6 module. See the class docstring for parameter semantics.

forward

→Tensor

forward(x: Tensor)

source

Apply the activation function element-wise.

Parameters

inputTensor

Input tensor of arbitrary shape.

Returns

Tensor

Output tensor of the same shape as input.