hardtanh

→Tensor

hardtanh(x: Tensor, min_val: float = -1.0, max_val: float = 1.0, inplace: bool = False)

source

Hardtanh — element-wise clamp to $[\text{min\_val}, \text{max\_val}]$ .

A cheap piecewise-linear surrogate for tanh whose forward and backward passes require no transcendental functions. Standard activation in compact / quantised RNN cells.

Parameters

xTensor

Input tensor of any shape.

min_valfloat= -1.0

Lower clamp. Default -1.0.

max_valfloat= 1.0

Upper clamp. Default 1.0.

inplacebool= False

Accepted for API compatibility; currently ignored.

Returns

Tensor

Clamped tensor with the same shape as x.

Notes

\text{HardTanh}(x) = \begin{cases} \text{max\_val} & x > \text{max\_val} \\ x & \text{min\_val} \le x \le \text{max\_val} \\ \text{min\_val} & x < \text{min\_val} \end{cases}

Derivative is $1$ inside the linear region and $0$ in the saturated regions — gradients stop flowing through saturated units, a drawback compared to smooth tanh but acceptable when the linear region covers the bulk of activations.

Examples

>>> import lucid
>>> from lucid.nn.functional import hardtanh
>>> x = lucid.tensor([-2.0, -0.5, 0.0, 0.5, 2.0])
>>> hardtanh(x)
Tensor([-1.0000, -0.5000,  0.0000,  0.5000,  1.0000])