softshrink

→Tensor

softshrink(x: Tensor, lambd: float = 0.5)

source

Soft shrinkage operator — proximal operator of $\lambda \|\cdot\|_1$ .

Translates the input toward zero by $\lambda$ and zeros out everything in the dead zone $[-\lambda, \lambda]$ . Used in iterative-shrinkage solvers (ISTA / FISTA) and in sparse autoencoder decoders.

Parameters

xTensor

Input tensor.

lambdfloat= 0.5

Shrinkage threshold

\lambda \ge 0

. Default 0.5.

Returns

Tensor

Shrunk tensor with the same shape as x.

Notes

\text{SoftShrink}(x) = \begin{cases} x - \lambda & x > \lambda \\ x + \lambda & x < -\lambda \\ 0 & \text{otherwise} \end{cases}

Continuous everywhere (unlike hardshrink), with derivative $1$ outside the dead zone and $0$ inside it. Coincides with the closed-form solution of $\arg\min_y \tfrac12 (y - x)^2 + \lambda |y|$ .

Examples

>>> import lucid
>>> from lucid.nn.functional import softshrink
>>> x = lucid.tensor([-1.0, -0.3, 0.0, 0.3, 1.0])
>>> softshrink(x, lambd=0.5)
Tensor([-0.5000,  0.0000,  0.0000,  0.0000,  0.5000])