class

Softshrink

extendsModule

Softshrink(lambd: float = 0.5)

source

Soft Shrinkage activation function.

Applies element-wise:

\text{Softshrink}(x) = \begin{cases} x - \lambda & \text{if } x > \lambda \\ x + \lambda & \text{if } x < -\lambda \\ 0 & \text{otherwise} \end{cases}

Shrinks all values towards zero by $\lambda$ : values outside the dead band are shifted, while values inside are zeroed. This is the proximal operator of the $\ell_1$ norm and appears in LASSO and iterative soft-thresholding algorithms (ISTA / FISTA).

Parameters

lambdfloat= 0.5

Threshold

\lambda \geq 0

. Default: 0.5.

Notes

Input: $(*)$ — any shape.
Output: $(*)$ — same shape as input.

Unlike Hardshrink, Softshrink shifts surviving values toward zero rather than preserving them exactly. This makes it a continuous function and gives it a well-defined subgradient everywhere.

Examples

>>> import lucid
>>> import lucid.nn as nn
>>> m = nn.Softshrink(lambd=0.5)
>>> x = lucid.tensor([-1.5, -0.3, 0.0, 0.3, 1.5])
>>> m(x)
tensor([-1. ,  0. ,  0. ,  0. ,  1. ])
>>> # L1-proximal layer in an unrolled ISTA network
>>> m = nn.Softshrink(lambd=0.1)
>>> x = lucid.randn(8, 128)
>>> out = m(x)
>>> out.shape
(8, 128)

Methods (3)

dunder

init

→None

__init__(lambd: float = 0.5)

source

Initialise the Softshrink module. See the class docstring for parameter semantics.

forward

→Tensor

forward(x: Tensor)

source

Apply the activation function element-wise.

Parameters

inputTensor

Input tensor of arbitrary shape.

Returns

Tensor

Output tensor of the same shape as input.

extra_repr

→str

extra_repr()

source

Return a string representation of the layer's configuration.