local_response_norm

→Tensor

local_response_norm(x: Tensor, size: int, alpha: float = 0.0001, beta: float = 0.75, k: float = 1.0)

source

Local response normalization (Krizhevsky, Sutskever & Hinton, 2012).

Implements lateral inhibition: each activation is divided by a function of the squared activations of its channel neighbours. Introduced with AlexNet and largely superseded by batch / layer normalisation, but still used in some classic architectures.

Parameters

xTensor

Input of shape (N, C, *spatial) (any number of spatial dims). Inputs with fewer than 2 dimensions are returned unchanged.

sizeint

Number of neighbouring channels (the size of the local window) used in the normalisation.

alphafloat= 0.0001

Multiplicative scaling factor on the squared-activation sum. Default 1e-4.

betafloat= 0.75

Exponent applied to the normalising factor. Default 0.75.

kfloat= 1.0

Additive constant guarding the division. Default 1.0.

Returns

Tensor

Same shape as x.

Notes

Math (the neighbourhood $\mathcal{N}(c)$ is the size channels centred at $c$ , zero-padded at boundaries):

y_{i,c,h,w} = \frac{x_{i,c,h,w}}{\left(k + \alpha \sum_{c' \in \mathcal{N}(c)} x_{i,c',h,w}^2 \right)^\beta}

Boundary handling uses asymmetric zero padding when size is even: (size - 1) // 2 on the left and size // 2 on the right, matching the original AlexNet / cuDNN convention. LRN predates BatchNorm and provides only lateral inhibition (no learnt affine), which is why modern architectures usually do not include it.

Examples

>>> import lucid
>>> from lucid.nn.functional import local_response_norm
>>> x = lucid.randn(1, 96, 27, 27)   # AlexNet conv1 output shape
>>> y = local_response_norm(x, size=5, alpha=1e-4, beta=0.75, k=2.0)
>>> y.shape
(1, 96, 27, 27)