instance_norm

→Tensor

instance_norm(x: Tensor, running_mean: Tensor | None = None, running_var: Tensor | None = None, weight: Tensor | None = None, bias: Tensor | None = None, use_input_stats: bool = True, momentum: float = 0.1, eps: float = 1e-05)

source

Instance normalization (Ulyanov, Vedaldi & Lempitsky, 2016).

Normalises each (sample, channel) slice against its own spatial mean and variance — i.e. statistics are reduced only over the spatial axes. Originally introduced for fast neural style transfer; widely used wherever per-image contrast should be made invariant (image generation, image-to-image translation).

Parameters

xTensor

Input of shape (N, C, *spatial) with ndim >= 3.

running_meanTensor or None= None

Running per-channel mean of shape (C,). Only consulted when use_input_stats=False.

running_varTensor or None= None

Running per-channel variance of shape (C,).

weightTensor= None

Per-channel scale

\gamma

of shape (C,).

biasTensor= None

Per-channel shift

\beta

of shape (C,).

use_input_statsbool= True

When True (default) statistics come from the current instance. When False and running buffers are supplied, those are used instead.

momentumfloat= 0.1

EMA coefficient for the running buffers (when externally updated).

epsfloat= 1e-05

Numerical safety added inside the square root.

Returns

Tensor

Same shape as x.

Notes

Math (reduction taken only over spatial axes $S$ ):

\begin{aligned} \mu_{n,c} &= \frac{1}{|S|}\sum_{s \in S} x_{n,c,s} \\ \sigma^2_{n,c} &= \frac{1}{|S|}\sum_{s \in S} (x_{n,c,s} - \mu_{n,c})^2 \\ y_{n,c,s} &= \gamma_c \cdot \frac{x_{n,c,s} - \mu_{n,c}}{\sqrt{\sigma^2_{n,c} + \epsilon}} + \beta_c \end{aligned}

InstanceNorm equals group_norm with num_groups == C. The lack of batch coupling also makes it the natural normaliser when adapting per-sample statistics matters more than population coherence — exactly the case in style transfer.

Examples

>>> import lucid
>>> from lucid.nn.functional import instance_norm
>>> x = lucid.randn(4, 3, 64, 64)
>>> y = instance_norm(x)
>>> y.shape
(4, 3, 64, 64)