RealNVP

RealNVP takes the coupling layer of NICE and makes it scale as well as shift, which is what finally let exact-likelihood flows model natural images. An affine coupling layer splits the input with a binary mask $b$ and rescales the unmasked half by a learned, input-dependent factor:

The Jacobian is still triangular, so its log-determinant is just $\sum (1 - b) \odot s$ — no matter how deep $s$ and $t$ are. Unlike NICE's additive coupling this map is no longer volume-preserving, so the flow can concentrate and dilate density where the data needs it rather than relying on one global diagonal scaling.

Two masks exploit image structure. The checkerboard mask is 1 where the sum of the spatial coordinates is odd; the channel-wise mask is 1 on the first half of the channels. Between the two, a squeeze trades space for depth — each $2 \times 2 \times c$ block becomes $1 \times 1 \times 4c$, turning an $s \times s \times c$ tensor into $\tfrac{s}{2} \times \tfrac{s}{2} \times 4c$ — so that channel-wise masking after the squeeze partitions along a different axis than the checkerboard did before it.

The third idea is the multi-scale architecture. Each scale runs three alternating checkerboard couplings, squeezes, runs three alternating channel-wise couplings, then factors out half the dimensions:

Factored-out dimensions are modelled as Gaussians immediately and stop paying compute, so the network spends its depth on the coarse-scale structure that still needs it; the final scale applies four alternating checkerboard couplings and emits the rest. Coupling networks are residual convolutional stacks (ReLU, skip connections) whose $s$ output is a $\tanh$ scaled by a learned factor — bounded, so the exponential cannot explode early in training.

Because pixel data is bounded and quantised, the model is fitted in logit space: $x \mapsto \mathrm{logit}\big(\alpha + (1 - 2 \alpha) x\big)$ with $\alpha = 0.05$, whose log-determinant is part of the reported likelihood. RealNVP reaches 3.49 bits/dim on CIFAR-10 with exact inference and exact single-pass sampling — neither of which a VAE or an autoregressive model gives you — and its coupling/multi-scale recipe is inherited almost unchanged by Glow.