RealNVPConfig

Configuration for every RealNVP variant.

Defaults reproduce the paper's Imagenet 32 x 32 setup (§4.1): four scales, four residual blocks per coupling network, 32 feature maps at the first scale, and a standard normal prior.

Args: num_scales: Levels of the multi-scale recursion L. Every level except the last squeezes once and factors out half of its dimensions, so the spatial size halves L - 1 times. The paper recurses "until the input of the last recursion is 4 x 4 x c" — four scales for a 32 x 32 input — and uses a single downscale for CIFAR-10. residual_blocks: Residual blocks inside each coupling network. Paper §4.1: 4 at 32 x 32, 2 at 64 x 64, 8 for CIFAR-10. base_dim: Feature maps in the first scale's coupling networks; doubled at every subsequent scale, as in the released implementation. Paper §4.1: 32, or 64 for CIFAR-10. use_weight_norm: Apply weight normalisation to every convolution inside the coupling networks (paper §4.1). use_batch_norm: Enable batch normalisation — inside the coupling networks and on each coupling layer's output, where it is an invertible rescaling that contributes to the log-determinant (paper §3.7). Inversion uses the running statistics, so decode is exact in eval() mode. batch_norm_running_stats_in_training: Normalise by the running average during training rather than by the current batch — the paper's "novel variant of batch normalization ... based on a running average over recent minibatches, and is thus more robust when training with very small minibatches" (§3.7 / Appendix E). Off by default: the two agree in the large-batch limit where most training happens, and turning it on changes the behaviour of runs already in flight. data_constraint: Width of the logit squash applied before the flow, matching the released implementation's data_constraint: x in [0, 1] is mapped into [(1 - c) / 2, (1 + c) / 2] before the logit, i.e. alpha = 0.05 at the default 0.9. num_bits: Bit depth of the discrete data the reported likelihood is measured against. The flow is fitted to x in [0, 1], but the paper models logit(alpha + (1 - alpha) * x / 256) and reports bits/dim on the 8-bit pixel scale, so the change of variables from {0, ..., 255} back to [0, 1] contributes num_bits to the metric. Table 1 is not comparable without it.

Notes: out_channels must equal in_channels — a flow is a bijection on its sample space. sample_size must be divisible by 2 ** (num_scales - 1) so every squeeze has an even spatial extent to work on.