Dynamically quantize a model — int8 weights, runtime-quantized activations.

quantize_dynamic is the whole dynamic-PTQ workflow in one call: unlike the static prepare → calibrate → convert flow it needs no calibration data and no observers. Every module whose type is in qconfig_spec (default {Linear, LSTM}) is swapped in place for its dynamic counterpart, which stores its weight as int8 codes but measures the activation range and quantizes activations per forward, at runtime. That makes it the go-to for models where activation statistics vary widely between inputs and a fixed calibrated range would be a poor fit — most notably the Linear-heavy stacks of Transformers and RNN language models, where the weight memory dominates and per-call activation quantization costs little.

Because activation qparams are recomputed each call, dynamic quantization captures the 4x weight-memory win without an offline calibration pass, at the cost of a small per-forward quantize overhead. Prefer static PTQ (prepare / convert) when you can calibrate and want the lowest inference latency; prefer this when calibration is impractical or the stack is Linear-only.

When lucid.backends.quantized.use_mlx() is true, the Linear family is routed to the real weight-only quantized_matmul MLX GEMM (the genuine speed + memory win, W8A16 numerics), falling back to the dequantize path for layers whose in_features is not group-size-divisible.