QDtype

Descriptor for a quantized integer grid — bits, signedness, range, storage.

Where a QScheme fixes the geometry of the affine map, a QDtype pins the target grid itself: how many bits the codes use, whether they are signed, the inclusive [quant_min, quant_max] range they span, and the ordinary Lucid dtype that physically stores them. Every observer and fake-quant reads quant_min / quant_max off a QDtype to clamp its codes.

Unlike the reference framework, Lucid does not add quantized dtypes to the C++ engine's dtype enum; a QDtype is a pure-Python descriptor and the integer codes live in an ordinary storage dtype. This keeps the whole quantization subsystem off the engine's critical path until a real low-precision GEMM is wired in, and lets 4-bit codes ride inside int8 until bit-packing lands.

Four instances are exported as module-level constants, spanning the grids:

• qint8 — 8-bit signed ([-128, 127]), stored in int8. The default weight grid: symmetric weights centre on 0, so a signed grid uses its full range. The workhorse of int8 inference.
• quint8 — 8-bit unsigned ([0, 255]), stored in int16 (the engine has no uint8, so codes ride in the next wider signed type). The default activation grid: post-ReLU activations are $\ge 0$, so an unsigned grid spends all 256 codes on the range that actually occurs.
• qint4 — 4-bit signed ([-8, 7]), stored (unpacked) in int8 until 4-bit packing lands. The aggressive weight grid for the real MLX GEMM: about 2x smaller than int8 for a modest accuracy cost, used for memory-bound decode.
• qint32 — 32-bit signed ([-2**31, 2**31 - 1]), stored in int32. The accumulator grid: int8 × int8 partial sums accumulate in int32 before requantization, so it rarely appears as a weight / activation dtype.