Linear

Quantized linear (fully-connected) layer — int8 weight, float compute.

The inference-time replacement that lucid.quantization.convert installs in place of a calibrated float lucid.nn.Linear. It is the quantized workhorse of every fully-connected stack — classifier heads, Transformer MLP blocks, projection layers — and the layer whose weight memory dominates most language models.

Representation (sidecar design B). The learned float weight is quantized once, at from_float time, into an int8 code tensor plus a per-output-channel scale / zero_point; the float weight is then dropped. Only the int8 codes, the qparams, and the (still-float) bias live in the module, so the checkpoint is roughly 4x smaller than the float layer's. Each forward dequantizes the weight back to float, runs the ordinary F.linear, and finally fake-quantizes the output onto the activation grid that calibration observed. Keeping the matmul itself in float means the result matches a genuine int8 kernel to within a rounding step and the layer runs unchanged on any device, with no int8 GEMM kernel required.

Per-channel weights. The weight (out_features, in_features) is quantized independently for every output channel (axis 0): each row carries its own scale and zero-point. Because different output neurons often span very different dynamic ranges, per-channel quantization tracks them far more tightly than a single per-tensor scale, and is the standard accuracy-preserving choice for linear and convolution kernels. The bias is small and precision-sensitive, so it is left in float.

Encoding happens once (at from_float); decode + matmul run on every forward:

where $s_i, z_i$ are the per-output-channel weight scale / zero-point and $S, Z$ the scalar calibrated output (scale, zero_point) with grid bounds $q_{\min}, q_{\max}$ (0, 255 for the default quint8). The decode → float-matmul → re-quantize round-trip is what makes the output bit- equivalent to a true int8 kernel without needing one.