Linear
LinearLinear(in_features: int, out_features: int, bias: bool = True, qconfig: QConfig | None = None)Implementing kernel
C++ engine symbols that back this Python API.Quantization-aware Linear — trainable float weight, fake-quant every forward.
The training-time stand-in that lucid.quantization.prepare_qat installs in
place of a float lucid.nn.Linear. It keeps a trainable float weight but
routes both the weight and the layer output through a
lucid.quantization.FakeQuantize on every forward, so the network feels the
int8 rounding error while it is still learning and can adapt its weights to
compensate. This is what closes the train / inference accuracy gap: plain
post-training quantization perturbs a network that never saw rounding, whereas a
QAT-trained network has already minimised its loss with the rounding baked in, so its
accuracy barely moves when the weights are finally frozen to int8.
Straight-through estimator (STE). Rounding is a step function; its derivative is
zero almost everywhere, so a literal round would block all gradient flow. The
fake-quant therefore rounds in the forward pass but, in the backward pass, pretends it
was the identity — the incoming gradient passes straight through to the underlying
float weight (clipped to the quantization range). The float weight thus stays fully
trainable while every forward value it produces is the dequantized number an int8
kernel would have computed.
Where it sits. lucid.quantization.prepare_qat deep-copies the float model
and swaps each lucid.nn.Linear for this class, attaching the weight and
activation FakeQuantize modules described by the layer's qconfig. After
training, lucid.quantization.convert reads the trained weight together with the
activation observer's final (scale, zero_point) and bakes them into an inference
lucid.nn.quantized.Linear whose weight is stored as true int8 codes.
On each forward the weight is fake-quantized, the ordinary F.linear runs, and the
result is fake-quantized onto the observed activation grid:
where are the scale / zero-point the enclosing FakeQuantize derives
from its observer and are the grid bounds (0, 255 for
the default quint8 activation, -128, 127 for the qint8 weight). The
straight-through unit derivative is what keeps the layer differentiable despite the
non-differentiable round.
Parameters
in_featuresintlucid.nn.Linear parent.out_featuresintbiasbool= Truelucid.quantization.FakeQuantize modules applied during training.
Required — constructing the layer without one raises ValueError.Attributes
weight_fake_quantFakeQuantizeqconfig.weight(); rounds the float
weight each forward and tracks its range so convert can pick the int8 qparams.activation_post_processFakeQuantizeqconfig.activation(); calibrates
the activation grid whose (scale, zero_point) convert later bakes in.Notes
- STE differentiability.
roundis applied in the forward pass but the gradient is passed through as if it were the identity, so the float weight trains normally — no gradient is lost to the non-differentiable rounding step. - Both directions are wired for you.
lucid.quantization.prepare_qatswaps the float layer in;lucid.quantization.convertfolds this layer out into the matchinglucid.nn.quantized.Linear. Manual construction is rarely needed and requires an explicitqconfig. - Training is slower, not faster. Only the numerics of int8 are simulated; the
matmul itself still runs in float and each forward carries two extra fake-quant ops,
so a QAT model runs slower than the float baseline. The speed / memory win arrives
only after
convertreplaces this layer with the int8 inference module.
Examples
>>> import lucid, lucid.nn as nn
>>> import lucid.quantization as Q
>>> model = nn.Sequential(nn.Linear(16, 4))
>>> qat = Q.prepare_qat(model) # nn.Linear -> qat.Linear
>>> type(qat[0]).__name__
'Linear'
>>> loss = (qat(lucid.randn(8, 16)) ** 2).mean()
>>> loss.backward() # STE routes grads to the float weight
>>> qat.eval()
>>> qmodel = Q.convert(qat) # qat.Linear -> quantized.Linear (int8)
>>> type(qmodel[0]).__name__
'Linear'
A qconfig is mandatory — constructing the layer directly without one raises:
>>> import lucid.nn.qat as nnqat
>>> nnqat.Linear(8, 8)
Traceback (most recent call last):
...
ValueError: qat.Linear requires a qconfigSee Also
- lucid.nn.quantized.Linear—The int8 inference layer
convertbakes this into. - lucid.nn.qat.LinearReLU—QAT
Linearwith a fused ReLU before the output observer. - lucid.quantization.prepare_qat—Swaps the float
Linearfor this QAT layer. - lucid.quantization.convert—Bakes the trained QAT layer into the quantized layer.
Used by 1
Constructors
1__init__(in_features: int, out_features: int, bias: bool = True, qconfig: QConfig | None = None)Construct a Tensor from Python data, a NumPy array, or another Tensor.
The data is funnelled through lucid._factories.converters._to_impl,
the canonical bridge between the outside world and Lucid's C++
TensorImpl. Python scalars become 0-d tensors; nested lists are
recursively flattened with shape inference; NumPy arrays cross the
single sanctioned host-to-engine boundary; and existing Tensor
sources are rewrapped (optionally with cast and/or device transfer).
Parameters
datandarray | list | int | float | bool | Tensorlucid._factories.converters; another Tensor
is shallow-copied to a new Tensor with possibly different
dtype / device / requires_grad.dtypedtype | str | NoneNone, inferred from data.devicedevice | str | None"cpu" or "metal"). If None, defaults
to the source device or "cpu" for host data.requires_gradboolReturns
TensorA freshly constructed tensor whose storage is owned by the engine.
Notes
Constructor is one of the six sanctioned host-to-engine bridge boundaries (rule H4). NumPy arrays cross the bridge exactly once here; afterwards the data is owned by Lucid's engine. The resulting layout is C-contiguous row-major:
where is the element size in bytes and are the dimension sizes.
Examples
>>> import lucid
>>> lucid.Tensor([1.0, 2.0, 3.0])
tensor([1., 2., 3.])
>>> lucid.Tensor([[1, 2], [3, 4]], dtype=lucid.int64).shape
(2, 2)
>>> x = lucid.Tensor(3.14, requires_grad=True)
>>> x.requires_grad
True