Float-domain quantizable merge ops whose output prepare observes.
Element-wise merges — residual skip-adds, gated multiplies, concatenations —
carry no learnable weight, so the module-swap machinery that quantizes
Linear / Conv never sees a bare x + y or lucid.cat. Yet a
residual network cannot be quantized end-to-end unless the merge result is
observed and later requantized to a consistent grid: otherwise the two
branches of an add arrive on different scales and the sum leaks precision.
FloatFunctional is the fix — a stateless module you call in place of the
raw op (ff.add(x, y) instead of x + y) so the merge gets a named site
the tooling can attach an observer to.
It is the calibration-time half of the pair. Until
lucid.quantization.prepare installs an activation_post_process
observer, every method is a plain float op routed through the _obs
passthrough — the numerics are identical to writing the operator by hand.
Once prepared, each call additionally feeds its result to the observer, which
records the merged activation's dynamic range. lucid.quantization.convert
then swaps the whole module for a QFunctional carrying those qparams.
The exposed merges are add (residual skip-add), mul, add_relu
(the fused add-then-ReLU of a ResNet block), cat (concatenation along a
dim), and the scalar variants add_scalar / mul_scalar.
Parameters
Nonenn.quantized.FloatFunctional())
and placed inside a float model. It gains its observer from prepare
rather than from constructor arguments, and is replaced — not converted
in place — by QFunctional at convert time.Notes
- Reuse one
FloatFunctionalper merge site; do not share a single instance across unrelated adds, since they would then share one observer and be forced onto a common, looser grid. - Before
prepareit is a transparent passthrough, so inserting it into a float model never changes float-mode numerics — safe to add pre-emptively. - Only the output is observed; the two operands keep whatever grids their producing layers assigned.
Examples
Residual skip-add inside a block — call add instead of +:
>>> import lucid.nn as nn
>>> class Block(nn.Module):
... def __init__(self) -> None:
... super().__init__()
... self.conv = nn.Conv2d(8, 8, 3, padding=1)
... self.skip = nn.quantized.FloatFunctional()
... def forward(self, x):
... return self.skip.add(self.conv(x), x) # not `self.conv(x) + x`
The common mistake is leaving the raw operator in place — the sum then has no
observer and cannot be requantized to a single grid:
>>> ff = nn.quantized.FloatFunctional()
>>> import lucid
>>> a, b = lucid.randn(4), lucid.randn(4)
>>> bool((ff.add(a, b) == a + b).all().item()) # identity pre-prepare
TrueSee Also
- lucid.nn.quantized.QFunctional—The converted, requantizing counterpart.
- lucid.quantization.prepare—Attaches the observer these merges feed.
- lucid.quantization.convert—Swaps this module for
QFunctional.
Used by 1
Instance methods
6Observed x + y (a residual skip-add).
Observed relu(x + y) — the fused residual-add of a ResNet block.
Observed x + scalar.
Observed concatenation along dim.
Observed x * y.
Observed x * scalar.