LeakyReLU
_QuantizedActivationLeakyReLU(scale: Tensor, zero_point: Tensor, qdtype: QDtype = quint8)Quantized lucid.nn.LeakyReLU — leaky_relu(x, slope) then fake-quant.
The inference-time replacement for a calibrated float nn.LeakyReLU,
installed by lucid.quantization.convert / from_float. Under
the sidecar representation (design B) the quantized layer is the float
leaky-ReLU followed by a fake-quantize to the calibrated output grid. Like
ELU its range is unbounded on both sides (linear far from the
origin), so it is calibrated with a real observer, not a
lucid.quantization.FixedQParamsObserver.
The negative_slope — the small gradient applied to that
keeps dead-unit gradients alive — is a parameter of the activation;
from_float copies it from the source nn.LeakyReLU (via the
internal _configure hook) so the quantized curve matches the float one
before the output is snapped to the grid.
with and the
calibrated output (scale, zero_point) .
Parameters
Attributes
negative_slopefloatnn.LeakyReLU by from_float. Defaults to 0.01 on a bare
instance.Notes
- Built via
convert/from_float, which reads the output observer and copiesnegative_slopefrom the float module. - Range is unbounded on both sides, so calibrate through the prepared model
before
convertrather than using a fixed grid. - The forward is
float leaky_relu(x, slope) → fake-quant; only the output is snapped to the grid.
Examples
>>> import lucid.nn as nn
>>> src = nn.LeakyReLU(negative_slope=0.2)
>>> qlr = nn.quantized.LeakyReLU.from_float(src) # (needs a calibrated obs.)
>>> qlr.negative_slope
0.2See Also
- lucid.nn.quantized.ELU—Also carries a shape parameter across convert.
- lucid.nn.quantized.ConvReLU2d—Fused ReLU folded into a quantized conv.
- lucid.quantization.convert—Installs this layer from a calibrated model.