class

LeakyReLU

extends_QuantizedActivation

LeakyReLU(scale: Tensor, zero_point: Tensor, qdtype: QDtype = quint8)

source edit

Quantized lucid.nn.LeakyReLU — leaky_relu(x, slope) then fake-quant.

The inference-time replacement for a calibrated float nn.LeakyReLU, installed by lucid.quantization.convert / from_float. Under the sidecar representation (design B) the quantized layer is the float leaky-ReLU followed by a fake-quantize to the calibrated output grid. Like ELU its range is unbounded on both sides (linear far from the origin), so it is calibrated with a real observer, not a lucid.quantization.FixedQParamsObserver.

The negative_slope — the small gradient applied to $x < 0$ that keeps dead-unit gradients alive — is a parameter of the activation; from_float copies it from the source nn.LeakyReLU (via the internal _configure hook) so the quantized curve matches the float one before the output is snapped to the grid.

y = \operatorname{fake\_quant}\bigl(\operatorname{leaky\_relu}(x)\bigr), \qquad \operatorname{leaky\_relu}(x) = \begin{cases} x & x \ge 0 \\ \text{slope}\cdot x & x < 0 \end{cases}

with $\operatorname{fake\_quant}(t) = (\operatorname{clamp}( \operatorname{round}(t/S) + Z, q_{\min}, q_{\max}) - Z)\, S$ and the calibrated output (scale, zero_point) $S, Z$ .

Parameters

scaleTensor

Per-tensor output scale from the calibrating observer. Supplied by from_float; construct directly only with qparams in hand.

zero_pointTensor

Per-tensor output zero-point from the calibrating observer.

qdtypeQDtype= quint8

Quantized dtype bounding the grid. Defaults to lucid.quantization.quint8 ([0, 255]).

Attributes

negative_slopefloat

The slope applied to

x < 0

, copied from the source nn.LeakyReLU by from_float. Defaults to 0.01 on a bare instance.

Notes

Built via convert / from_float, which reads the output observer and copies negative_slope from the float module.
Range is unbounded on both sides, so calibrate through the prepared model before convert rather than using a fixed grid.
The forward is float leaky_relu(x, slope) → fake-quant; only the output is snapped to the grid.

Examples

>>> import lucid, lucid.nn as nn
>>> import lucid.quantization as Q
>>> model = nn.Sequential(nn.LeakyReLU(negative_slope=0.2))
>>> model.qconfig = Q.get_default_qconfig()
>>> prepared = Q.prepare(model)               # attach the output observer
>>> _ = prepared(lucid.randn(32, 8))          # calibrate its range
>>> qlr = Q.convert(prepared)[0]              # from_float reads the observer
>>> type(qlr).__name__, qlr.negative_slope
('LeakyReLU', 0.2)

Used by 1

lucid.nn.quantized

>>> import lucid, lucid.nn as nn >>> import lucid.quantization as Q >>> model = nn.Sequential(nn.LeakyReLU(negative_slope=0.2)) >>> model.qconfig = Q.get_default_qconfig() >>> prepared = Q.prepare(model) # attach the output observer >>> _ = prepared(lucid.randn(32, 8)) # calibrate its range >>> qlr = Q.convert(prepared)[0] # from_float reads the observer >>> type(qlr).__name__, qlr.negative_slope ('LeakyReLU', 0.2)

LeakyReLU

Parameters

Attributes

Notes

Examples

See Also

Used by 1

LeakyReLU

Parameters

Attributes

Notes

Examples

See Also

Used by 1