scaled_noise

→Tensor

scaled_noise(x: Tensor)

source edit

The factorised-noise transform of Fortunato et al. (2017).

f(x) = \mathrm{sign}(x)\sqrt{|x|}

Parameters

xTensor

Unit Gaussian samples, any shape.

Returns

Tensor

The same shape, magnitude-compressed with the sign kept.

Notes

Written out because noisy_linear applies it to both noise vectors and the paper is explicit that the bias uses it too: "for the bias we could have set $f(x) = x$ , but we decided to keep the same output noise for weights and biases."

Composed rather than fused, matching lucid.nn.functional.symlog — the same $\mathrm{sign}(x)\,g(|x|)$ shape, and the same decision.

Examples

>>> import lucid
>>> from lucid.nn.functional import scaled_noise
>>> scaled_noise(lucid.tensor([-4.0, 0.0, 9.0]))
tensor([-2., 0., 3.])

Used by 1

lucid.nn.functional