noisy_linear(x: Tensor, weight_mu: Tensor, weight_sigma: Tensor, bias_mu: Tensor | None = None, bias_sigma: Tensor | None = None, epsilon_in: Tensor | None = None, epsilon_out: Tensor | None = None)A linear layer whose parameters carry learnable noise.
Parameters
(*, in_features).weight_muTensor(out_features, in_features) — the mean weight and the
scale of the noise on it.weight_sigmaTensor(out_features, in_features) — the mean weight and the
scale of the noise on it.(out_features,), or None for an unbiased layer.(out_features,), or None for an unbiased layer.(in_features,) unit Gaussian samples. None draws them.(out_features,) unit Gaussian samples. None draws them.Returns
Tensor(*, out_features).
Raises
ValueErrorbias_mu and bias_sigma is given.Notes
Reference: Fortunato, Azar, Piot, Menick, Hessel, Osband, Graves, Mnih, Munos, Hassabis, Pietquin, Blundell, and Legg, "Noisy Networks for Exploration", ICLR, 2018 (arXiv:1706.10295), equations 9-11.
Factorised noise, which is the variant the paper uses for the single-threaded agents: rather than one sample per weight, it draws for the inputs and for the outputs and forms
so a layer costs random numbers instead of . The paper's stated reason is compute time, and it is the reason to prefer this form here too.
The two noise vectors are arguments rather than always drawn, because a sample has to be held fixed across a whole forward pass — and, for an agent, often across a whole episode. Drawing inside the function would make every call a different network.
Gradients reach through the noise, which is the point: the scale of the exploration is learned rather than annealed.
Examples
>>> import lucid
>>> from lucid.nn.functional import noisy_linear
>>> x = lucid.zeros((2, 4))
>>> mu, sigma = lucid.zeros((3, 4)), lucid.ones((3, 4))
>>> noisy_linear(x, mu, sigma).shape
(2, 3)See Also
- lucid.nn.NoisyLinear—
nn.Modulewrapper that owns the parameters. linear—the deterministic layer this perturbs.