normal_

→Tensor

normal_(tensor: Tensor, mean: float = 0.0, std: float = 1.0)

source

Initialise tensor in-place with samples from a Gaussian distribution.

Each entry is drawn independently from $\mathcal{N}(\text{mean}, \text{std}^2)$ . This is the simplest initialisation scheme and serves as the building block for xavier_normal_ / kaiming_normal_, which choose std so that the activation variance is preserved layer-by-layer.

Parameters

tensorTensor

Tensor to fill in place; any shape is accepted.

meanfloat= 0.0

Mean of the Gaussian distribution. Default 0.0.

stdfloat= 1.0

Standard deviation of the Gaussian distribution. Must be non-negative. Default 1.0.

Returns

Tensor

tensor (mutated) for chaining.

Notes

The samples have

\mathbb{E}[W] = \text{mean}, \qquad \mathrm{Var}(W) = \text{std}^2.

For unbounded tails consider trunc_normal_ instead, which truncates extreme draws and is often preferred in transformer pre-training recipes.

Examples

>>> import lucid
>>> from lucid.nn.init import normal_
>>> w = lucid.empty(64, 32)
>>> normal_(w, mean=0.0, std=0.02)