get_total_norm

→Tensor

get_total_norm(parameters: Iterable[Parameter], norm_type: float = 2.0, error_if_nonfinite: bool = False, foreach: bool | None = None)

source

Compute the global gradient $\ell_p$ norm without clipping.

Mirrors the measurement step of clip_grad_norm_ but returns the magnitude without rescaling any gradients — handy for logging the gradient size every step regardless of whether you intend to clip.

Parameters

parametersiterable of Parameter

Parameters whose gradients participate in the norm computation. Entries with grad is None are skipped; the empty case returns a zero tensor.

norm_typefloat= 2.0

Order

p

of the norm. Default 2.0; use math.inf for the max-norm.

error_if_nonfinitebool= False

Raise on inf / nan results instead of returning them.

foreachbool= None

Accepted for API compatibility; ignored — Lucid processes the parameters sequentially through the C++ engine.

Returns

Tensor

Scalar tensor with the combined gradient norm.

Notes

The returned scalar is

\|g\|_p \;=\; \left(\sum_i |g_i|^p\right)^{1/p}

taken over every element of every gradient — i.e. the same quantity that clip_grad_norm_ thresholds against max_norm.

Examples

>>> from lucid.nn.utils.clip_grad import get_total_norm
>>> g_norm = get_total_norm(model.parameters())