module

utils

17 members
lucid.nn.utils

lucid.nn.utils: utility functions for neural networks.

Classes

Functions

fnclip_grad_norm_Tensor

Clip the global gradient norm of `parameters` in place.

fnclip_grad_value_None

Clamp every gradient element to $[-\text{clip\_value}, \text{clip\_value}]$ in place.

fnget_total_normTensor

Compute the global gradient $\ell_p$ norm without clipping.

fnparameters_to_vectorTensor

Flatten an iterable of parameters into a single 1-D `Tensor`.

fnvector_to_parametersNone

Copy a flat 1-D tensor back into the supplied parameters in place.

fnremove_weight_normModule

Reverse `weight_norm` and restore a plain leaf parameter.

fnremove_spectral_normModule

Reverse `spectral_norm` and restore a plain leaf parameter.

fnpack_padded_sequencePackedSequence

Pack a padded $(T, B, *)$ batch into a `PackedSequence`.

fnpack_sequencePackedSequence

One-shot pack-from-list — equivalent to `pad_sequence` then `pack_padded_sequence`.

fnpad_packed_sequenceTensor

Inverse of `pack_padded_sequence` — produce a padded tensor.

fnpad_sequenceTensor

Stack a list of variable-length tensors into a padded batch.

fncopy_parameters_and_buffersNone

Copy all parameter / buffer values from `source` into `dest`.

fnfuse_conv_bn_evalModule

Fold a BatchNorm layer into the preceding Conv weights (inference-only).

fnfuse_conv_bn_weights(Tensor, Tensor)

Low-level form of `fuse_conv_bn_eval` operating on raw tensors.

fnfuse_linear_bn_evalModule

Fold a BatchNorm1d into the preceding Linear weights (inference-only).

fnfuse_linear_bn_weights(Tensor, Tensor)

Low-level form of `fuse_linear_bn_eval` operating on raw tensors.