Python wrappers
Public Python APIs implemented by this engine symbols.Autograd node for alpha-dropout — element-wise dropout that preserves the self-normalising statistics required by SELU activations.
Dropped elements are not zeroed but replaced with the SELU
saturation value ;
an affine rescaling is then applied so
that the mean and variance of the post-dropout distribution match
those of the input. The chain rule therefore collapses to
, which is what
apply evaluates using the saved raw mask.
Attributes
schema_v1OpSchemaRegistered schema (name
"alpha_dropout", version 1, AmpPolicy::KeepInput).p_doubleDrop probability used by the forward.
a_coef_doubleAffine scale factor saved for the backward.
mask_StorageRaw Bernoulli mask before any scaling.
Static methods
1static
forward
→TensorImplPtrint forward(const int & a, double p, bool training, Generator * gen)Apply alpha dropout with SELU-preserving affine correction.
Math
\quad m_i \sim \text{Bernoulli}(1 - p),$$ with $a = \bigl(\text{keep}\,(1 + p\,\alpha'^2)\bigr)^{-1/2}$ and $b = -a\,p\,\alpha'$.Parameters
aTensorImplPtrInput tensor of any shape. Expected to follow a SELU activation; other activations break the statistical invariant the affine correction restores.
pdoubleDrop probability in .
trainingboolIf
false, the forward is a pass-through.genGenerator*Optional RNG (see
DropoutBackward).Returns
TensorImplPtrOutput tensor with the same shape and dtype as a.