Autograd node for DropPath — per-sample stochastic depth (Huang, Sun, Liu & Weinberger 2016) used in residual / transformer blocks.
A single Bernoulli value is drawn per sample (mask shape
) and broadcast over every other dimension, so
either the entire residual contribution of a sample survives or it
is killed entirely. When scale_by_keep is true the surviving
samples are pre-scaled by so that the expectation
of the residual branch matches a deterministic forward.
Attributes
schema_v1OpSchemaRegistered schema (name
"drop_path", version 1, AmpPolicy::KeepInput).mask_StoragePer-sample mask broadcast to the full input shape; saved for the backward.
Static methods
1static
forward
→TensorImplPtrint forward(const int & a, double p, bool scale_by_keep, Generator * gen)Apply per-sample stochastic depth.
Parameters
aTensorImplPtrInput tensor of any rank ; the first axis is the batch axis.
pdoubleDrop probability in .
scale_by_keepboolIf
true, surviving samples are multiplied by to preserve the expectation of the residual branch.genGenerator*Optional RNG (see
DropoutBackward).Returns
TensorImplPtrOutput tensor with the same shape as a.
Raises
LucidErrorIf
p is outside or deterministic mode is on without a Generator.