class

HardSwishBackward

HardSwishBackward

source edit

Autograd node for Hard Swish $y = x \cdot \mathrm{hard\_sigmoid}(x)$ (Howard et al., MobileNetV3).

The derivative is the three-region piecewise function $0$ for $x \le -3$ , $(2x + 3)/6$ for $-3 < x < 3$ , and $1$ for $x \ge 3$ . Backward is delegated to the backend for a single fused pass.

Math

\qquad \frac{\partial y}{\partial x} = \begin{cases} 0, & x \le -3 \\ (2x + 3)/6, & -3 < x < 3 \\ 1, & x \ge 3 \end{cases}$$

Attributes

schema_v1OpSchema

"hard_swish", AmpPolicy::KeepInput.

Static methods

static

dispatch

int dispatch(backend::IBackend & be, const int & a, const int & s, Dtype dt)

source edit

Forward — calls IBackend::hard_swish to compute $y = x \cdot \mathrm{hard\_sigmoid}(x)$ .

Methods

grad_formula

int grad_formula(const int & g)

source edit

Eager-mode backward: dispatches to IBackend::hard_swish_backward.

class

HardSwishBackward

HardSwishBackward

source edit

Autograd node for Hard Swish $y = x \cdot \mathrm{hard\_sigmoid}(x)$ (Howard et al., MobileNetV3).

The derivative is the three-region piecewise function $0$ for $x \le -3$ , $(2x + 3)/6$ for $-3 < x < 3$ , and $1$ for $x \ge 3$ . Backward is delegated to the backend for a single fused pass.

Math

\qquad \frac{\partial y}{\partial x} = \begin{cases} 0, & x \le -3 \\ (2x + 3)/6, & -3 < x < 3 \\ 1, & x \ge 3 \end{cases}$$

Attributes

schema_v1OpSchema

"hard_swish", AmpPolicy::KeepInput.

Static methods

static

dispatch

int dispatch(backend::IBackend & be, const int & a, const int & s, Dtype dt)

source edit

Forward — calls IBackend::hard_swish to compute $y = x \cdot \mathrm{hard\_sigmoid}(x)$ .

Methods

grad_formula

int grad_formula(const int & g)

source edit

Eager-mode backward: dispatches to IBackend::hard_swish_backward.