class

TanhBackward

TanhBackward

source edit

Python wrappers

Autograd node for element-wise hyperbolic tangent $y = \tanh(x)$ .

Saves the output $y$ rather than the input because the backward formula $1 - y^2$ is cheaper than re-evaluating $\tanh(x)$ and avoids a second vvtanhf pass. kSavesInput = false opts out of the default input-save behaviour and kSavesOutput = true registers the output instead. Range is $(-1, 1)$ on all of $\mathbb{R}$ .

Math

\frac{\partial y}{\partial x} = 1 - \tanh^2(x) = 1 - y^2, \qquad \frac{\partial \mathcal{L}}{\partial x} = (1 - y^2)\,\frac{\partial \mathcal{L}}{\partial y}.$$

Attributes

schema_v1OpSchema

Registered as "tanh" with AmpPolicy::Promote.

kSavesInputbool

false.

kSavesOutputbool

true — the saved tensor is

y = \tanh(x)

Notes

Dispatch: Accelerate vvtanhf/vvtanh (CPU) / MLX tanh (GPU).

Static methods

static

dispatch

int dispatch(backend::IBackend & be, const int & a, const int & shape, Dtype dt)

source edit

Forward — calls IBackend::tanh to compute $y = \tanh x$ .

Methods

grad_formula

int grad_formula(const int & g)

source edit

Backward — $\partial y/\partial x = 1 - y^2$ using the saved output, scaled by grad_out.

grad_formula_impl

int grad_formula_impl(const int & g, const int &, const int & out)

source edit

Graph-mode backward: $\partial x = (1 - \mathrm{out}^2) \cdot g$ where $\mathrm{out} = \tanh x$ ; kept inside the autograd graph.

class

TanhBackward

TanhBackward

source edit

Python wrappers

Autograd node for element-wise hyperbolic tangent $y = \tanh(x)$ .

Math

\frac{\partial y}{\partial x} = 1 - \tanh^2(x) = 1 - y^2, \qquad \frac{\partial \mathcal{L}}{\partial x} = (1 - y^2)\,\frac{\partial \mathcal{L}}{\partial y}.$$

Attributes

schema_v1OpSchema

Registered as "tanh" with AmpPolicy::Promote.

kSavesInputbool

false.

kSavesOutputbool

true — the saved tensor is

y = \tanh(x)

Notes

Dispatch: Accelerate vvtanhf/vvtanh (CPU) / MLX tanh (GPU).

Static methods

static

dispatch

int dispatch(backend::IBackend & be, const int & a, const int & shape, Dtype dt)

source edit

Forward — calls IBackend::tanh to compute $y = \tanh x$ .

Methods

grad_formula

int grad_formula(const int & g)

source edit

Backward — $\partial y/\partial x = 1 - y^2$ using the saved output, scaled by grad_out.

grad_formula_impl

int grad_formula_impl(const int & g, const int &, const int & out)

source edit

Graph-mode backward: $\partial x = (1 - \mathrm{out}^2) \cdot g$ where $\mathrm{out} = \tanh x$ ; kept inside the autograd graph.