relu_inplace_op

→TensorImplPtr

int relu_inplace_op(const int & a)

source edit

Python wrappers

pylucid.nn.functional.relu_function

In-place rectified linear unit: writes $\max(0, x)$ back into a.

Math

$a_i \leftarrow \max(0, a_i)$

See Also

relu_op — out-of-place counterpart.

Parameters

aTensorImplPtr

Tensor to apply ReLU to in place. Must be non-null.

Returns

TensorImplPtr

The same pointer as a.

Notes

The most common in-place op in practice: ResNet-style residual blocks save a per-layer activation by writing ReLU back over the convolution output. Safe under autograd as long as the convolution was the only saved-for-backward dependency.