GridSampleBackward

Autograd node for differentiable image resampling (grid sample).

Reads a 4-D input image at the (typically non-integer) positions specified by a sampling grid:

where $\mathcal{I}$ is the chosen interpolation kernel and the grid stores normalised $(x, y)$ coordinates in $[-1, 1]$. For bilinear interpolation the four neighbouring pixels around the unnormalised coordinate $u = (x + 1) \cdot (W_{\text{in}} - 1) / 2$, $v = (y + 1) \cdot (H_{\text{in}} - 1) / 2$ (under align_corners_=true) are combined with the standard bilinear weights; analogous neighbourhoods of four (nearest) or sixteen (bicubic) pixels are used in the other modes.

Out-of-bounds samples are handled by padding_mode_: zeros returns $0$, border clamps the source coordinate, reflection reflects it off the image boundary. The backward distributes grad_out back to both the input pixels (through the interpolation weights) and the grid (through the analytical derivative of the interpolation kernel).

Math

Coordinate unnormalisation:

(analogous formula for $v$).

References

Jaderberg et al., "Spatial Transformer Networks" (NeurIPS 2015).