nn.functional.grid_sample

lucid.nn.functional.grid_sample(input_: Tensor, grid: Tensor, mode: Literal['bilinear', 'nearest', 'area'] = 'bilinear', padding_mode: Literal['zeros', 'border'] = 'zeros', align_corners: bool = True) → Tensor

The grid_sample function applies a sampling grid to an input tensor using interpolation. It is used to spatially transform inputs according to the grid, often generated by affine_grid.

Function Signature

def grid_sample(
    input_: Tensor,
    grid: Tensor,
    mode: _InterpolateType = 'bilinear',
    padding_mode: _PaddingType = 'zeros',
    align_corners: bool = True
) -> Tensor

Parameters

• input_ (Tensor): Input tensor of shape (N, C, H_in, W_in), where N is the batch size, C is the number of channels, H_in is input height, and W_in is input width.

• grid (Tensor): A tensor of shape (N, H_out, W_out, 2) containing normalized coordinates in the range [-1, 1]. Each coordinate specifies the sampling location in the input image.

• mode (str, optional): Interpolation method to use. Options:

  • ‘bilinear’ (default): Performs bilinear sampling.

  • ‘nearest’: Picks the value from the nearest neighbor.

• padding_mode (str, optional): Padding strategy for out-of-bound grid locations:

  • ‘zeros’ (default): Pads with zero.

  • ‘border’: Uses the value at the border.

  • ‘reflection’: Not yet implemented.

• align_corners (bool, optional): If True, the extreme values of the grid (-1 and 1) map to the corners of the input. If False, they map to the centers of corner pixels. Default is True.

Returns

• Tensor: A tensor of shape (N, C, H_out, W_out) resulting from sampling the input at grid locations using the specified interpolation and padding mode.

Forward Calculation

• The grid is normalized to [-1, 1] and mapped to the input coordinate system.

• Each output pixel is computed by interpolating values from the input image.

Note

The grid is usually generated using affine_grid, but can be any differentiable tensor of coordinates.

Examples

Apply identity transform:

>>> import lucid
>>> from lucid import Tensor
>>> import lucid.nn.functional as F

>>> x = Tensor([[[[0, 1, 2],
...                [3, 4, 5],
...                [6, 7, 8]]]], requires_grad=True)
>>> theta = Tensor([[[1, 0, 0], [0, 1, 0]]], requires_grad=True)
>>> grid = F.affine_grid(theta, size=(1, 1, 3, 3))
>>> out = F.grid_sample(x, grid, mode='bilinear')
>>> print(out)
Tensor([[[[0.0, 1.0, 2.0],
          [3.0, 4.0, 5.0],
          [6.0, 7.0, 8.0]]]])

Backpropagate through the spatial transform:

>>> out.sum().backward()
>>> print(x.grad)
# Tensor with gradients accumulated from bilinear sampling

>>> print(theta.grad)
# Tensor with gradients showing how output sum changes w.r.t the affine transform

Notes

• grid_sample is commonly used in spatial transformer networks (STNs).

• Differentiability is preserved in ‘bilinear’ mode only. In ‘nearest’ mode, gradients w.r.t. grid are zero.

• reflection padding is not yet supported in Lucid.