ConvNeXt

ConvNeXt is a pure convolutional architecture obtained by systematically modernizing a ResNet using design choices borrowed from the Swin Transformer. Starting from a ResNet-50 the authors apply a one-at-a-time roadmap: change the stage compute ratio to $(3, 3, 9, 3)$, swap the stem for a non-overlapping $4 \times 4$ patchify convolution, replace the bottleneck with a depthwise $7 \times 7$ convolution followed by an inverted bottleneck, move the activation and normalization to a single GELU + LayerNorm after the depthwise conv, and replace BatchNorm with LayerNorm. The resulting block is

where $\gamma \in \mathbb{R}^d$ is a layer-scale parameter initialized to $10^{-6}$ (Touvron et al., 2021).

The pyramid follows the four-stage Swin layout — channel widths $(C, 2C, 4C, 8C)$ with downsampling between stages — and the canonical variants Tiny / Small / Base / Large / XLarge scale the stem width $C$ and the depth of stage 3 from $9$ to $27$. The paper's headline result is that the modernised ConvNet matches or exceeds Swin's ImageNet accuracy at the same FLOPs while remaining a simple all-convolutional design.