ResNeSt

ResNeSt fuses three previously separate ideas — ResNeXt's cardinality, SKNet's soft kernel selection, and SENet's channel-wise attention — into a single unified Split-Attention block that can replace the central $3\times3$ convolution of any ResNet bottleneck.

The block first splits the input into $K$ groups (cardinality), and inside each group it further splits the feature map into $R$ parallel branches called radixes. Each radix runs a different transformation $\mathcal{F}_r$ and produces a $C$ -channel feature map $U_r$. The branch outputs are summed to form a single combined response $U = \sum_r U_r$, which is then pooled and passed through a small bottleneck network to emit per-radix, per-channel attention weights:

The block's output is the radix-weighted sum

applied independently inside every cardinal group and then concatenated. When $R = 1$ the block reduces to ResNeXt; when $R \geq 2$ it generalises SKNet to arbitrary radix counts with a softmax (rather than two-way sigmoid) gate.

Two further refinements complete the architecture. A deep stem replaces the original $7\times7$ ResNet stem with three $3\times3$ convolutions, and an average-pool downsampling path (controlled by the avg_down and avd fields) replaces strided convolutions in residual shortcuts and inside the Split-Attention block itself. With these changes ResNeSt-50 outperforms a plain ResNet-50 by roughly 3 ImageNet top-1 percentage points at the same parameter count, and serves as a strong backbone for downstream detection / segmentation tasks.