vision

AlexNet is the architecture that re-ignited deep learning, winning the ILSVRC-2012 ImageNet classification challenge with a top-5 error of 15.3% — more than ten percentage points below the runner-up — and demonstrating for the first time that a large convolutional network trained end-to-end on raw pixels could decisively beat hand-engineered feature pipelines.

Attention U-Net augments the standard U-Net skip pathways with soft spatial attention gates. In a plain U-Net every encoder feature is concatenated unconditionally into the decoder, so irrelevant background activations still propagate to the output. For small structures like the pancreas — where the target occupies a tiny fraction of the field of view — this leaks gradient signal into the wrong regions and hurts both convergence and final Dice.

CoAtNet hybridises depthwise convolution and relative self-attention in a single backbone, motivated by the observation that convolutions excel at generalization (strong local inductive bias) while attention excels at capacity (data-dependent global mixing). The network is a four-stage pyramid preceded by a two-layer convolutional stem; the early stages use MBConv blocks (squeeze-and-excitation, expansion ratio 4) and the later stages use relative-attention transformer blocks operating on $\sqrt{HW}$-length token sequences.

ConvNeXt is a pure convolutional architecture obtained by systematically modernizing a ResNet using design choices borrowed from the Swin Transformer. Starting from a ResNet-50 the authors apply a one-at-a-time roadmap: change the stage compute ratio to $(3, 3, 9, 3)$, swap the stem for a non-overlapping $4 \times 4$ patchify convolution, replace the bottleneck with a depthwise $7 \times 7$ convolution followed by an inverted bottleneck, move the activation and normalization to a single GELU + LayerNorm after the depthwise conv, and replace BatchNorm with LayerNorm. The resulting block is

CrossViT explicitly models multi-scale visual information by running two parallel ViT branches with different patch sizes and fusing them through a lightweight cross-attention mechanism. A small-patch branch (patch size 12) processes a long sequence of fine-grained tokens, while a large-patch branch (patch size 16) processes a short sequence of coarse tokens. The two branches operate on different input resolutions — the small-patch branch on the full image (e.g. $240\times240$), the large-patch branch on a slightly down-sampled version (e.g. $224\times224$) — and each has its own class token plus learnable positional embedding.

CSPNet ("Cross-Stage Partial Network") is a generic backbone transformation that can be wrapped around almost any dense-or-residual stage — ResNet, ResNeXt, DenseNet, Darknet — to cut its compute and memory while improving its accuracy.

The Convolutional vision Transformer (CvT) reinjects the locality inductive bias of CNNs into ViT through two changes: (i) replacing the non-overlapping patch embedding by overlapping convolutional token embedding, and (ii) replacing the linear projections inside self-attention with depthwise-separable convolutional projections. CvT also drops positional embeddings entirely — locality is supplied implicitly by the convolutional projections.

DenseNet takes the residual-shortcut idea to its logical extreme. Inside each dense block, every layer receives as input the concatenated feature maps of all previous layers in the block, not just the immediately preceding layer. For a block of $L$ layers the $\ell$-th layer computes

DETR (DEtection TRansformer) reframes object detection as a direct set prediction problem. A CNN backbone produces a feature map $f \in \mathbb{R}^{C \times H \times W}$, which is flattened, projected to $d_{\mathrm{model}}$, augmented with 2-D sinusoidal positional encodings, and fed into a standard Transformer encoder–decoder. The decoder additionally takes $N$ learnable object queries; each query produces one detection through FFN heads (class logits + normalised box $(c_x, c_y, w, h)$ with a sigmoid output).

EfficientDet jointly applies the compound scaling idea of EfficientNet to the entire detection pipeline — backbone, feature network, and prediction heads — instead of scaling any single axis (width, depth, resolution) in isolation. A single compound coefficient $\varphi$ controls all three:

EfficientFormer is a mobile-grade vision backbone designed so that its end-to-end on-device latency, rather than its FLOP count, matches the latency of MobileNet on the same hardware while retaining transformer-level accuracy. The authors profile real iOS / NPU latency and identify three transformer operators that are unexpectedly slow: the reshape between spatial and token layouts, GeLU compared to ReLU, and LayerNorm compared to BatchNorm. EfficientFormer is the result of a latency-aware architecture search that designs around these costs.

EfficientNet is built on the empirical observation that the three axes used to scale a ConvNet — depth $d$ (number of layers), width $w$ (number of channels per layer), and input resolution $r$ — are not independent. Scaling any one of them in isolation gives rapidly diminishing returns, but scaling all three together along a carefully chosen ratio gives a much better accuracy/FLOPs frontier.

Fast R-CNN closes the dominant performance gap of R-CNN — the one-CNN-forward-per-proposal cost — by sharing convolutional computation across all proposals in an image.

Faster R-CNN removes the last hand-engineered component of the R-CNN family — the external selective-search proposal step — by introducing a Region Proposal Network (RPN) that shares convolutional features with the detection head.

The Fully Convolutional Network is the first deep model trained end-to-end, pixels-to-pixels, for semantic segmentation. Its core idea is to convert a classification CNN into a dense predictor by reinterpreting its fully-connected layers as $1 \times 1$ convolutions and learning to up-sample the coarse spatial output back to input resolution.

GoogLeNet (Inception v1) is the ILSVRC-2014 classification winner and introduced the Inception module — a multi-branch building block that lets a single layer apply convolutions at several receptive-field scales in parallel and concatenate their outputs. The motivation is that natural images contain salient structure at multiple sizes, and forcing the network to commit to a single kernel per layer wastes capacity.

Inception v3 is the third generation of the Inception family and crystallises a set of design principles that the authors extracted from earlier experiments — most notably that one should avoid representational bottlenecks (do not aggressively shrink spatial resolution in early layers) and should factorise large convolutions to gain both parameter efficiency and additional nonlinearity.

InceptionNeXt is a drop-in replacement for the ConvNeXt block that eliminates the cost of a single large $7 \times 7$ depthwise convolution by factorizing it into four parallel, low-cost Inception-style branches. The motivation is that large-kernel depthwise convs dominate ConvNeXt's wall-clock latency on modern accelerators despite their relatively small FLOP count: depthwise convolutions are memory-bandwidth bound and large kernels amplify that bottleneck.

Inception-ResNet hybridises the two dominant architectural ideas of the early ResNet era: the multi-branch Inception module and the residual shortcut. Each Inception sub-network is wrapped in an identity skip connection, so the block computes

LeNet-5 is the prototypical convolutional neural network and the direct ancestor of every modern vision architecture. Designed for handwritten-digit recognition on 32×32 grayscale inputs, it interleaves learned $5\times5$ convolutions with fixed $2\times2$ sub-sampling (average pooling) layers, ending in two fully-connected layers and a Gaussian-RBF output layer.

Mask R-CNN extends Faster R-CNN with a parallel mask branch that predicts a per-RoI binary segmentation, turning the two-stage detector into a unified instance-segmentation model with negligible runtime overhead.

Mask2Former generalises MaskFormer into a single architecture that wins on semantic, instance, and panoptic segmentation simultaneously. It keeps the mask-classification framing — $N$ object queries each predict a class and a binary mask — and improves it with three orthogonal changes.

MaskFormer recasts semantic segmentation as a mask-classification problem rather than the standard per-pixel-classification one. Instead of predicting a $K$-way softmax at every pixel, the model emits $N$ queries; each query produces a class label and a binary mask, and the final segmentation is the soft sum

MaxViT combines a convolutional MBConv stage with two complementary sparse self-attention mechanisms — block attention and grid attention — to obtain global receptive fields at linear complexity in the number of tokens. The backbone is a four-stage pyramid, and each stage stacks repeats of the MaxViT block consisting of a MobileNetV3-style MBConv followed by block and then grid attention.

MobileNet is built around the depthwise separable convolution, a factorisation that decomposes a standard convolution into two much cheaper stages: a per-channel depthwise $3 \times 3$ spatial filter followed by a $1 \times 1$ pointwise convolution that mixes channels. For an input of $M$ channels and an output of $N$ channels with kernel size $D_K$, the cost drops from $D_K \cdot D_K \cdot M \cdot N \cdot D_F \cdot D_F$ to

MobileNet-v2 replaces the plain depthwise-separable stack of v1 with the inverted residual bottleneck block. Each block receives a low-dimensional tensor of $c$ channels, projects it up to a much wider $t \cdot c$ channel space with a $1 \times 1$ convolution, applies a $3 \times 3$ depthwise convolution in that expanded space, then projects back down to $c'$ channels with another $1 \times 1$. When the input and output shapes match, an identity shortcut is added around the whole block:

MobileNet-v3 is the result of an explicit neural architecture search layered on top of the inverted-residual block of v2. A platform-aware NAS (MnasNet-style) selects the per-stage expansion ratio, kernel size, and channel count to minimise a latency-and-accuracy joint objective on a real mobile CPU, after which a NetAdapt pass fine-tunes the channel counts of each layer individually under a hard latency budget. The output is two hand-tuned variants — Large and Small — targeting different parts of the latency/accuracy frontier.

The Pyramid Vision Transformer (PVT) introduces a hierarchical feature pyramid into a pure-transformer backbone so that it can serve as a drop-in replacement for ResNet-style backbones in dense prediction tasks such as detection and segmentation. Plain ViT produces a single-scale, low-resolution feature map and scales quadratically with the input resolution; PVT addresses both problems with two design changes.

R-CNN (Regions with CNN features) is the first deep-learning detector to demonstrate that ImageNet-pretrained CNN features transfer to localisation. The pipeline is deliberately modular:

ResNeSt fuses three previously separate ideas — ResNeXt's cardinality, SKNet's soft kernel selection, and SENet's channel-wise attention — into a single unified Split-Attention block that can replace the central $3\times3$ convolution of any ResNet bottleneck.

Deep residual networks introduce identity shortcuts around every pair (or triple) of stacked convolutions. A residual block computes

ResNeXt augments the ResNet bottleneck with a third axis of network design that the authors term cardinality: the number of parallel transformation branches inside each block. Where ResNet improves accuracy by making the network deeper and Wide ResNet improves it by making each layer wider, ResNeXt improves it by making each block more parallel.

SENet introduces the Squeeze-and-Excitation block, a tiny drop-in channel-attention module that recalibrates the relative importance of each feature-map channel after every convolutional stage. The block runs in three phases.

SKNet builds on the biological observation that the receptive field size of cortical neurons is not fixed but is modulated by the stimulus. Standard ConvNets bake the receptive field into the architecture (a $3\times3$ kernel always sees a $3\times3$ neighbourhood); SKNet replaces that with a Selective Kernel unit that dynamically chooses how much of each receptive-field size to use for every input, on a per-channel basis.

The Swin Transformer reintroduces the hierarchical feature pyramid of classical CNNs into a pure-transformer backbone, making it a general-purpose vision model that scales linearly with image resolution rather than quadratically (as plain ViT does). The image is first split into non-overlapping $4 \times 4$ patches and projected to $C$ channels. Four stages then follow, each halving the spatial resolution via a patch merging layer that concatenates $2 \times 2$ neighbours and projects $4C \to 2C$, doubling the channel width — giving the canonical $C, 2C, 4C, 8C$ pyramid.

U-Net is a symmetric encoder–decoder architecture with dense skip connections, designed for biomedical segmentation where training data is scarce and every pixel matters.

VGG demonstrated that uniformity and depth — rather than bespoke layer designs — were the dominant factors driving image- classification accuracy in 2014. The network is built from a single recipe: stack $3\times3$ convolutions with stride 1 and padding 1, periodically halve the spatial resolution with a $2\times2$ max-pool, double the channel count at each pooling boundary, and end with three large fully-connected layers.

The Vision Transformer (ViT) shows that convolutions are not required for state-of-the-art image classification given enough data. An input image $x \in \mathbb{R}^{H \times W \times C}$ is reshaped into a sequence of non-overlapping square patches $x_p \in \mathbb{R}^{N \times (P^2 C)}$ with $N = HW / P^2$, each patch flattened and projected to a $D$-dimensional token by a single linear map. A learnable class token $x_{\text{cls}}$ is prepended, and learnable positional embeddings $E_{\text{pos}}$ are added:

Xception ("Extreme Inception") interprets the Inception module as an intermediate point on a continuum between an ordinary convolution and a fully decoupled spatial/channel filter. A standard Inception block splits the input into a few parallel branches, each of which is processed by a small $1\times1$ pointwise convolution followed by a spatial convolution, and the branch outputs are concatenated. Xception pushes this idea to its limit: instead of a handful of branches, every output channel of the $1\times1$ pointwise convolution gets its own independent $3\times3$ spatial filter. This is exactly the depthwise separable convolution — a pointwise mix in channel space followed by a per-channel spatial filter.

YOLOv1 reframes detection as a single regression problem. The input image is divided into an $S \times S$ grid; each cell is responsible for objects whose centre falls inside it. For each cell the network predicts $B$ candidate boxes $(x, y, w, h, \mathrm{conf})$ plus one shared class distribution over $C$ categories, so the head output has shape $S \times S \times (B \cdot 5 + C)$.

ZFNet is the ILSVRC-2013 winner and is best understood as an AlexNet that was tuned by looking at what the network had learned. Zeiler and Fergus introduced the deconvolutional network visualisation technique — projecting individual feature- map activations back to the input pixel space — and used the resulting diagnostic images to reshape the early layers of the network.