GoogLeNet

GoogLeNet (Inception v1) is the ILSVRC-2014 classification winner and introduced the Inception module — a multi-branch building block that lets a single layer apply convolutions at several receptive-field scales in parallel and concatenate their outputs. The motivation is that natural images contain salient structure at multiple sizes, and forcing the network to commit to a single kernel per layer wastes capacity.

Each Inception module computes, in parallel, a $1\times1$ convolution, a $3\times3$ convolution, a $5\times5$ convolution, and a $3\times3$ max-pool branch. To keep the computation tractable, cheap $1\times1$ bottlenecks are placed before the larger spatial convolutions and after the pooling branch. Concatenating along the channel axis gives the next layer an enriched, multi-scale feature representation. Formally, an Inception block produces

where each $f$ is a learned convolution and each $g, h, p_{1\times1}$ is a dimensionality-reducing $1\times1$ projection.

GoogLeNet stacks nine such modules into a 22-layer network with only 5 M parameters — roughly 12× fewer than AlexNet — while achieving a top-5 error of 6.67%. Two auxiliary classifiers were attached to intermediate layers during training to combat vanishing gradients and add a regularising effect; they are discarded at inference. The Inception family launched here led to Inception v2/v3 (factorised convolutions, label smoothing) and eventually Inception-ResNet.