GoogLeNet

GoogLeNet (Inception v1) is the ILSVRC-2014 classification winner and introduced the Inception module — a multi-branch building block that lets a single layer apply convolutions at several receptive-field scales in parallel and concatenate their outputs. The motivation is that natural images contain salient structure at multiple sizes, and forcing the network to commit to a single kernel per layer wastes capacity.

Each Inception module computes four branches in parallel and concatenates them along the channel axis, giving the next layer an enriched, multi-scale representation. To keep the computation tractable, cheap $1\times1$ bottlenecks sit before the larger spatial convolutions and after the pooling branch:

where each $f$ is a learned convolution and each $g, h, p_{1\times1}$ is a dimensionality-reducing $1\times1$ projection.

Divergence, deliberate: the paper's third branch is a $5\times5$ convolution. The reference implementation the shipped weights come from uses a $3\times3$ there instead — a documented deviation on its side — and this build follows the weights, so the third branch is $3\times3$ in the code and in the equation above.

GoogLeNet stacks nine such modules into a 22-layer network with only 5 M parameters — roughly 12× fewer than AlexNet — while achieving a top-5 error of 6.67%. Two auxiliary classifiers were attached to intermediate layers during training to combat vanishing gradients and add a regularising effect; they are discarded at inference. The Inception family launched here led to Inception v2/v3 (factorised convolutions, label smoothing) and eventually Inception-ResNet.