AlexNet

AlexNet is the architecture that re-ignited deep learning, winning the ILSVRC-2012 ImageNet classification challenge with a top-5 error of 15.3% — more than ten percentage points below the runner-up — and demonstrating for the first time that a large convolutional network trained end-to-end on raw pixels could decisively beat hand-engineered feature pipelines.

The architecture stacks five convolutional layers followed by three fully-connected layers, taking $3\times224\times224$ RGB inputs and producing 1000-way softmax logits. Three contributions made training such a deep network tractable on 2012-era hardware: (i) the rectified linear unit $\phi(x) = \max(0, x)$ replaced saturating nonlinearities, cutting training time by several factors; (ii) dropout with $p=0.5$ in the two large 4096-dim fully-connected layers regularised the ≈61 M parameters against overfitting; and (iii) overlapping max-pooling combined with heavy data augmentation (random crops, horizontal flips, AlexNet-style PCA colour jitter) closed the train-test gap.

The original 2012 paper split conv filters across two GPUs (model parallel) because a single GTX 580's 3 GB memory budget could not hold the network — Krizhevsky 2014 ("One weird trick for parallelizing convolutional neural networks") later re-derived the network as a single merged stream with adjusted channel widths $(64, 192, 384, 256, 256)$ and no local response normalisation; this single-stream variant is what Lucid (and every published reference checkpoint) ships. AlexNet's success established the now-standard recipe of deep ConvNet + ReLU + dropout + heavy data augmentation + SGD with momentum, and every subsequent ImageNet-scale vision model descends from it directly.