ZFNet

ZFNet is the ILSVRC-2013 winner and is best understood as an AlexNet that was tuned by looking at what the network had learned. Zeiler and Fergus introduced the deconvolutional network visualisation technique — projecting individual feature- map activations back to the input pixel space — and used the resulting diagnostic images to reshape the early layers of the network.

Two specific changes distinguish ZFNet from AlexNet. The first convolution was reduced from an $11\times11$ stride-4 kernel to a $7\times7$ stride-2 kernel, addressing the observation that AlexNet's first layer learned a mixture of extremely high-frequency and dead filters. The second convolution's stride was simultaneously dropped from 2 to a more moderate value, giving denser low-level feature coverage and avoiding aliasing artefacts that the visualisations had revealed. The remaining layers retain AlexNet's topology.

These modest topological changes — together with the methodology of using visualisation to drive architectural choices — were influential out of proportion to their parameter cost. ZFNet achieved a top-5 error of 11.7% on ImageNet, and the deconvolutional-network analysis it popularised is the conceptual ancestor of every later interpretability tool from Grad-CAM to feature inversion.