module

ResNet

28 members

lucid.models.vision.resnet

ResNet model family — feature-extracting backbones and image classifiers.

Implements the residual-learning architecture from He et al., "Deep Residual Learning for Image Recognition", CVPR 2016 (arXiv:1512.03385), plus a few widely-cited follow-up variants:

Canonical ResNets — resnet_18, resnet_34, resnet_50, resnet_101, resnet_152 and their *_cls classification counterparts.
Wide ResNets (Zagoruyko & Komodakis, BMVC 2016) — wide_resnet_50, wide_resnet_101.
Deep bottleneck variants — resnet_200, resnet_269.

Every variant shares the same architecture-specifying dataclass ResNetConfig. Use the factory functions for paper-cited configurations; pass **overrides to tweak individual fields without writing a config by hand.

Image Classification

Paper

He, Kaiming, et al. "Deep Residual Learning for Image Recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.

Overview

Deep residual networks introduce identity shortcuts around every pair (or triple) of stacked convolutions. A residual block computes

y = \mathcal{F}(x, \{W_i\}) + x,

where $\mathcal{F}$ is the stacked-convolution branch and the addition is the identity shortcut. Reformulating the layer to learn a residual $\mathcal{F}$ rather than the full mapping $\mathcal{H}(x) = \mathcal{F}(x) + x$ makes the optimisation surface dramatically easier — the network only has to fit the correction to the identity, which is closer to zero at initialisation than the full mapping is to the target.

Stacking these blocks lets gradient signals flow through dozens or hundreds of layers without vanishing. The paper showed networks up to 152 layers deep training stably on ImageNet, with each successive depth setting a new state-of-the-art at the time (3.57% top-5 error, winning ILSVRC 2015).

Two block topologies are used. BasicBlock (two 3×3 convs) is used in ResNet-18/34. Bottleneck (1×1 → 3×3 → 1×1 with a 4× channel expansion) is used in ResNet-50/101/152 and all deeper / wider variants. The four ImageNet variants from Table 1 of the paper differ only in (block, [n₁, n₂, n₃, n₄]) where $n_i$ is the block-repetition count at stage $i$ .