SK-ResNet feature-extracting backbone (no classification head).
Implements the Selective Kernel augmentation of the ResNet topology from Li et al., "Selective Kernel Networks", CVPR 2019 (arXiv:1903.06586). Each convolution inside a residual block is replaced by a two-branch Selective Kernel unit: a convolution and a dilation-2 convolution (effective receptive field ) run in parallel, with a softmax gate that produces per-radix, per-channel attention weights
Because and are computed from the input itself, every channel of every feature map can effectively choose its own receptive-field size at every spatial location.
The block topology supports both basic (SK-ResNet-18/34) and
bottleneck (SK-ResNet-50/101) variants, plus ResNeXt-style
grouped widening via the cardinality and base_width
knobs (sk_resnext_50_32x4d).
Parameters
configSKNetConfigsk_resnet_18, sk_resnet_50,
sk_resnext_50_32x4d, …) for paper-cited variants.Attributes
configSKNetConfigstemnn.Sequentialmaxpoolnn.MaxPool2dlayer1, layer2, layer3, layer4nn.Sequentialfeature_infolist[FeatureInfo]BackboneMixin.Notes
The Selective Kernel attention is a generalisation of SE
attention to multiple parallel branches with a softmax gate;
when only one branch is used it reduces to plain SE. At
radix == 2 (the default), the block roughly matches the
receptive-field diversity of an Inception module while
keeping the parameter overhead small.
Examples
Build an SK-ResNet-50 backbone and run a forward pass:
>>> import lucid
>>> from lucid.models.vision.sknet import sk_resnet_50
>>> backbone = sk_resnet_50()
>>> x = lucid.randn(2, 3, 224, 224)
>>> out = backbone(x)
>>> out.last_hidden_state.shape
(2, 2048, 7, 7)