data

SwinConfig

extendsModelConfig

SwinConfig(image_size: int = 224, patch_size: int = 4, in_channels: int = 3, num_classes: int = 1000, embed_dim: int = 96, depths: tuple[int, ...] = (2, 2, 6, 2), num_heads: tuple[int, ...] = (3, 6, 12, 24), window_size: int = 7, mlp_ratio: float = 4.0, dropout: float = 0.0, attention_dropout: float = 0.0, drop_path_rate: float = 0.0)

source edit

Configuration dataclass for every Swin Transformer variant.

SwinConfig is an immutable container that fully specifies the architecture of a Swin Transformer. It is consumed by both SwinTransformer (backbone) and SwinTransformerForImageClassification (classifier). All canonical variants described in Liu et al. (2021) — Tiny / Small / Base / Large — can be expressed by choosing different per-stage depths, channel widths, and head counts.

The backbone is a four-stage pyramid: the stem patch embedding maps $(H, W, 3) \to (H/p, W/p, C)$ , then each stage applies a stack of pairs of Swin blocks (window + shifted-window attention) and a patch-merging downsampler that halves the spatial size while doubling the channel width, giving the canonical $C, 2C, 4C, 8C$ pyramid.

Parameters

image_sizeint= 224

Spatial side length (in pixels) of the square input image. Defaults to 224. Must be divisible by patch_size.

patch_sizeint= 4

Side length of the non-overlapping patches extracted by the stem convolution. Defaults to 4.

in_channelsint= 3

Number of input image channels. Defaults to 3 (RGB).

num_classesint= 1000

Number of output classes for the classification head. Defaults to 1000 (ImageNet-1k).

embed_dimint= 96

Channel width

C

after patch embedding (stage-1 dim). Each subsequent stage doubles, so the final stage has width

8C

. Defaults to 96 (Swin-T).

depthstuple of int= (2, 2, 6, 2)

Number of Swin blocks in each of the four stages. Defaults to (2, 2, 6, 2) (Swin-T).

num_headstuple of int= (3, 6, 12, 24)

Number of attention heads in each stage. Defaults to (3, 6, 12, 24) (Swin-T).

window_sizeint= 7

Side length

M

of the local attention window inside each Swin block. Defaults to 7.

mlp_ratiofloat= 4.0

Expansion ratio of the MLP block. Defaults to 4.0.

dropoutfloat= 0.0

Dropout probability inside the MLP feed-forward layer and after the patch embedding. Defaults to 0.0.

attention_dropoutfloat= 0.0

Dropout probability on post-softmax attention weights. Defaults to 0.0.

drop_path_ratefloat= 0.0

Stochastic-depth maximum rate; linearly scheduled across all blocks of the trunk (Liu et al., 2021, Appendix A). The paper uses 0.2 for Swin-T and 0.3 for Swin-S/B/L. Defaults to 0.0.

Attributes

model_typeClassVar[str]

Constant string "swin" used by the model registry.

Notes

The canonical variants registered as factory functions in lucid.models.vision.swin are (see Table 7 of the paper):

============= ========== ================== =================== Variant embed_dim depths num_heads ============= ========== ================== =================== Swin-T 96 (2, 2, 6, 2) (3, 6, 12, 24) Swin-S 96 (2, 2, 18, 2) (3, 6, 12, 24) Swin-B 128 (2, 2, 18, 2) (4, 8, 16, 32) Swin-L 192 (2, 2, 18, 2) (6, 12, 24, 48) ============= ========== ================== ===================

Reference: Ze Liu et al., "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows", ICCV 2021, arXiv:2103.14030.

Examples

Build a Swin-Tiny configuration for CIFAR-100 (100 classes):
>>> from lucid.models.vision.swin import SwinConfig
>>> cfg = SwinConfig(num_classes=100)
>>> cfg.embed_dim, cfg.depths, cfg.num_heads
(96, (2, 2, 6, 2), (3, 6, 12, 24))
Override the stem patch size for a higher-resolution variant:
>>> cfg = SwinConfig(patch_size=8, image_size=384)
>>> cfg.image_size // cfg.patch_size  # tokens per side
48

Used by 3

Constructors

dunder

init

→None

__init__(image_size: int = 224, patch_size: int = 4, in_channels: int = 3, num_classes: int = 1000, embed_dim: int = 96, depths: tuple[int, ...] = (2, 2, 6, 2), num_heads: tuple[int, ...] = (3, 6, 12, 24), window_size: int = 7, mlp_ratio: float = 4.0, dropout: float = 0.0, attention_dropout: float = 0.0, drop_path_rate: float = 0.0)

source edit

SwinConfig(image_size: int = 224, patch_size: int = 4, in_channels: int = 3, num_classes: int = 1000, embed_dim: int = 96, depths: tuple[int, ...] = (2, 2, 6, 2), num_heads: tuple[int, ...] = (3, 6, 12, 24), window_size: int = 7, mlp_ratio: float = 4.0, dropout: float = 0.0, attention_dropout: float = 0.0, drop_path_rate: float = 0.0)

Build a Swin-Tiny configuration for CIFAR-100 (100 classes): >>> from lucid.models.vision.swin import SwinConfig >>> cfg = SwinConfig(num_classes=100) >>> cfg.embed_dim, cfg.depths, cfg.num_heads (96, (2, 2, 6, 2), (3, 6, 12, 24)) Override the stem patch size for a higher-resolution variant: >>> cfg = SwinConfig(patch_size=8, image_size=384) >>> cfg.image_size // cfg.patch_size # tokens per side 48

__init__(image_size: int = 224, patch_size: int = 4, in_channels: int = 3, num_classes: int = 1000, embed_dim: int = 96, depths: tuple[int, ...] = (2, 2, 6, 2), num_heads: tuple[int, ...] = (3, 6, 12, 24), window_size: int = 7, mlp_ratio: float = 4.0, dropout: float = 0.0, attention_dropout: float = 0.0, drop_path_rate: float = 0.0)