vit_base_32

→ViT

vit_base_32(pretrained: bool = False, overrides: object = {})

source edit

ViT-Base/32 backbone (Dosovitskiy et al., 2020).

Same ViT-Base trunk as vit_base_16 (dim=768, depth=12, num_heads=12) but with patch size 32, which reduces the sequence length to $(224/32)^2 = 49$ patch tokens (+1 CLS = 50). The larger patches trade fine-grained spatial detail for ~4x cheaper self-attention and ~88M parameters in total.

Model Size

Parameters

pretrainedbool= False

If True, loads ImageNet-21k pretrained weights when available. Defaults to False.

**overridesobject= {}

Keyword overrides on top of the canonical ViT-Base/32 config.

Returns

ViT

A ViT backbone returning a flat (B, 768) CLS feature.

Notes

Reference: Dosovitskiy et al. (2020), Table 1. The /32 variant is most useful when latency / FLOPs dominate over absolute accuracy.

Examples

>>> import lucid
>>> from lucid.models.vision.vit import vit_base_32
>>> model = vit_base_32()
>>> x = lucid.randn(1, 3, 224, 224)
>>> model.forward_features(x).shape
(1, 768)

Used by 1

lucid.models.vision.vit