ViT-Large/16 backbone (Dosovitskiy et al., 2020).
The canonical ViT-Large configuration with patch size 16:
dim=1024, depth=24, num_heads=16, mlp_ratio=4.0. With
a 224x224 input it produces 197 tokens (196 patches + CLS) feeding 24
pre-norm transformer blocks of width 1024. Approximately
307M parameters.
Model Size
Parameters
pretrainedbool= FalseIf
True, loads ImageNet-21k pretrained weights when available.
Defaults to False.**overridesobject= {}Keyword overrides on top of the canonical ViT-Large/16 config.
Returns
ViTA ViT backbone returning a flat (B, 1024) CLS feature.
Notes
ViT-Large/16 reaches 85.30% top-1 on ImageNet-1k when fine-tuned at 384x384 after ImageNet-21k pretraining (Dosovitskiy et al., 2020, Table 5). Best accuracy/cost trade-off among the Large variants.
Examples
>>> import lucid
>>> from lucid.models.vision.vit import vit_large_16
>>> model = vit_large_16()
>>> x = lucid.randn(1, 3, 224, 224)
>>> model.forward_features(x).shape
(1, 1024)