vit_large_32

→ViT

vit_large_32(pretrained: bool = False, overrides: object = {})

source edit

ViT-Large/32 backbone (Dosovitskiy et al., 2020).

Same ViT-Large trunk as vit_large_16 (dim=1024, depth=24, num_heads=16) but with patch size 32, producing only $(224/32)^2 = 49$ patch tokens (+1 CLS). Approximately 307M parameters with substantially reduced attention cost.

Model Size

Parameters

pretrainedbool= False

If True, loads ImageNet-21k pretrained weights when available. Defaults to False.

**overridesobject= {}

Keyword overrides on top of the canonical ViT-Large/32 config.

Returns

ViT

A ViT backbone returning a flat (B, 1024) CLS feature.

Notes

Reference: Dosovitskiy et al. (2020), Table 1. Useful when Large capacity is desired but the /16 token count is too expensive.

Examples

>>> import lucid
>>> from lucid.models.vision.vit import vit_large_32
>>> model = vit_large_32()
>>> x = lucid.randn(1, 3, 224, 224)
>>> model.forward_features(x).shape
(1, 1024)

Used by 1

lucid.models.vision.vit