CrossViT backbone (Chen et al., ICCV 2021).
Builds two parallel ViT branches that operate on different
resolutions and exchange information through K=3 cross-attention
fusion blocks. See CrossViTConfig for the per-variant
hyperparameters.
Examples
The backbone alone, built from a config rather than a factory. It
answers with a pooled vector, not a feature map — the two branches
have already been fused and their class tokens concatenated.
>>> import lucid
>>> from lucid.models.vision.crossvit import CrossViT, CrossViTConfig
>>> model = CrossViT(CrossViTConfig()).eval()
>>> out = model(lucid.randn(1, 3, 240, 240))
>>> out.last_hidden_state.shape
(1, 288)
240 pixels rather than 224: the branches take 12- and 16-pixel
patches, and 240 is the smaller size both divide.