CoAtNet-6 backbone (Dai et al., 2021).
Mixed-stage variant: S3 carries an MBConv sub-stage and a transformer sub-stage at a wider dimension. Spec from paper §A.2 / Table 12:
- S0 conv stem: D=192, L=2 (handled by
stem_width) - S1 MBConv: L=2, D=192
- S2 MBConv: L=4, D=384
- S3 mixed: MBConv L=8 D=768 → expand→1536 → TFMRel L=42 D=1536
- S4 TFMRel: L=2, D=2048
Approximately 1.5B parameters — only meaningful with very large pretraining (the paper's headline 88.4-89.0% ImageNet numbers come from JFT-3B pretraining + 512×512 finetune).
Model Size
Notes
Footprint at default image_size=224 exceeds what a 16 GB host
can comfortably instantiate (≈ 6 GB of params alone before
activations); use a larger GPU. Building this variant is
reserved for inference on dedicated hardware.