EfficientFormer
12 memberslucid.models.vision.efficientformerEfficientFormer — Li et al., 2022.
Li, Yanyu, et al. "EfficientFormer: Vision Transformers at MobileNet Speed." Advances in Neural Information Processing Systems, vol. 35, 2022.
EfficientFormer is a mobile-grade vision backbone designed so that its end-to-end on-device latency, rather than its FLOP count, matches the latency of MobileNet on the same hardware while retaining transformer-level accuracy. The authors profile real iOS / NPU latency and identify three transformer operators that are unexpectedly slow: the reshape between spatial and token layouts, GeLU compared to ReLU, and LayerNorm compared to BatchNorm. EfficientFormer is the result of a latency-aware architecture search that designs around these costs.
The backbone is a four-stage pyramid. Stages 1-3 use pooling mixer blocks — a MetaFormer-style token mixer that replaces the quadratic self-attention with a parameter-free average pooling — and operate entirely in the 4-D layout so no expensive reshape is needed. Each pooling block computes
with initialised to — the layer scale trick from CaiT. Stage 4 is the only stage that pays the cost of a reshape and runs standard multi-head self-attention with relative position bias on the now-tiny token grid. The resulting design (variants L1 / L3 / L7) matches ResNet-50 / DeiT-S accuracy at MobileNetV2-class latency on an iPhone 12 NPU.
Classes
Functions
efficientformer_l1→ EfficientFormerEfficientFormer-L1 backbone (Li et al., 2022).
efficientformer_l1_cls→ EfficientFormerForImageClassificationEfficientFormer-L1 image classifier (Li et al., 2022).
efficientformer_l3→ EfficientFormerEfficientFormer-L3 backbone (Li et al., 2022).
efficientformer_l3_cls→ EfficientFormerForImageClassificationEfficientFormer-L3 image classifier (Li et al., 2022).
efficientformer_l7→ EfficientFormerEfficientFormer-L7 backbone (Li et al., 2022).
efficientformer_l7_cls→ EfficientFormerForImageClassificationEfficientFormer-L7 image classifier (Li et al., 2022).