data

EfficientFormerConfig

extendsModelConfig

EfficientFormerConfig(num_classes: int = 1000, in_channels: int = 3, depths: tuple[int, ...] = (3, 2, 6, 4), embed_dims: tuple[int, ...] = (48, 96, 224, 448), mlp_ratios: tuple[float, ...] = (4.0, 4.0, 4.0, 4.0), num_vit: int = 1, pool_size: int = 3, key_dim: int = 32, num_heads: int = 8, attn_ratio: float = 4.0, resolution: int = 7, drop_path_rate: float = 0.0, layer_scale_init: float = 1e-05)

source edit

Configuration dataclass for every EfficientFormer variant.

EfficientFormerConfig is an immutable container that fully specifies the architecture of an EfficientFormer (Li et al., 2022). The model is a mobile-grade vision backbone whose on-device latency, rather than its FLOP count, matches MobileNet on the same hardware while retaining transformer-level accuracy.

Stages 1-3 use pooling mixer MetaFormer blocks (a parameter-free $3 \times 3$ average pool replacing self-attention) and operate entirely in $(B, C, H, W)$ layout. Stage 4 is the only stage that pays the cost of a reshape and runs standard multi-head self-attention on the now-tiny token grid.

Parameters

num_classesint= 1000

Number of output classes for the classification head. Defaults to 1000 (ImageNet-1k).

in_channelsint= 3

Number of input image channels. Defaults to 3 (RGB).

depthstuple of int= (3, 2, 6, 4)

Number of blocks per stage. Defaults to (3, 2, 6, 4) (EfficientFormer-L1).

embed_dimstuple of int= (48, 96, 224, 448)

Output channel width per stage. Defaults to (48, 96, 224, 448) (EfficientFormer-L1).

mlp_ratiostuple of float= (4.0, 4.0, 4.0, 4.0)

Per-stage MLP expansion ratio. Defaults to (4.0, 4.0, 4.0, 4.0).

num_vitint= 1

Number of trailing blocks in the last stage that run 3-D multi-head self-attention (the remainder run the 4-D pooling token mixer). Defaults to 1 (EfficientFormer-L1); L3 uses 4 and L7 uses 8.

pool_sizeint= 3

Spatial extent of the average-pool token mixer in the 4-D pooling blocks. Defaults to 3.

key_dimint= 32

Per-head query/key dimension in the attention token mixer. Defaults to 32 (fixed across all paper variants).

num_headsint= 8

Number of attention heads in the attention token mixer. Defaults to 8 (fixed across all paper variants).

attn_ratiofloat= 4.0

Value/key dimension ratio in the attention token mixer; the per-head value dimension is attn_ratio * key_dim. Defaults to 4.0.

resolutionint= 7

Side length of the (square) token grid feeding the attention stage; the learned attention-bias table has resolution ** 2 entries per head. Defaults to 7 (a 7 x 7 grid at the default 224 input).

drop_path_ratefloat= 0.0

Maximum stochastic-depth rate; linearly scheduled across all blocks of the trunk. The paper uses 0.0 for L1, 0.1 for L3, 0.2 for L7. Defaults to 0.0.

layer_scale_initfloat= 1e-05

Initial value of the layer-scale parameter on every residual branch. Defaults to 1e-5 (Li et al., 2022, appendix).

Attributes

model_typeClassVar[str]

Constant string "efficientformer" used by the model registry.

Notes

The canonical variants registered as factory functions in lucid.models.vision.efficientformer are:

============================ ================ =================== Variant depths embed_dims ============================ ================ =================== EfficientFormer-L1 (3, 2, 6, 4) (48, 96, 224, 448) EfficientFormer-L3 (4, 4, 12, 6) (64, 128, 320, 512) EfficientFormer-L7 (6, 6, 18, 8) (96, 192, 384, 768) ============================ ================ ===================

Reference: Yanyu Li et al., "EfficientFormer: Vision Transformers at MobileNet Speed", NeurIPS 2022, arXiv:2206.01191.

Examples

Build an EfficientFormer-L1 configuration with a 100-class head:
>>> from lucid.models.vision.efficientformer import EfficientFormerConfig
>>> cfg = EfficientFormerConfig(num_classes=100)
>>> cfg.depths, cfg.embed_dims, cfg.num_classes
((3, 2, 6, 4), (48, 96, 224, 448), 100)

Used by 3

Constructors

dunder

init

→None

__init__(num_classes: int = 1000, in_channels: int = 3, depths: tuple[int, ...] = (3, 2, 6, 4), embed_dims: tuple[int, ...] = (48, 96, 224, 448), mlp_ratios: tuple[float, ...] = (4.0, 4.0, 4.0, 4.0), num_vit: int = 1, pool_size: int = 3, key_dim: int = 32, num_heads: int = 8, attn_ratio: float = 4.0, resolution: int = 7, drop_path_rate: float = 0.0, layer_scale_init: float = 1e-05)

source edit

EfficientFormerConfig(num_classes: int = 1000, in_channels: int = 3, depths: tuple[int, ...] = (3, 2, 6, 4), embed_dims: tuple[int, ...] = (48, 96, 224, 448), mlp_ratios: tuple[float, ...] = (4.0, 4.0, 4.0, 4.0), num_vit: int = 1, pool_size: int = 3, key_dim: int = 32, num_heads: int = 8, attn_ratio: float = 4.0, resolution: int = 7, drop_path_rate: float = 0.0, layer_scale_init: float = 1e-05)

Build an EfficientFormer-L1 configuration with a 100-class head: >>> from lucid.models.vision.efficientformer import EfficientFormerConfig >>> cfg = EfficientFormerConfig(num_classes=100) >>> cfg.depths, cfg.embed_dims, cfg.num_classes ((3, 2, 6, 4), (48, 96, 224, 448), 100)

__init__(num_classes: int = 1000, in_channels: int = 3, depths: tuple[int, ...] = (3, 2, 6, 4), embed_dims: tuple[int, ...] = (48, 96, 224, 448), mlp_ratios: tuple[float, ...] = (4.0, 4.0, 4.0, 4.0), num_vit: int = 1, pool_size: int = 3, key_dim: int = 32, num_heads: int = 8, attn_ratio: float = 4.0, resolution: int = 7, drop_path_rate: float = 0.0, layer_scale_init: float = 1e-05)