EfficientNet

EfficientNet is built on the empirical observation that the three axes used to scale a ConvNet — depth $d$ (number of layers), width $w$ (number of channels per layer), and input resolution $r$ — are not independent. Scaling any one of them in isolation gives rapidly diminishing returns, but scaling all three together along a carefully chosen ratio gives a much better accuracy/FLOPs frontier.

The paper formalises this as compound scaling: a single user-facing scalar $\phi$ controls all three axes through

subject to the FLOPs-balance constraint $\alpha \cdot \beta^2 \cdot \gamma^2 \approx 2$. The constants $\alpha, \beta, \gamma$ are found once by a small grid search on the baseline network (giving roughly $\alpha = 1.2, \beta = 1.1, \gamma = 1.15$); from then on, a single $\phi$ indexes the whole family. Successive integer values produce the standard B0 → B7 variants, each requiring approximately $2^\phi$ more FLOPs than B0.

The baseline B0 itself is not manually designed. It is the output of a multi-objective neural-architecture search that optimises accuracy and FLOPs jointly. Each block is an inverted-residual MBConv (MobileNet-v2 style) augmented with a squeeze-and-excitation module, and the network uses the swish activation throughout. Stochastic depth with linearly increasing drop rate (the drop_connect_rate field) acts as a strong regulariser as depth grows, which is what allows B6/B7 to keep gaining accuracy where naively-deepened baselines would overfit or fail to train.