MobileNet V2

MobileNet-v2 replaces the plain depthwise-separable stack of v1 with the inverted residual bottleneck block. Each block receives a low-dimensional tensor of $c$ channels, projects it up to a much wider $t \cdot c$ channel space with a $1 \times 1$ convolution, applies a $3 \times 3$ depthwise convolution in that expanded space, then projects back down to $c'$ channels with another $1 \times 1$. When the input and output shapes match, an identity shortcut is added around the whole block:

This is the inverse of the classical bottleneck shape (which is wide → narrow → wide), and is why the authors call it an "inverted residual".

The second insight is the linear bottleneck. The final $1 \times 1$ projection is followed by no non-linearity — a deliberate departure from the ReLU-after-every-conv pattern. The paper's manifold argument shows that ReLU on a low-dimensional activation collapses information that cannot be recovered; keeping the bottleneck output linear preserves the representational capacity of the narrow channels while still letting the wide interior of the block use ReLU6 freely.

These two ideas together give a network that, at comparable accuracy to MobileNet-v1, uses roughly $30\%$ fewer parameters and $30\%$ fewer multiply-adds, and runs measurably faster on mobile CPUs. As with v1, a single width multiplier $\alpha$ scales every channel count uniformly, exposing the same accuracy/latency trade-off curve without retraining.