Fast R-CNN

Fast R-CNN closes the dominant performance gap of R-CNN — the one-CNN-forward-per-proposal cost — by sharing convolutional computation across all proposals in an image.

The CNN is run once on the full image, producing a feature map at stride $s$ (e.g. $s = 16$ for VGG-16). For each region proposal, RoI Pooling extracts a fixed-size $H \times W$ (paper: $7 \times 7$) sub-window from the shared feature map by quantising the proposal coordinates to the feature grid and max-pooling adaptively inside each cell.

The pooled features feed two FC layers and two sibling output heads:

where $L_{\mathrm{cls}}$ is softmax log-loss over $K+1$ classes and $L_{\mathrm{loc}}$ is the smooth $\ell_1$ regression loss on the parameterised offsets $t^u$ for the predicted class $u$. The Iverson bracket $[u \ge 1]$ disables regression on background RoIs.

Beyond the speedup (≈9× training, ≈213× inference over R-CNN), Fast R-CNN replaces the SVM + post-hoc regressor with a single multi-task network, which improves accuracy because gradients from both heads jointly tune the shared trunk.