DETR

DETR (DEtection TRansformer) reframes object detection as a direct set prediction problem. A CNN backbone produces a feature map $f \in \mathbb{R}^{C \times H \times W}$, which is flattened, projected to $d_{\mathrm{model}}$, augmented with 2-D sinusoidal positional encodings, and fed into a standard Transformer encoder–decoder. The decoder additionally takes $N$ learnable object queries; each query produces one detection through FFN heads (class logits + normalised box $(c_x, c_y, w, h)$ with a sigmoid output).

Training enforces a one-to-one assignment between predictions and ground-truth objects via the Hungarian algorithm. The optimal permutation $\hat{\sigma}$ over $N$ slots minimises

where the per-pair cost combines class probability and a box-similarity term (an $\ell_1$ distance plus generalised IoU). Given the matched assignment, the final training loss is

Because each query is matched to at most one ground-truth object, DETR needs neither anchor priors, nor a separate proposal stage, nor non-maximum suppression at inference — duplicates are suppressed by the bipartite matching during training. The result is the first fully end-to-end detector with a Transformer trunk; later variants (Deformable DETR, DAB-DETR, DINO, …) build on this set-prediction framing while accelerating convergence.