Flow Matching

A continuous normalizing flow is defined by a vector field: push samples of a simple distribution along $d\mathbf{x}/dt = v_t(\mathbf{x})$ and read off what arrives. Training one by maximum likelihood, as Neural ODE does, means solving that ODE on every gradient step — the cost that kept continuous flows off the scale diffusion models reached.

Flow Matching removes the solve. Suppose a probability path $p_t$ interpolating noise and data were known, together with the field $u_t$ that generates it; then a regression would suffice,

$$\mathcal{L}_{\mathrm{FM}}(\theta) = \mathbb{E}_{t, p_t(x)} \bigl\| v_t(x; \theta) - u_t(x) \bigr\|^2 .$$

Neither $p_t$ nor $u_t$ is available in closed form. The paper's key result — Theorem 2 — is that they need not be. Build the path per data point, as a mixture of conditional paths $p_t(x \mid x_1)$ each generated by a tractable $u_t(x \mid x_1)$, and regress against the conditional field instead:

$$\mathcal{L}_{\mathrm{CFM}}(\theta) = \mathbb{E}_{t, q(x_1), p_t(x \mid x_1)} \bigl\| v_t(x; \theta) - u_t(x \mid x_1) \bigr\|^2 .$$

The two objectives differ by a constant, so $\nabla_\theta \mathcal{L}_{\mathrm{FM}} = \nabla_\theta \mathcal{L}_{\mathrm{CFM}}$. Training is now a regression against a target with a closed form: simulation-free.

For Gaussian conditional paths $p_t(x \mid x_1) = \mathcal{N}(x \mid \mu_t(x_1), \sigma_t(x_1)^2 I)$, Theorem 3 names the unique generating field,

$$u_t(x \mid x_1) = \frac{\sigma_t'(x_1)}{\sigma_t(x_1)} \bigl(x - \mu_t(x_1)\bigr) + \mu_t'(x_1),$$

so a path is fully specified by two scalar schedules and their derivatives. Two instances matter. The diffusion path recovers the variance-preserving process score-based models use — the probability path they were already implicitly training, now reachable by regression. The optimal-transport path takes the schedules straight,

$$\mu_t(x_1) = t\,x_1, \qquad \sigma_t(x_1) = 1 - (1 - \sigma_{\min})\,t,$$

which makes each conditional flow a constant-speed straight line between a noise sample and a data point, and collapses the regression target to something that does not depend on $t$ at all:

$$\mathcal{L}_{\mathrm{CFM}}(\theta) = \mathbb{E}_{t, q(x_1), p(x_0)} \bigl\| v_t(\psi_t(x_0); \theta) - \bigl(x_1 - (1 - \sigma_{\min})\,x_0\bigr) \bigr\|^2, \qquad \psi_t(x_0) = \bigl(1 - (1 - \sigma_{\min})t\bigr) x_0 + t\,x_1 .$$

Diffusion paths bend; these do not, and the particles never overshoot the target they are heading for. Straighter trajectories are cheaper for a solver to follow, which is why sampling needs fewer function evaluations and why the same solver reaches a better FID.

Time runs from noise to data: $p_0 = \mathcal{N}(0, I)$ and $p_1 \approx q$. Sampling integrates $v_t$ from $t = 0$ to $t = 1$; likelihood integrates the same field backwards while accumulating $-\operatorname{tr}(\partial v / \partial x)$, exactly as any continuous flow does.