Probability flow ODEs offer a powerful and elegant way to describe the generative process in diffusion models. Instead of treating the reverse process as a stochastic differential equation (SDE) that samples data by reversing the corruption of noise, the probability flow ODE reformulates this process into a deterministic ordinary differential equation (ODE). This means you can map pure noise to data points without the randomness of sampling at every step, enabling exact likelihood computation and more controlled generation.

To understand how probability flow ODEs arise, recall that in the SDE formulation of diffusion models, you have a forward process that gradually adds noise to the data, and a reverse SDE that removes this noise. The reverse SDE typically takes the form:

where $f(x, t)$ and $g(t)$ are drift and diffusion coefficients, and $d\bar{w}$ is a Wiener process.

The key insight is that you can construct an ODE that shares the same marginal distributions as the SDE at every time $t$. By removing the stochastic term and adjusting the drift, you get the probability flow ODE:

This ODE deterministically transports noise to data, following the probability flow of the underlying SDE. The term $\nabla_x \log p_t(x)$ is known as the score function, typically learned by the model. By integrating this ODE from pure noise at $t=1$ to data at $t=0$, you can generate samples without random noise at each step.