To understand why reverse diffusion is such a challenging problem, you first need to recall the nature of the forward diffusion process. In the forward process, you start with a clean data point and gradually corrupt it by adding small amounts of noise at each step. This process is straightforward: each step is well-defined and easy to compute, as you simply inject more noise according to a known schedule. However, reversing this process — starting from pure noise and trying to recover the original data — is not simply a matter of "subtracting" the noise. This is because, at each step in the forward process, information about the original data is lost and the noisy observation becomes increasingly ambiguous. The reverse process is therefore ill-posed: for any given noisy observation, there are many possible original data points that could have produced it. Without additional information or modeling, you cannot uniquely recover the original data from the noise.

Mathematically, the intractability of the reverse process becomes clear when you consider the probability distributions involved. In the forward process, you can write the transition as $q(x_t | x_{t-1})$, which is typically a simple Gaussian distribution with known parameters. But the reverse transition, $q(x_{t-1} | x_t)$, is not directly accessible. To compute it, you would need to know the true posterior distribution over all possible previous states, given the current noisy state. This posterior depends on the entire data distribution, which is unknown and highly complex in real-world scenarios. As a result, you cannot write down or sample from the reverse process analytically, and any attempt to do so without learning or modeling will fail to recover realistic data.

This is where Denoising Diffusion Probabilistic Models (DDPM) offer a solution. Instead of trying to analytically invert the diffusion process, DDPMs introduce a learnable model — typically a neural network — that is trained to approximate the reverse transitions. The core idea is to parameterize the reverse process as $p_θ(x_{t-1} | x_t)$, where the parameters θ are learned from data. During training, the model is shown pairs of noisy and less - noisy samples and learns to predict either the original data or the noise that was added. By optimizing an appropriate loss function, DDPMs gradually learn how to denoise even heavily corrupted samples, effectively modeling the complex posterior required for the reverse process. This approach makes it possible to generate realistic data from pure noise, overcoming the ill-posedness and intractability of the true reverse diffusion.