To understand the training of Generative Adversarial Networks (GANs), you need to look at their mathematical foundation. The process is formalized as a minimax game between two neural networks: the generator ($G$) and the discriminator ($D$). The objective function that governs this game is:

This equation describes how $G$ and $D$ are trained in opposition to each other. The generator tries to produce data that looks like the real data, while the discriminator tries to distinguish between real and generated data.

In this context, the expectations $\mathbb{E}_{x \sim p_{data}}$ and $\mathbb{E}_{z \sim p_z}$ represent averages over samples drawn from specific probability distributions. The distribution $p_{data}$ is the true data distribution, meaning real examples from your dataset. The distribution $p_z$ is a simple, known distribution such as uniform or normal, from which you sample random noise vectors $z$.

The generator, $G(z)$, takes a noise vector $z$ and transforms it into a synthetic data sample. The discriminator, $D(x)$, takes a data sample $x$ (which can be real or generated) and outputs the probability that $x$ is real. The loss function combines two terms: one that rewards the discriminator for correctly identifying real data, and one that rewards it for correctly identifying fake data generated by $G$.

To see how the loss is computed, imagine a single training step. First, you sample a batch of real data $x$ from $p_{data}$ and a batch of noise $z$ from $p_z$. The discriminator receives both real and generated samples. For each real sample, it computes $D(x)$, and for each fake sample $G(z)$, it computes $D(G(z))$.

The discriminator's loss is the sum of the log-probabilities for real samples being classified as real and fake samples being classified as fake:

The generator aims to make $D(G(z))$ close to 1, so its loss is:

Both networks update their parameters using gradients from these losses, leading to the adversarial process that defines GAN training.