Learn Probabilistic Latent Variables In VAEs

Autoencoders and Representation Learning

When working with variational autoencoders (VAEs), you move beyond mapping each input to a single point in the latent space. Instead, VAEs use a probabilistic encoding: each input is mapped to a probability distribution over possible latent variables. Rather than compressing an input directly into a fixed vector, the encoder produces two outputs for each input: a mean ( $μ$ ) and a variance ( $σ²$ ). These parameters define a normal distribution in the latent space for each data point, allowing the model to capture uncertainty and variability in how inputs are represented.

Definition

A probabilistic latent variable is a variable in a model that is not assigned a fixed value, but rather is drawn from a probability distribution. In VAEs, this approach enables the model to generate diverse outputs and capture the inherent randomness in data, which is crucial for effective generative modeling.

Mathematically, this process is written as:

z \sim N(μ(x), σ²(x))

Here, $z$ is the latent variable, and it is sampled from a normal distribution whose mean $μ(x)$ and variance $σ²(x)$ are both functions of the input $x$ . This means that for each data point, the encoder network outputs the parameters for its own unique distribution in the latent space, rather than a single deterministic code.