When you explore autoencoders, you encounter a fundamental pattern for learning useful data representations: the encoder-decoder architecture. This can be visualized as a simple information flow diagram:

Here, x represents the original input data, such as an image or a vector. The Encoder is a neural network that transforms this input into a compressed, lower-dimensional representation called z—the latent code. This latent code passes through what is known as the bottleneck. The Decoder then attempts to reconstruct the original input, producing x̂ (the reconstruction) from this compact code. The entire process encourages the model to distill the essential features of the input into z so that the decoder can recover the input as accurately as possible.

By requiring all information about $x$ to pass through the bottleneck, the model cannot rely on simply copying the input. Instead, the encoder must learn to extract and encode only the most salient, or important, features of the data. The decoder then uses this compressed information to reconstruct $x̂$. This selective compression is crucial: it helps the autoencoder capture underlying patterns rather than noise or irrelevant details. The architecture diagram above highlights how the bottleneck acts as a filter, ensuring that only the most representative aspects of the data reach the decoder.

The width of the bottleneck — how many units or dimensions $z$ contains — has a profound impact on the model's behavior. You can compare wide and narrow bottlenecks to understand their effects: