When attention-based generative models produce text, images, or other outputs, they sometimes generate information that is not grounded in the input data or context. This phenomenon is known as hallucination — the model creates plausible but incorrect or fabricated content. Another related issue is context drift, where the model gradually loses track of the relevant information as it generates longer sequences. Both of these failures become particularly pronounced during long generations, challenging the reliability of attention mechanisms in extended tasks.

The accumulation of uncertainty and the decay of relevance are central to understanding why hallucinations and context drift occur. As each new token or output is generated, the model relies on its internal attention weights to determine which parts of the previous context are most important. Over time, small errors or ambiguities in these attention patterns can build up, leading to greater uncertainty about what the model should focus on. Simultaneously, as the sequence grows longer, the model's ability to retain and prioritize the original context diminishes. This relevance decay means that the model may begin to attend to less relevant or even irrelevant information. The combination of these two processes — uncertainty accumulation and relevance decay — makes it increasingly likely that the model will produce hallucinated content or lose track of the intended context, especially in long-form generation tasks.