Attention relationship

An attention relationship is the connection between two tokens in the context window. When a model processes a request, each token looks at every other token and assigns it a weight: how much does that one matter for understanding this one? The variable name on line 40 and its declaration on line 3 have a strong relationship; the variable and a log line from an unrelated test run have a weak one. This pairwise weighing is the attention mechanism that transformers are built on.

The number of relationships grows much faster than the context does. Double the tokens and you roughly quadruple the pairs the model has to sort through, and the pairs that carry real meaning (a bug and its cause, a rule and the code it governs) become a smaller and smaller share of the total. Nothing about the important pairs got weaker; they are simply outnumbered.

You do not manage attention relationships directly, but you shape them by what you load. A tight context with only the relevant files keeps the meaningful pairs dense; a sprawling session with dozens of unrelated reads buries them. This is the mechanism underneath attention budget and attention degradation, and the reason 'just load more' so often makes results worse.