Focus Weight Hunt

Focus Weight Hunt is an attention mechanism game that teaches the algorithm inside every transformer model ever built — not through equations, but through an interactive heat map you control with your hands. The attention mechanism allows a language model to resolve ambiguity, track pronouns, match verbs to their subjects across long distances, and understand the same word differently in different contexts. It does this by computing, for every token in a sequence, a weight distribution over every other token — how much should this word attend to every other word when building its representation? The weights must sum to one. Competition for attention is the mechanism. The game gives you the heat map and asks you to fill it in. A sentence appears. A target word is highlighted. You set the attention weights by tapping a grid — dim cells for low attention, vivid amber for high. The softmax…