Self-attention — AI Dictionary

The mechanism where every token compares itself to every other token to decide which ones are relevant right now.