Self-Attention Mechanism

An advancement used in transformer networks allowing simultaneous consideration of every token in a sequence to understand context and relationships, improving NLP tasks.