RoPE: rotary position embedding

A · 10-second intuition RoPE rotates each query and key by an angle that grows with its position, so their dot product depends on how far apart they are.