Keys, queries, and values are all vectors during the LLMs. RoPE [sixty six] requires the rotation from the question and key representations at an angle proportional for their absolute positions of the tokens inside the enter sequence. What can be achieved to mitigate such challenges? It