3 ms·
Hadn't heard of folks using 2D RoPE before. Is that new work here, or an established technique? (The way the authors present it, makes it seem like they are in
by jasonjmcghee 3y ago
Hadn't heard of folks using 2D RoPE before. Is that new work here, or an established technique?
(The way the authors present it, makes it seem like they are introducing it as a concept, but they never outright say they came up with it, AFAICT)
- alterframe 3y agoI didn't go through all literature, but it's possible that some other team focused on 1D, made one toy experiment with 2D just for giggles. That happens a lot, but it's just my speculation that this happened in this case.