Whoops, yeah it's been a while since I worried about stuff like this in javascript. I guess what I remembered doing was keeping track of cursor position in a gl

Whoops, yeah it's been a while since I worried about stuff like this in javascript. I guess what I remembered doing was keeping track of cursor position in a global, updated on every mousemove to simulate immediate mode in a requestAnimationFrame callback. But as you say, that retains all the disadvantages of event latency. requestAnimationFrame is designed for immediate mode rendering, so it should probably pass its callback some information about input states or offer some other way of getting that.