If latency is critical then you might want to consider a memory-in-pixel display instead of e-ink. They don't go down to zero-power on static content like e-ink

If latency is critical then you might want to consider a memory-in-pixel display instead of e-ink. They don't go down to zero-power on static content like e-ink does, but they are very efficient, very quick to update and don't suffer from ghosting like e-ink does. If you've seen the Pebble smartwatches, Garmins non-OLED watches or the Playdate console, those are all using MIP displays.