3 ms·

The thing is, the ReMarkable isn't doing anything special hardware-wise to achieve that latency, as they're based on NXP reference designs with the same integra

The thing is, the ReMarkable isn't doing anything special hardware-wise to achieve that latency, as they're based on NXP reference designs with the same integrated EPD every other E-ink device is using (besides the Android-based devices, of course). The magic is essentially that they have nothing running on the device except a Qt app which writes directly to the framebuffer, and they have an extremely simplified framebuffer driver that allows the Qt app to implement the complex parts of updating an E-ink screen in userspace. The Wacom input is standard, the other kernel bits are standard, and they're even running systemd.

danShumway 5y ago
That's really positive if that's the case -- I was under the impression that some of this had to do with the hardware itself. At the very least though, aren't they are doing some special stuff with how they mount the screen and where they're sourcing their plastic texture cover from? I thought that offset was one of the big reasons they didn't have a backlight -- which the Pinenote will have I think. But this makes me feel a lot more confident about the PineNote if most of the latency at least comes down to framebuffer access.
Groxx 5y ago
Yeah - the big advantage they have is that they're not running Android. A greatly simplified and optimized stack is really all you need to get stupidly fast black<->white updates to small eink regions, and that fits absolutely perfectly with all their tools[1] and for writing purposes. It takes a fairly impressive effort and attention to detail to get to the "whole package" they have, but it's nothing that can't be replicated on other standard hardware. [1]: They're all pure black and white and don't change once drawn (no smoothing a frame later), e.g. the pencil has varying fine texture but notably it does not use any actual grays for lighter shading - just pure-black texture density. Even the "gray" mode for tools is just a 50% black/white checkerboard regardless of zoom level, and tbh it looks sorta weird and shimmery because of it. Everything is relentlessly optimized for rendering speed. When you get into true grays like with text in ebooks, the page-turns are normal for e-ink, though it has a respectably quick "gesture -> start transition" time... especially if you compare to cheaper devices. Some of those can lag for a second or more before they respond. That has nothing to do with eink (and everything to do with bad optimization) and it contributes a ton to the slow feel of eink devices.
- chrismorgan 5y ago
  I’ve definitely come to appreciate the way in which straight black and white can achieve better results on an e-paper display than greys. Actually it does use real grey in one place: the highlighter. While you draw, it renders in chequered black and white like tools in grey, except that where it overlaps existing black-and-white chequering it turns it into solid black so you can see what you’re doing, and then it redraws the full highlighted area afterwards, with highlighting-on-white being true grey, and highlighting-on-faux-grey becoming, I think (don’t have a microscope or sufficiently capable macro lens handy, and ghosting makes it hard to be sure anyway) darker-faux-grey, chequered black and light grey. If you go using the highlighter extensively as part of drawing, you rapidly discover just how awful the ghosting is until you can trigger a full redraw, which admittedly happens more often once actual greys are in use. reMarkable also uses true greys in antialiasing of its own UI icons and text.
  - Groxx 5y ago
    Yep, good point on the highlighter! I forgot that one. When you overlap grays and layers of highlighter (as it gets darker with each layer... maybe in 1/8ths?), I suspect it's just doing a gray/black checkerboard - I can't tell for sure with one or two layers of highlighter, but it's pretty clear to me on the third layer that there are gray pixels between the black pixels. And yeah, the highlighter's ghosting can get pretty extreme :) Since it clears up with a back/forward page turn I think it's a reasonable tradeoff... but it does serve as an interactive example of how much eink struggles with gray<->grey transition accuracy, which is why anti-aliased text is so much of a problem after a couple pages on literally every device.
rjzzleep 5y ago
It absolutely goes beyond just having the right hardware for pen optimization. I have an old rockchip based eink tablet from onyx here. Yes it's Android based but there are a few interesting things to observe here: 1. Onyx provides a few libraries that give access to a special surfaceview for pen input. 2. The pen drawing latency is completely different between this special input view and and any other native Android drawing app. 3. When sharing the screen from the tablet the surfaceview drawn lines have a delay from when they were drawn to when they should up on the shared screen (this is different from the network latency, it seems like the lines on an optimized surfaceview are drawn on screen before the OS sees it) 4. Remarkable has had its fair share of straight line aliasing problems as well. 5. Even more interesting is that there is a preliminary update to android 9 for the old onyx device and the same surfaceview has a lot of issues with initiating the drawing. That said this device looks fantastic. I've been happily using the onyx devices but I'm glad I finally get a chance to improve the parts that bother me or collaborate with others that do. EDIT: two things I am a bit unclear about is: 1. I wish the SoC was better. This is comparable to an snapdragon 450. It doesn't really matter for an SBC but for a tablet it would be nice to have something better. I'd gladly pay more. 2. At some point there were extra chips used for the display refresh. Is that nowadays part of the display assembly?
- tadfisher 5y ago
  > At some point there were extra chips used for the display refresh. Is that nowadays part of the display assembly? Yep, E-Ink has moved to an in-display EPD that accepts a more traditional framebuffer and leaves it up to the driver/userspace to perform the temperature/waveform/partial-update logic. This is partially why it's been so difficult to reverse-engineer the ReMarkable 2 display, because the closed-source ReMarkable app now owns the framebuffer device and handles absolutely everything related to refresh.

danShumway 5y ago

That's really positive if that's the case -- I was under the impression that some of this had to do with the hardware itself. At the very least though, aren't they are doing some special stuff with how they mount the screen and where they're sourcing their plastic texture cover from? I thought that offset was one of the big reasons they didn't have a backlight -- which the Pinenote will have I think. But this makes me feel a lot more confident about the PineNote if most of the latency at least comes down to framebuffer access.

Groxx 5y ago

Yeah - the big advantage they have is that they're not running Android. A greatly simplified and optimized stack is really all you need to get stupidly fast black<->white updates to small eink regions, and that fits absolutely perfectly with all their tools[1] and for writing purposes. It takes a fairly impressive effort and attention to detail to get to the "whole package" they have, but it's nothing that can't be replicated on other standard hardware. [1]: They're all pure black and white and don't change once drawn (no smoothing a frame later), e.g. the pencil has varying fine texture but notably it does not use any actual grays for lighter shading - just pure-black texture density. Even the "gray" mode for tools is just a 50% black/white checkerboard regardless of zoom level, and tbh it looks sorta weird and shimmery because of it. Everything is relentlessly optimized for rendering speed. When you get into true grays like with text in ebooks, the page-turns are normal for e-ink, though it has a respectably quick "gesture -> start transition" time... especially if you compare to cheaper devices. Some of those can lag for a second or more before they respond. That has nothing to do with eink (and everything to do with bad optimization) and it contributes a ton to the slow feel of eink devices.

chrismorgan 5y ago
I’ve definitely come to appreciate the way in which straight black and white can achieve better results on an e-paper display than greys. Actually it does use real grey in one place: the highlighter. While you draw, it renders in chequered black and white like tools in grey, except that where it overlaps existing black-and-white chequering it turns it into solid black so you can see what you’re doing, and then it redraws the full highlighted area afterwards, with highlighting-on-white being true grey, and highlighting-on-faux-grey becoming, I think (don’t have a microscope or sufficiently capable macro lens handy, and ghosting makes it hard to be sure anyway) darker-faux-grey, chequered black and light grey. If you go using the highlighter extensively as part of drawing, you rapidly discover just how awful the ghosting is until you can trigger a full redraw, which admittedly happens more often once actual greys are in use. reMarkable also uses true greys in antialiasing of its own UI icons and text.
- Groxx 5y ago
  Yep, good point on the highlighter! I forgot that one. When you overlap grays and layers of highlighter (as it gets darker with each layer... maybe in 1/8ths?), I suspect it's just doing a gray/black checkerboard - I can't tell for sure with one or two layers of highlighter, but it's pretty clear to me on the third layer that there are gray pixels between the black pixels. And yeah, the highlighter's ghosting can get pretty extreme :) Since it clears up with a back/forward page turn I think it's a reasonable tradeoff... but it does serve as an interactive example of how much eink struggles with gray<->grey transition accuracy, which is why anti-aliased text is so much of a problem after a couple pages on literally every device.

rjzzleep 5y ago

It absolutely goes beyond just having the right hardware for pen optimization. I have an old rockchip based eink tablet from onyx here. Yes it's Android based but there are a few interesting things to observe here: 1. Onyx provides a few libraries that give access to a special surfaceview for pen input. 2. The pen drawing latency is completely different between this special input view and and any other native Android drawing app. 3. When sharing the screen from the tablet the surfaceview drawn lines have a delay from when they were drawn to when they should up on the shared screen (this is different from the network latency, it seems like the lines on an optimized surfaceview are drawn on screen before the OS sees it) 4. Remarkable has had its fair share of straight line aliasing problems as well. 5. Even more interesting is that there is a preliminary update to android 9 for the old onyx device and the same surfaceview has a lot of issues with initiating the drawing. That said this device looks fantastic. I've been happily using the onyx devices but I'm glad I finally get a chance to improve the parts that bother me or collaborate with others that do. EDIT: two things I am a bit unclear about is: 1. I wish the SoC was better. This is comparable to an snapdragon 450. It doesn't really matter for an SBC but for a tablet it would be nice to have something better. I'd gladly pay more. 2. At some point there were extra chips used for the display refresh. Is that nowadays part of the display assembly?

tadfisher 5y ago
> At some point there were extra chips used for the display refresh. Is that nowadays part of the display assembly? Yep, E-Ink has moved to an in-display EPD that accepts a more traditional framebuffer and leaves it up to the driver/userspace to perform the temperature/waveform/partial-update logic. This is partially why it's been so difficult to reverse-engineer the ReMarkable 2 display, because the closed-source ReMarkable app now owns the framebuffer device and handles absolutely everything related to refresh.