4 ms·
I suppose the author chose JPEG because that's easy to turn into MJPEG which is decoded for free (since you can just hand it off to something that supports it),
by sdflhasjd 3y ago
I suppose the author chose JPEG because that's easy to turn into MJPEG which is decoded for free (since you can just hand it off to something that supports it), but I suspect that contributes a lot towards the straining of the reMarkable CPU.
However, JPEG is more suitable for photography but the graphics displayed on the reMarkable are more illustration-like and to top it off, it's monochrome. I think another common image format (such as PNG), or even a crude RLE compression would be lighter on the CPU.
- owulveryck 3y agoHello, You are right, this is indeed why I choose JPEG in the first place. But this is also why I chose a client/server apporach: the encoding was done on the client (my laptop) instead of the tablet. Threrefore the encoding did not impact the CPU. I made some profiling, most of the CPU was used by the transfer of the data over the wire. This is the reason of the compression. Now, the CPU footprint is low.
- sdflhasjd 3y agoOops, I misunderstood where the JPEG was being created and completely missed your own section on RLE. The resuling 200K per frame does still seem a bit high though, I'm sure that could be reduced further
- owulveryck 3y agoI agree with you. It’s all about trading memory for computation. If you have any idea, feel free to share it. Thanks for the comments
- idorosen 3y agoTake a look at how mosh transfers deltas of terminal viewports over the wire using what it calls “SSP”. That protocol might have some advantages here, especially since you can access the state of the pre-rasterization drawn objects, not just the pixels, on the screen. Once you do that, you may obviate the need for any transcoding or conversion to MJPEG since you can just redraw the objects on the canvas. Also, RM2 seems to have a built in Screen Share feature. Might be worth describing the differences (besides not needing their cloud subscription service).
- owulveryck 3y agoI will try to answer to both points: In the first article, I described how I fetched the picture by reading the virtual framebuffer. I have not any knowledge of what’s being drawn. All I have from the beginning is a 2.5Mb byte array. I don’t use any jpeg compression anymore in this version And my understanding is that the native client is transmitting the vector representation to the client and the client redraws it with the same algorithm. It is only doable if you know what algorithm they use. I did a small test to decode their format, but it may change more often than the format of the picture. Does it provide you the answer? (thanks for the conversation)
- CaveTech 3y agoWithout knowing how often your RLE is hitting the max length of 16, but assuming it was often, a further optimization could be using one bit as a flag and to signal that the following block is pixel is either a small sequence of 1-8 pixels, or a large sequence of a multiple of 8 pixels (ie. 1 = 8x1, 2 = 8x2, 3 = 8x3). This lets you compress up to 64 pixels into the space of no more than 2.
- owulveryck 3y agoSounds like a good idea
- TuringTest 3y agoBeing nitpicky here, but the reMarkable is not monochrome, it's grayscale (with 16 gray levels iirc). It also has colour inks that will show in the companion app as blue or red for pens, yellow or green for highlighter. And the proprietary file format is keystroke-based, not bitmap.