4 ms·

Slides: https://filmroellchen.eu/talks/QOI%20EH25/#/0/1 https://filmroellchen.eu/talks/QOI%20EH25/#/0/1 As that data compression nerd, QOI is both a refresher

Slides: https://filmroellchen.eu/talks/QOI%20EH25/#/0/1 https://filmroellchen.eu/talks/QOI%20EH25/#/0/1 As that data compression nerd, QOI is both a refresher and an annoyance. It serves as a good example that input modelling is very important, so much that QOI almost beats PNG in spite of very suboptimal coding. But QOI also represents a missed opportunity because, well, we can do much better if we are willing to use something like QOI instead of standard formats! Weirdly enough people seem to value only the simplicity when it comes to QOI, which is not the main takeaway I believe. Maybe QOIR [1] might result in a better balanced format in the future though... [1] https://github.com/nigeltao/qoir/ https://github.com/nigeltao/qoir/

staunton 2y ago
Let's say you had a lossless image format that's 20% smaller (on average for pictures people send over networks) than PNG. Let's say it takes 10% more computing power than PNG. Do you stand to make money? What would it be used for? I can't imagine people will start storing their family pictures in a new format they've never heard of which is not supported by any software they use for "just" 20% better compression. Do they even want lossless compression in the first place (if you don't ask them directly and call it that)?
- lifthrasiir 2y ago
  That's the portability bit the presenter mentions, and is a very important concern in practice. But how about recompression? For example many PNG files are suboptimally compressed partly because PNG is an old format and also because many softwares have been too dumb to produce a well-optimized PNG. In that case we may benefit from a transparent recompression, which may be done either by using a better library like libdeflate or by internally using a separate format that can be quickly transformed from and back to PNG. In fact Dropbox did so for JPEG files [1]. When I'm saying "so much better" I was thinking about such opportunities that benefit end users. [1] https://github.com/dropbox/lepton https://github.com/dropbox/lepton
  - staunton 2y ago
    Dropbox apparently abandoned the project. Do you know what their takeaways were from trying to improve the JPEG storage? For example, was it worth it in the end? Did they announce anything? Did they switch to another method or give up on the idea, or do we not know?
    - lifthrasiir 2y ago
      Dropbox apparently still uses Lepton or any successor internally, but the open source version is abandoned because it posed larger maintenance burden than internal projects.
      - staunton 2y ago
        Has Dropbox said this anywhere? Or are you assuming it based on something like "this kind of project is very easy to maintain internally so there's no reason why they would have stopped using it"?
        lifthrasiir 2y ago
        At least at the time of abandonment, Dropbox did say (emphasis mine): > While we did ensure that the reported vulnerabilities don’t affect our internal use of Lepton, we unfortunately don’t have the capacity to properly fix these and future issues in this public repo. As far as I know this is indeed the last known public mention of its use, but given that Lepton was already in use and dropping it would substantially increase its traffic, it is reasonable to assume that its use somehow continues to this day.
        Dylan16807 2y ago
        That's pretty strange. If they're using it in a meaningful way, then they're applying it to arbitrary files, so they'd generally need fixes for almost all the bugs. So what resources would be lacking so badly that they give up on releasing?
deleted 2y ago
[deleted]
Nanopolygon 2y ago
QOI is just a simple filter. It cannot do full compression. In fact, in certain cases it can increase the size instead of compressing. It is unnecessarily overrated, of course, mostly because it is open source. The rest is irrelevant. There is another codec that is as fast as QOI (or even faster and multi-core) but with a much higher compression ratio. HALIC (High Availability Lossless Image Compression). But because it's closed source, it definitely didn't get the attention and respect it should have gotten. And that's why I think it stopped developing. https://github.com/Hakan-Abbas/HALIC-High-Availability-Lossless-Image-Compression https://github.com/Hakan-Abbas/HALIC-High-Availability-Lossl...
- lifthrasiir 2y ago
  A simple filter may still do wonder if it is carefully chosen ;-) The apparent death of HALIC was indeed unfortunate, I've heard that its decoder was less than 1K lines of code so it must have had a carefully designed and optimized context model in its heart. But there are so many other formats that are F/OSS but weren't highlighted enough anyway...
- lblume 2y ago
  For a format to proliferate as a standard, it has to be open source. Nobody cares about 5% smaller images if not everyone can view them.
  - otterley 2y ago
    Historically speaking, that’s not true; WMV and FLV were extremely popular for over a decade in the early 2000s. Closed source formats can, and have, become de facto standards if they solve a pressing need that open standards don’t (or provide a stepwise increase in functionality or performance) and if there’s sufficient market demand.
- ska 2y ago
  >>> In fact, in certain cases it can increase the size instead of compressing. Fwiw, all (lossless) compression algorithms will increase the size of some inputs.

staunton 2y ago

Let's say you had a lossless image format that's 20% smaller (on average for pictures people send over networks) than PNG. Let's say it takes 10% more computing power than PNG. Do you stand to make money? What would it be used for? I can't imagine people will start storing their family pictures in a new format they've never heard of which is not supported by any software they use for "just" 20% better compression. Do they even want lossless compression in the first place (if you don't ask them directly and call it that)?

lifthrasiir 2y ago
That's the portability bit the presenter mentions, and is a very important concern in practice. But how about recompression? For example many PNG files are suboptimally compressed partly because PNG is an old format and also because many softwares have been too dumb to produce a well-optimized PNG. In that case we may benefit from a transparent recompression, which may be done either by using a better library like libdeflate or by internally using a separate format that can be quickly transformed from and back to PNG. In fact Dropbox did so for JPEG files [1]. When I'm saying "so much better" I was thinking about such opportunities that benefit end users. [1] https://github.com/dropbox/lepton https://github.com/dropbox/lepton
- staunton 2y ago
  Dropbox apparently abandoned the project. Do you know what their takeaways were from trying to improve the JPEG storage? For example, was it worth it in the end? Did they announce anything? Did they switch to another method or give up on the idea, or do we not know?
  - lifthrasiir 2y ago
    Dropbox apparently still uses Lepton or any successor internally, but the open source version is abandoned because it posed larger maintenance burden than internal projects.
    - staunton 2y ago
      Has Dropbox said this anywhere? Or are you assuming it based on something like "this kind of project is very easy to maintain internally so there's no reason why they would have stopped using it"?
      - lifthrasiir 2y ago
        At least at the time of abandonment, Dropbox did say (emphasis mine): > While we did ensure that the reported vulnerabilities don’t affect our internal use of Lepton, we unfortunately don’t have the capacity to properly fix these and future issues in this public repo. As far as I know this is indeed the last known public mention of its use, but given that Lepton was already in use and dropping it would substantially increase its traffic, it is reasonable to assume that its use somehow continues to this day.
        Dylan16807 2y ago
        That's pretty strange. If they're using it in a meaningful way, then they're applying it to arbitrary files, so they'd generally need fixes for almost all the bugs. So what resources would be lacking so badly that they give up on releasing?

deleted 2y ago

[deleted]

Nanopolygon 2y ago

QOI is just a simple filter. It cannot do full compression. In fact, in certain cases it can increase the size instead of compressing. It is unnecessarily overrated, of course, mostly because it is open source. The rest is irrelevant. There is another codec that is as fast as QOI (or even faster and multi-core) but with a much higher compression ratio. HALIC (High Availability Lossless Image Compression). But because it's closed source, it definitely didn't get the attention and respect it should have gotten. And that's why I think it stopped developing. https://github.com/Hakan-Abbas/HALIC-High-Availability-Lossless-Image-Compression https://github.com/Hakan-Abbas/HALIC-High-Availability-Lossl...

lifthrasiir 2y ago
A simple filter may still do wonder if it is carefully chosen ;-) The apparent death of HALIC was indeed unfortunate, I've heard that its decoder was less than 1K lines of code so it must have had a carefully designed and optimized context model in its heart. But there are so many other formats that are F/OSS but weren't highlighted enough anyway...
lblume 2y ago
For a format to proliferate as a standard, it has to be open source. Nobody cares about 5% smaller images if not everyone can view them.
- otterley 2y ago
  Historically speaking, that’s not true; WMV and FLV were extremely popular for over a decade in the early 2000s. Closed source formats can, and have, become de facto standards if they solve a pressing need that open standards don’t (or provide a stepwise increase in functionality or performance) and if there’s sufficient market demand.
ska 2y ago
>>> In fact, in certain cases it can increase the size instead of compressing. Fwiw, all (lossless) compression algorithms will increase the size of some inputs.