11 ms·
Quite OK Image is now my favorite asset format
- citizenpaul 4y agoMy fav format is whatever is widely supported so I don't have to bash my head on the wall dealing with all kinds of conversions and other issues. My least fav is when some system i have to deal with creator chose their superior fav obscure format that no one uses and now i have to deal with it too.
- WirelessGigabit 4y ago> With error checking removed, my decoder is ~600 bytes of x86 object code — a great story for embedding alongside assets. Soon in an IOT device near you. Ready to be exploited.
- andreareina 4y ago"linear" is not a color space. Presumably they mean something like, sRGB but with linear gamma, but without understanding the design goals of this format it's hard to say what purpose the linear encoding serves.
- lifthrasiir 4y agoIIRC it is a late addition and only added because of people wanting some kind of color space guarantee. I don't think it was ever accurately specified what they do mean---presumably both share the sRGB color gamut and only differ by transfer functions, but it's a frequent misconception [1] that transfer functions can be parametrized with a single value, gamma, so I guess that's where the "linear" came from (implying gamma = 1.0). [1] The sRGB EOTF has a nominal gamma of 2.2 but it is actually a piecewise function with gamma varying from 1.0 to 2.4.
- esperent 4y ago> I guess that's where the "linear" came from (implying gamma = 1.0). "Linear" means the colors are stored in a state that means you can do linear transformations on them (basic math in other words) without loss. You can't do this with gamma corrected colors - for example as you say, the sRGB EOTF is a piecewise function so by definition it's not a linear transformation. So yes, when talking about "linear" colors, they still come from some gamut like sRGB or a larger HDR gamut. The linear part means they are safe for use in a renderer's internals.
- andreareina 4y agoI've implemented color-managed workflows so I should have suspected that of course sRGB would be like that, it has to.
- lifthrasiir 4y agoYeah, if you have the slightest experience with color space you would know that's a misconception, but I've seen enough cases where the 2.2 gamma is uniformly applied throughout the brightness curve...
- mananaysiempre 4y agoCompare how Netpbm declared[1] that colour images are BT.709 (HDTV) relatively late in its existence, and seemingly every other implementation ignores that and treats them as sRGB (same—or close enough—primaries, different transfer function). [1] https://netpbm.sourceforge.net/doc/ppm.html https://netpbm.sourceforge.net/doc/ppm.html
- kaetemi 4y agoPretty much all modern 3D rendering is done in this color space that supposedly doesn't exist. Also, normal maps should be flagged as linear, since they should not get converted from the sRGB curve upon sampling.
- bscphil 4y agoDoes that just mean that the color space is untagged / undefined? (That is in practice how I see a lot of games rendered - way oversaturated on my wide gamut screen.)
- csande17 4y agoFor normal maps, the correct color space is definitely "no color space", because the R/G/B channels are used to store a 3D vector and not an actual color. For video game graphics, the ideal color space is "looks good on the median user's television once we apply our gamma curve", which is not actually a real color space. It's more like a statistical distribution of color spaces.
- bscphil 4y agoNormal maps, certainly. I was more confused by the parent's claim that this applied to "pretty much all modern 3D rendering". With video game graphics I'm surprised that PC games aren't handled differently. To the extent that TVs aren't garbage (huge assumption) traditional SDR TVs mostly target sRGB. But PC monitors can be almost anything, there are a ton of wide gamut (e.g. P3) screens out there, and saturated games can be retina-searing on them.
- csande17 4y agoIt'd probably be reasonable to say the output of a game's 3D renderer is sRGB or something, and to call the eye-searing colors on your P3 monitor a bug. But I think kaetemi's comment was talking about the intermediate calculations, which don't usually use sRGB. The "linear" color space used for those doesn't need to have a rigorous definition; even if there's no mathematically correct way to map a video-game-linear color to an sRGB color, the game company just hand-tweaks it until the game looks good.
- iainmerrick 4y agoPresumably they mean something like, sRGB but with linear gamma Yes, it’s exactly that, because: - almost all the time, the display is sRGB, so that’s your gamut; - almost all the time, you want to do your lighting and compositing in a linear space. - and sometimes your channels are normals or something, not colors at all, as others have noted. So it’s linear in the sRGB 0..255 gamut. It is a bit of a shame that most easy-to-use tools and workflows are limited to sRGB, so it’s really fiddly to support HDR displays and print.
- esperent 4y ago> almost all the time, the display is sRGB, so that’s your gamut; This is becoming less and less of a safe statement. On the one hand "yay HDR"! On the other hand, we're in for many years of weird bugs. For example recently I've been working on an app that I want to look good in HDR and I tried to share screenshots. Everything looked good to me. People on the other side were complaining about weird colors. Several hours of investigation later I realized that Windows was switching color spaces in my screenshots. Pressing the print screen button gave me a subtly different result than what I was actually seeing and of course I didn't spot it when emailing off the screen shots.
- jiggawatts 4y agoI'm reading this on an OLED display with about 90% of the gamut of Rec.2020. It's just a laptop, nothing special. Restricting colour to sRGB is going to be the CGA/EGA in a VGA world of the future.
- iainmerrick 4y agoI hope you’re right!
- Dylan16807 4y agoOkay, the decoder is less than a kilobyte. But where is that an advantage over a 50KB PNG decoder, when the compression ratio suffers so much?
- whateveracct 4y agogolf
- Dylan16807 4y agoGolf of what sort? It's hard to beat RLE or RLE+palette for minimal code size. I'd expect either one of those to be a tenth the size of a QOI decoder.
- adgjlsfhk1 4y agothese do well on synthetic images but really poorly with noise. QOI manages pretty good performance even with noise.
- Dylan16807 4y agoThat doesn't really explain the code golf answer.
- eps 4y agoQOI is in the same ballpark as RLE simplicty-wise but performs much better in a larger number of common cases. That's very much "golf".
- Dylan16807 4y agoI don't know about that. My estimate would be that QOI is ten times as complex as RLE and ten times less complex than DEFLATE-style compression. It could (barely) share a ballpark with either one.
- 4y ago
- eska 4y agoIs RGBA order really less common than ARGB?
- kevingadd 4y agoDepends on your platform, there are some platforms where the compositor/hardware demand ARGB and some where they demand RGBA. As a result if your backbuffer/etc are not in that format you eat some overhead. This isn't a big justification for it in a texture format, though. A channel swap isn't that expensive to do at load time and the driver can often do it for you during the upload depending on the API you're using. If you really want to load blazing fast your textures should be pre-compressed in hardware formats that you can mmap in, not stored in QOI or PNG.
- dottedmag 4y agoWithout sacrificing much simplicity channel swap can be made even more inexpensive with PSHUFB on x86 or VTBL on aarch64.
- dannyw 4y agoSounds like a great way to lose your files in 10 years unless you re-implement the decoder after wading through link rot.
- ardwindo 4y agoThe author notes his decoder is 100 lines of libc-free C code: these files are safe!
- qayxc 4y agoWhy wouldn't they just use TGA? Just as simple, more flexible if need be and understood even today by most image manipulation programs and 3d modelling software.
- dottedmag 4y agoThe size of decoder is so small, so one can append the source code of decoder to most images and not make them much bigger in the process.
- WirelessGigabit 4y agoBut what if the decoder rots?
- dottedmag 4y agoYou mean standalone C99 code becomes non-compilable? Absurd.
- aaron695 4y ago
- nness 4y agoI spend some free time reverse-engineering obscure game formats, and my observations is that this is universally true of everything.
- bsdetector 4y agoThere's a run length for identical pixels but no run length for literal pixels. Some images barely compress which means there should be many runs of uncompressed pixels. Even if only a few pixels long, not having a byte of overhead for each should improve size on these pictures. Maybe they just wanted it to win on throughput by having fewer branches.
- gregp87 4y agoI'm developing game engine + game and since several months I use the QOI format. This format is amazing in my situation: I can load the textures much faster compare to PNG. Generally, it is 2-3 times faster even if the size on disk is 1.5x-2.5x larger. Note: I use a NVME disk and I'm not sure my statement is still true on less performant hard drive.
- ninepoints 4y agoWhy are you using PNGs as a texture format? Generally, you'll use BC textures for the most part, or barring that, your images will be compressed as part of your archive format with some sort of super-compression algorithm that compresses the pakfile block-by-block. Perhaps you're referring to your texture compression/conversion pipeline which ingests PNG from your DCC tool of choice?
- Dylan16807 4y agoTry Zstandard. It can decompress about four times as fast as zlib, so it should do very well on PNGs. Also give LZ4 a shot.
- mhd 4y agoBack in the (DOS) days, Targa/TGA was somewhat popular for the asset usage. Simple format plus RLE for a small size benefit.
- qayxc 4y agoAmazing how few people do some basic research - this isn't new or exciting at all. TGA is fine and has been around for decades. It also has the added benefit that many applications still support it natively. I really don't get the appeal of inventing yet another custom file format that has so few objective benefits.
- layer8 4y agoUnlike QOI, TGA only has RLE as a means of compression, and therefore has worse compression on many images than QOI. At the same time, TGA supports a much larger range of format variants (arbitrary color depths, color palettes, grayscale mode), making it much more complex to implement than QOI.
- qayxc 4y agoAnd what exactly stops a decoder from simply disregarding unnecessary format variants? If you go as far as to implement your own decoder anyway, this argument is simply not applicable. Supporting only a subset of the format is perfectly acceptable - see TIFF, for example; not every application supports every possible feature.
- layer8 4y agoYou still need extra code to check that the format variant matches your expectations. I guess you could go very bare-bones and just assume that it’s in the right format and only check for buffer overflows, at the cost that errors will be harder to diagnose when the format is wrong. And you’d still have the overhead of the larger metadata structures of TGA. The fact is, none of the existing formats is a perfect fit for the embedded use-case here. It makes sense to introduce a new, simple and straightforward format if it addresses certain typical use-cases better than existing formats do.
- smallstepforman 4y agoIt’s amazing how unaware people are of lossless WebP. Compared to PNG, it decodes faster, compresses better, with a minor code size increase. When it comes to games though, BC7 all the way …
- fedeb95 4y agoSince when writing encoders and decoders in one sitting is a metric for a good asset format?
- rsp1984 4y agoThere are a lot of dimensions on which QOI could be improved, but most cases involve trade-offs, e.g. more complexity for better compression. The areas where QOI could have been strictly better, the dimensions on which it is not on the Pareto frontier, are more meaningful criticisms — missed opportunities. This summer I spent a lot of time researching lossless compression formats as a streaming format for our 3D capture and reconstruction engine. QOI was on our list too. My biggest criticism about QOI is that in many cases it doesn't compress at all, or even increases size. It works well on the demo images that come with the package but these are carefully selected. It also works well on high-res, high-quality images with lots of single-color background and/or smooth gradients. But give it grainy, noisy, smaller-format images and it completely breaks down. We ended up sticking with PNG, which btw. has fast open source implementations too [1][2]. [1] https://libspng.org/ https://libspng.org/ [2] https://github.com/richgel999/fpng https://github.com/richgel999/fpng
- IvanK_net 4y agoI guess you are taking photographs. One should never try to store photographs with no loss. With a lossy format, like JPG or WebP, you can get 5x smaller file size, while giving up 0.000001% of the original information (which usually corresponds to specific values of noise, that are not important in practice).
- p0nce 4y agoAll of these QOI derivatives improves upon the coding efficiency AFAIK. In QOIP, opcodes themselves are stored in the format, LZ4 or zstd compression https://github.com/chocolate42/qoipond https://github.com/chocolate42/qoipond QOIR adds: color profile, EXIF, premultiplied alpha, lossy mode, tile support, LZ4 compression, AFAIK it does this while beating the original QOI in terms of speed AND efficiency. Lots of benchmarks in: https://github.com/nigeltao/qoir https://github.com/nigeltao/qoir QOIX is my very own version that adds: Greyscale [+alpha] 8-bit image support, lossy 10-bit support (16-bit encoded as 10-bit, intended for elevation maps), so it's really 3 codecs , optional LZ4. https://github.com/AuburnSounds/gamut https://github.com/AuburnSounds/gamut
- phoboslab 4y ago