7 ms·

This is really interesting. One of the most interesting parts is the progressive decoding/responsive images. http://flif.info/example.php http://flif.info/examp

This is really interesting. One of the most interesting parts is the progressive decoding/responsive images. http://flif.info/example.php http://flif.info/example.php (go to the bottom of the page). Basically, the last example shows that, if you want a scaled version of the image, you can simply stop decompressing. No need for multiple image files. Just create one with very high quality and decompress until you get the quality you want and scale it down with html. edit: more info on the responsive side of FLIF: http://flif.info/responsive.php http://flif.info/responsive.php

digi_owl 11y ago
Makes me think of the web of old, where sometimes an image would load in multiple passes.
- michaelmior 11y ago
  I assume you're talking about progressive JPEGs. These are still fairly common in my experience. It's just that with today's typical connection speed, you'll rarely notice.
  - masklinn 11y ago
    There's also progressive PNGs. Progressive JPEGs have the advantage that they increase compression ratio (though they're more expensive to decode, as you need multiple passes and refresh)
    - darkmighty 11y ago
      They increase compression ratio? That's counter-intuitive. You're effectively imposing an ordering requiring certain information to be available first, so you'd expect the compression be at most as good as non-progressive. I guess the changed ordering makes the statistics simpler and easier to compress, or something like that.
      - mzs 11y ago
        The initial passes are lower freq, so they compress better. More over you have control over them when encoding so you can optimize there (and there are tools that do so), but the defaults used by all encoders are very good across most all images.
      - cbr 11y ago
        The downside is memory usage during compression. A non-progressive jpeg can be compressed locally but a progressive encoding requires the whole image. But this is only a small downside. (Converting JPEG images to progressive format is one of the optimizations mod_pagespeed makes.)
michaelmior 11y ago
That part is pretty cool. If this were widely deployed, I could imagine this making the use of responsive images much easier.
linkydinkandyou 11y ago
JPEG 2000 has that same feature. FLIF claims it will beat the lossless compression ratio of JPEG 2000
- espadrine 11y ago
  It also claims to beat the lossy version, which cannot even be encoded for the transparent fish image at the tested file sizes: http://flif.info/example.php http://flif.info/example.php [ed. image size → file size to emphasize that they had a target byte count for the purpose of comparison]
  - jpambrun 11y ago
    JPEG 2000 is capable of compressing 1969x1307 images without issues. I work with JPEG 2000 codecs every day. It is commonly used in Virtual Microscopy with images in the tens of gigabytes as well as with small 256x256 MRI. I don't know why the author does not present "JPEG 2000 at this size". Edit: I was confused since with most coders you can specify precise mean squared error optimal truncation points (called quality layers) and it should have been very easy to obtain a stream of any particular file size.
    - deleted 11y ago
      [deleted]
    - vidarh 11y ago
      That example is looking at a specific file size. So what he's saying is that he was unable to produce a JPEG 2000 image in a file that size from that image, not from an image of that resolution in general.
      - silverbacknet 11y ago
        He didn't try hard enough. OpenJPEG has a file size parameter, which can go well below its -q 0 size. I was easily able to generate a 16.9K jp2 by specifying the size: https://dl.dropboxusercontent.com/u/54412753/doom9/fish.jp2 https://dl.dropboxusercontent.com/u/54412753/doom9/fish.jp2 which in png looks like: https://dl.dropboxusercontent.com/u/54412753/doom9/fish.jp2.png https://dl.dropboxusercontent.com/u/54412753/doom9/fish.jp2.... Looks better than anything else but BPG at that size, to me, although FLIF obviously isn't optimized for lossy.
rplnt 11y ago
This would be great. A lot of "responsive" sites nowadays use only one image as well, the highest resolution available. So you are downloading that 4K jpeg whether you are on mobile or on a desktop.
- colanderman 11y ago
  Progressive JPEG has supported this for decades. Anyone over 25 should remember them from dial-up years.
  - hansjorg 11y ago
    I think the suggestion is that the client could just stop downloading at a set threshold, maybe taking current bandwidth, cost, etc. into consideration. Judging from the video, something like that could work very well (and much better than with progressive jpegs).
    - radarsat1 11y ago
      But how is that different from progressive JPEG?
    - guelo 11y ago
      It's an interesting concept. But I'm not sure how it could work. Resizing introduces artifacts so best quality is always going to be achieved by reencoding for different resolutions. Also, how would the client know when to stop downloading?
      - iopq 11y ago
        The client would download the first 1KB for an icon, 4K for a thumbnail and the whole image for the full size. The best part is that all of those are actually the same file, not separate embedded thumbnails inside the file. The client could decide BY ITSELF to download only 2K for a thumbnail because it's currently on 3g.
deleted 11y ago
[deleted]
est 11y ago
JPEG has something cooler, you can first load a black-white smaller picture, then load more color and more detail.
- theoh 11y ago
  Thinking back to the old days, are you sure the black and white image you have in mind didn't come from the "lowsrc" attribute of an img tag? Progressive jpeg is typically full colour from the start, and it does offer the progressive enhancement of resolution that you mention.
  - mzs 11y ago
    It depends on the encoder, it very well could be the first pass is just Y.
    - theoh 11y ago
      I'd call that "grayscale" rather than black and white, but interesting suggestion. I've never seen it done but looks like all sorts of things are possible: http://hodapple.com/blag/2011/11/24/obscure-features-of-jpeg.html http://hodapple.com/blag/2011/11/24/obscure-features-of-jpeg...
      - mzs 11y ago
        Here's the Mandrill test image from SIPI and Utah as a 75% progressive JPEG with just the first scan of the most significant bit of just the DC for Y only, which is a totally expected scheme for the web back in the '90s: https://lh3.googleusercontent.com/-4tX6mfFoY4c/Vg7bg85RwcI/AAAAAAAArjY/Ze2YOUq3Q0M/s512-no/mandrill_grey.jpg https://lh3.googleusercontent.com/-4tX6mfFoY4c/Vg7bg85RwcI/A...
        theoh 11y ago
        What do you mean by "totally expected"? A Turing machine "totally expects" an infinite ribbon.
hngiszmo 11y ago
Also no more .thumbnails folders I imagine images appearing as fast as I can scroll, with enough visual clue to them to find images way way more quickly than with thumbnails today. As a start, thumbnails could use flif.
- panzi 11y ago
  I think some formats already support embedded thumbnails. (EXIF metadata can contain thumbnails, I believe)
  - jo909 11y ago
    But that is not the same. That is a small, but fixed size, second version of the image, embedded into the same file. With FLIF you simply read the first N bytes of the full image and have a resonable preview. You choose how big or small N has to be, depending on the size of the thumbnail you want to show. Maybe first read N bytes for each image to get a quick but rough preview, then repeatedly read a few bytes more to enhance the thumbnail.
    - magicalist 11y ago
      To be fair, that's not the same thing either. A thumbnail is resampled in a way to resemble the original at a smaller size. This will be resampled with (more or less) nearest neighbor, which means lots of aliasing and possibly looking nothing like the original, depending on the subject.
      - DiThi 11y ago
        I haven't checked, but I suspect that a better down scaling method can be used while still being compatible with the same decoding algorithm.
        magicalist 11y ago
        hmm, how could it be? The pixels values have to fit into the eventual reconstructed image. If the values were different than any pixels found in the final image, it wouldn't be progressively loading, it would have several different size images embedded in it.
sergiotapia 11y ago
If you want to use a similar feature -today-, there's this service called Imgix: https://www.imgix.com/ https://www.imgix.com/ Basically, it allows you to use responsive images using 1 single master image. Makes for a really snappy user experience, and it's very easy to integrate into any project, new or old.
- rjcaricio 11y ago
  Free & Open Source version: http://thumbor.org/ http://thumbor.org/
isthmus_ 11y ago
Who needs mipmaps?
- malkia 11y ago
  The GPU!- skipping those pixels hurts the cache, even if swizzled :) And my eyes - shimmering car grills and garden fences begone!
  - alanh 11y ago
    Wow, okay, this comment and its parent lost me :)
    - malkia 11y ago
      Rough context: When the GPU (or the CPU back in the days) had to draw a texture, it had to sample pixels, maybe one, maybe four and do some say bi-linear filtering in between them, and then use that pixel as a result. Now several problems: 1. If your texture is sampled roughly one pixel from it to one pixel on the screen, and if all pixels are read linearly ("horizontally"), then you are good with the cache, cause for the first pixel you've read the cache maybe cold at that address, but it'll load the next pixels just in case you need them, and here is the catch - you need to get always benefit from that - it's like always using your coupons, and deals, and your employer perks. So the CPU/GPU might read 4, 8, 16 who really knows (but you) bytes in advance, or around that pixel. 2. But then you turn the texture 90 degrees, and suddenly it's much slower. You draw a pixel from the texture, but then the next pixel is 256, 512, or more away ("vertically), and then next too, your cache line of 4, 8, 16, 32 or 64 read bytes that you did not used, and by the time you might need them again, the cache discarded them. Hence the slowness now - much much slower! 4. To fix it, you come up with "swizzling" - e.g. instead of the texture being purely scan-line by scan-line, you kind of split your image in blocks - maybe 8x8 tiles, or 32x32, and then make sure those tiles are linearly written one to each other. You can go even further, but the idea is that under any angle, if you decide to read, you most likely would hit pixels from the same cache line you've read before. It's not that simple, and my explantation is poor, but here is someone who can do that better than me: https://fgiesen.wordpress.com/2011/01/17/texture-tiling-and-swizzling/ https://fgiesen.wordpress.com/2011/01/17/texture-tiling-and-... 8. But even with swizzling, tiling, whatever you call that magic to keep pixels together in any direction really together stops working as soon as you have to draw that big texture/image on a much smaller scale. 16. Say 256x256 would have to be drawn as 16x16 - And you say I don't need mipmapping, I don't need smaller versions of my image, I can just use my own image - well then nomatter how you swizzle/tile, you'll be skipping a lot of pixels - hop from here to there, and lost cache lines. 32. For that reasons mipmaps are here to help, but stay with me for one more minute - and see how they almost fix the shimmering problem of textures with "fences", "grills on a car", a pet's cage, or something like it. 64. And when you hear the artist ready to put real spider-man logo on the character made out of real polygons, and real grill in front of the car, made out of real polygons, and real barbed-wired made out of polygons very nice looking fence - then stay away, as these polys woulds shimmer, no good level of detail can be done for them (it'll pop) and such things are just much easily done with textures and mipmaps - it kind of solves a lot of problems. 128. Read about impostor textures.
silverbacknet 11y ago
I don't feel like that page is entirely fair. The PNG and GIF comparisons are unfair; they're the same low resolution as FLIF, but given nearest-neighbor upsampling instead of bilinear. Most web browsers use bilinear or bicubic upsampling for progressive pictures now, so FLIF isn't nearly as unique as it presents itself. Aside from that, progressive is only useful if you're on a slow connection or the file is broken; in other cases re-rendering multiple times becomes a bottleneck and battery drain. In general it's rare that progressive decoding is a win anymore (although progressive encoding of JPEG is always a win). In another comment I pointed out that jpeg2000 can be encoded to any size, even 1k, even though the q-scale doesn't go that low, by using the target size parameter instead. It looks better than anything but BPG and is fully progressive. Of course, J2K is a dead format outside of DejaVu, PDF, and the medical field, but it serves as a useful baseline. A comparison to JXR would have been nice; it's an open, patent-indemnified format that fully supports progressive decoding, and compresses about as well as WebP. That format's the biggest disappointment for me, I thought it had everything in place to displace JPEG. BPG could be made somewhat progressive, but only by changing the underlying HEVC bitstream, while one of its goals is to stay as close as possible. The inclusion of a thumbnail is the only recourse.
- jonsneyers 11y ago
  You're right about the interpolation method (bilinear vs nearest-neighbor), it's indeed not a fair comparison in that respect. However, I do think that most PNG viewers use nearest-neighbor (at least that's what my viewer did when I gave it a partial PNG file). But it's not just the interpolation that makes a difference: PNG interlaces full RGB pixels (all 3 channels at the same time), while FLIF gives priority to luma so you get a chroma subsampling effect on partial files. Also GIF only has vertical interlacing and PNG only does 7 passes of 2D interlacing, which means that for huge images, the first pass can still take a while. Another point: I am proposing to use progressive decoding to avoid having to download the full lossless image (so a browser supporting FLIF would need some mechanism to stop downloading the image file when "enough" detail is available). Repeated re-rendering is not necessary. E.g. on a mobile phone you would rather just download some prefix of the file (the size of the prefix could depend on the available bandwidth) and render it once, at least until the user zooms in on the image, or saves or prints it or something -- then the download should be resumed in order to load more/all detail.

digi_owl 11y ago

Makes me think of the web of old, where sometimes an image would load in multiple passes.

michaelmior 11y ago
I assume you're talking about progressive JPEGs. These are still fairly common in my experience. It's just that with today's typical connection speed, you'll rarely notice.
- masklinn 11y ago
  There's also progressive PNGs. Progressive JPEGs have the advantage that they increase compression ratio (though they're more expensive to decode, as you need multiple passes and refresh)
  - darkmighty 11y ago
    They increase compression ratio? That's counter-intuitive. You're effectively imposing an ordering requiring certain information to be available first, so you'd expect the compression be at most as good as non-progressive. I guess the changed ordering makes the statistics simpler and easier to compress, or something like that.
    - mzs 11y ago
      The initial passes are lower freq, so they compress better. More over you have control over them when encoding so you can optimize there (and there are tools that do so), but the defaults used by all encoders are very good across most all images.
    - cbr 11y ago
      The downside is memory usage during compression. A non-progressive jpeg can be compressed locally but a progressive encoding requires the whole image. But this is only a small downside. (Converting JPEG images to progressive format is one of the optimizations mod_pagespeed makes.)

michaelmior 11y ago

That part is pretty cool. If this were widely deployed, I could imagine this making the use of responsive images much easier.

linkydinkandyou 11y ago

JPEG 2000 has that same feature. FLIF claims it will beat the lossless compression ratio of JPEG 2000

espadrine 11y ago
It also claims to beat the lossy version, which cannot even be encoded for the transparent fish image at the tested file sizes: http://flif.info/example.php http://flif.info/example.php [ed. image size → file size to emphasize that they had a target byte count for the purpose of comparison]
- jpambrun 11y ago
  JPEG 2000 is capable of compressing 1969x1307 images without issues. I work with JPEG 2000 codecs every day. It is commonly used in Virtual Microscopy with images in the tens of gigabytes as well as with small 256x256 MRI. I don't know why the author does not present "JPEG 2000 at this size". Edit: I was confused since with most coders you can specify precise mean squared error optimal truncation points (called quality layers) and it should have been very easy to obtain a stream of any particular file size.
  - deleted 11y ago
    [deleted]
  - vidarh 11y ago
    That example is looking at a specific file size. So what he's saying is that he was unable to produce a JPEG 2000 image in a file that size from that image, not from an image of that resolution in general.
    - silverbacknet 11y ago
      He didn't try hard enough. OpenJPEG has a file size parameter, which can go well below its -q 0 size. I was easily able to generate a 16.9K jp2 by specifying the size: https://dl.dropboxusercontent.com/u/54412753/doom9/fish.jp2 https://dl.dropboxusercontent.com/u/54412753/doom9/fish.jp2 which in png looks like: https://dl.dropboxusercontent.com/u/54412753/doom9/fish.jp2.png https://dl.dropboxusercontent.com/u/54412753/doom9/fish.jp2.... Looks better than anything else but BPG at that size, to me, although FLIF obviously isn't optimized for lossy.

rplnt 11y ago

This would be great. A lot of "responsive" sites nowadays use only one image as well, the highest resolution available. So you are downloading that 4K jpeg whether you are on mobile or on a desktop.

colanderman 11y ago
Progressive JPEG has supported this for decades. Anyone over 25 should remember them from dial-up years.
- hansjorg 11y ago
  I think the suggestion is that the client could just stop downloading at a set threshold, maybe taking current bandwidth, cost, etc. into consideration. Judging from the video, something like that could work very well (and much better than with progressive jpegs).
  - radarsat1 11y ago
    But how is that different from progressive JPEG?
  - guelo 11y ago
    It's an interesting concept. But I'm not sure how it could work. Resizing introduces artifacts so best quality is always going to be achieved by reencoding for different resolutions. Also, how would the client know when to stop downloading?
    - iopq 11y ago
      The client would download the first 1KB for an icon, 4K for a thumbnail and the whole image for the full size. The best part is that all of those are actually the same file, not separate embedded thumbnails inside the file. The client could decide BY ITSELF to download only 2K for a thumbnail because it's currently on 3g.

deleted 11y ago

[deleted]

est 11y ago

JPEG has something cooler, you can first load a black-white smaller picture, then load more color and more detail.

theoh 11y ago
Thinking back to the old days, are you sure the black and white image you have in mind didn't come from the "lowsrc" attribute of an img tag? Progressive jpeg is typically full colour from the start, and it does offer the progressive enhancement of resolution that you mention.
- mzs 11y ago
  It depends on the encoder, it very well could be the first pass is just Y.
  - theoh 11y ago
    I'd call that "grayscale" rather than black and white, but interesting suggestion. I've never seen it done but looks like all sorts of things are possible: http://hodapple.com/blag/2011/11/24/obscure-features-of-jpeg.html http://hodapple.com/blag/2011/11/24/obscure-features-of-jpeg...
    - mzs 11y ago
      Here's the Mandrill test image from SIPI and Utah as a 75% progressive JPEG with just the first scan of the most significant bit of just the DC for Y only, which is a totally expected scheme for the web back in the '90s: https://lh3.googleusercontent.com/-4tX6mfFoY4c/Vg7bg85RwcI/AAAAAAAArjY/Ze2YOUq3Q0M/s512-no/mandrill_grey.jpg https://lh3.googleusercontent.com/-4tX6mfFoY4c/Vg7bg85RwcI/A...
      - theoh 11y ago
        What do you mean by "totally expected"? A Turing machine "totally expects" an infinite ribbon.

hngiszmo 11y ago

Also no more .thumbnails folders I imagine images appearing as fast as I can scroll, with enough visual clue to them to find images way way more quickly than with thumbnails today. As a start, thumbnails could use flif.

panzi 11y ago
I think some formats already support embedded thumbnails. (EXIF metadata can contain thumbnails, I believe)
- jo909 11y ago
  But that is not the same. That is a small, but fixed size, second version of the image, embedded into the same file. With FLIF you simply read the first N bytes of the full image and have a resonable preview. You choose how big or small N has to be, depending on the size of the thumbnail you want to show. Maybe first read N bytes for each image to get a quick but rough preview, then repeatedly read a few bytes more to enhance the thumbnail.
  - magicalist 11y ago
    To be fair, that's not the same thing either. A thumbnail is resampled in a way to resemble the original at a smaller size. This will be resampled with (more or less) nearest neighbor, which means lots of aliasing and possibly looking nothing like the original, depending on the subject.
    - DiThi 11y ago
      I haven't checked, but I suspect that a better down scaling method can be used while still being compatible with the same decoding algorithm.
      - magicalist 11y ago
        hmm, how could it be? The pixels values have to fit into the eventual reconstructed image. If the values were different than any pixels found in the final image, it wouldn't be progressively loading, it would have several different size images embedded in it.

sergiotapia 11y ago

If you want to use a similar feature -today-, there's this service called Imgix: https://www.imgix.com/ https://www.imgix.com/ Basically, it allows you to use responsive images using 1 single master image. Makes for a really snappy user experience, and it's very easy to integrate into any project, new or old.

rjcaricio 11y ago
Free & Open Source version: http://thumbor.org/ http://thumbor.org/

isthmus_ 11y ago

Who needs mipmaps?

malkia 11y ago
The GPU!- skipping those pixels hurts the cache, even if swizzled :) And my eyes - shimmering car grills and garden fences begone!
- alanh 11y ago
  Wow, okay, this comment and its parent lost me :)
  - malkia 11y ago
    Rough context: When the GPU (or the CPU back in the days) had to draw a texture, it had to sample pixels, maybe one, maybe four and do some say bi-linear filtering in between them, and then use that pixel as a result. Now several problems: 1. If your texture is sampled roughly one pixel from it to one pixel on the screen, and if all pixels are read linearly ("horizontally"), then you are good with the cache, cause for the first pixel you've read the cache maybe cold at that address, but it'll load the next pixels just in case you need them, and here is the catch - you need to get always benefit from that - it's like always using your coupons, and deals, and your employer perks. So the CPU/GPU might read 4, 8, 16 who really knows (but you) bytes in advance, or around that pixel. 2. But then you turn the texture 90 degrees, and suddenly it's much slower. You draw a pixel from the texture, but then the next pixel is 256, 512, or more away ("vertically), and then next too, your cache line of 4, 8, 16, 32 or 64 read bytes that you did not used, and by the time you might need them again, the cache discarded them. Hence the slowness now - much much slower! 4. To fix it, you come up with "swizzling" - e.g. instead of the texture being purely scan-line by scan-line, you kind of split your image in blocks - maybe 8x8 tiles, or 32x32, and then make sure those tiles are linearly written one to each other. You can go even further, but the idea is that under any angle, if you decide to read, you most likely would hit pixels from the same cache line you've read before. It's not that simple, and my explantation is poor, but here is someone who can do that better than me: https://fgiesen.wordpress.com/2011/01/17/texture-tiling-and-swizzling/ https://fgiesen.wordpress.com/2011/01/17/texture-tiling-and-... 8. But even with swizzling, tiling, whatever you call that magic to keep pixels together in any direction really together stops working as soon as you have to draw that big texture/image on a much smaller scale. 16. Say 256x256 would have to be drawn as 16x16 - And you say I don't need mipmapping, I don't need smaller versions of my image, I can just use my own image - well then nomatter how you swizzle/tile, you'll be skipping a lot of pixels - hop from here to there, and lost cache lines. 32. For that reasons mipmaps are here to help, but stay with me for one more minute - and see how they almost fix the shimmering problem of textures with "fences", "grills on a car", a pet's cage, or something like it. 64. And when you hear the artist ready to put real spider-man logo on the character made out of real polygons, and real grill in front of the car, made out of real polygons, and real barbed-wired made out of polygons very nice looking fence - then stay away, as these polys woulds shimmer, no good level of detail can be done for them (it'll pop) and such things are just much easily done with textures and mipmaps - it kind of solves a lot of problems. 128. Read about impostor textures.

silverbacknet 11y ago

I don't feel like that page is entirely fair. The PNG and GIF comparisons are unfair; they're the same low resolution as FLIF, but given nearest-neighbor upsampling instead of bilinear. Most web browsers use bilinear or bicubic upsampling for progressive pictures now, so FLIF isn't nearly as unique as it presents itself. Aside from that, progressive is only useful if you're on a slow connection or the file is broken; in other cases re-rendering multiple times becomes a bottleneck and battery drain. In general it's rare that progressive decoding is a win anymore (although progressive encoding of JPEG is always a win). In another comment I pointed out that jpeg2000 can be encoded to any size, even 1k, even though the q-scale doesn't go that low, by using the target size parameter instead. It looks better than anything but BPG and is fully progressive. Of course, J2K is a dead format outside of DejaVu, PDF, and the medical field, but it serves as a useful baseline. A comparison to JXR would have been nice; it's an open, patent-indemnified format that fully supports progressive decoding, and compresses about as well as WebP. That format's the biggest disappointment for me, I thought it had everything in place to displace JPEG. BPG could be made somewhat progressive, but only by changing the underlying HEVC bitstream, while one of its goals is to stay as close as possible. The inclusion of a thumbnail is the only recourse.

jonsneyers 11y ago
You're right about the interpolation method (bilinear vs nearest-neighbor), it's indeed not a fair comparison in that respect. However, I do think that most PNG viewers use nearest-neighbor (at least that's what my viewer did when I gave it a partial PNG file). But it's not just the interpolation that makes a difference: PNG interlaces full RGB pixels (all 3 channels at the same time), while FLIF gives priority to luma so you get a chroma subsampling effect on partial files. Also GIF only has vertical interlacing and PNG only does 7 passes of 2D interlacing, which means that for huge images, the first pass can still take a while. Another point: I am proposing to use progressive decoding to avoid having to download the full lossless image (so a browser supporting FLIF would need some mechanism to stop downloading the image file when "enough" detail is available). Repeated re-rendering is not necessary. E.g. on a mobile phone you would rather just download some prefix of the file (the size of the prefix could depend on the available bandwidth) and render it once, at least until the user zooms in on the image, or saves or prints it or something -- then the download should be resumed in order to load more/all detail.