5 ms·

As high DPI displays become more common, it seems like the time has come for more advanced file formats (or just using the more advanced features of existing fo

As high DPI displays become more common, it seems like the time has come for more advanced file formats (or just using the more advanced features of existing formats). For example, PNG already has an interlacing mode that progressively adds detail to the image (http://en.wikipedia.org/wiki/Adam7_algorithm http://en.wikipedia.org/wiki/Adam7_algorithm). It could easily be co-opted such that a normal image contains 6 layers of detail, with an additional "retina extension" image that high DPI browsers grab to refine the quality. This avoids the waste of downloading a low DPI image, then throwing it away to replace it with high DPI. Even more interesting is http://en.wikipedia.org/wiki/JPEG_2000 http://en.wikipedia.org/wiki/JPEG_2000, which has "truncatable" bitstreams. You can stop getting data at any point, and depending on the encoding choices, you'll just lose fidelity in colour, resolution etc. Encoders can reorder the bitstream to deliver whatever is most useful for the image first. Browsers could then just stop when "enough" has been downloaded to satisfy the demands of the device; high DPI devices would just continue to grab more of the bitstream. It's really useful for devices on low bandwidth links, as you start getting visual results with very little data. JPEG 2000 hasn't been widely implemented outside of specialised devices, mostly because it's computationally heavy compared to JPEG, and the patent situation is unclear. Although this does mean that it could be implemented in a targeted way, designed to solve these problems (the spec for the entire format is huge). Moreover, one of the patent holders is, IIRC, Apple.

ned 14y ago
The idea of compound images is really interesting, although it would require an extra HTTP request to fetch the hight-DPI portion of the image. But maybe less of a problem over SPDY? The idea truncatable bitstreams is fascinating too. I'm not well versed in networking, but wouldn't the latency of a mobile network kill the benefit of this technique? e.g, by the time the server receives the "connection closed" signal, a large part of the extra data would have been sent, no?
- archgrove 14y ago
  It would need an extra HTTP request, yes. In practice for the PNG solution, one would be better off with sending the DPI in the request headers, so that the "correct" image is the only one sent. For JPEG 2000, the network characteristics are important, but I don't think it would be too bad on a mobile network. Low DPI devices might get a bit "too much", but it wouldn't be a problem - they can just throw it out (or incorporate more detail).
  - jentulman 14y ago
    Headers do seem the best place for a non vendor specific standard to be set. I'd love to see something like viewport-dpi - the dpi viewport-max - the maximum possible pixel dimension and possibly viewport-current - the pixel dimensions at the time of the request Decent responsive design should deal with the differing viewport sizes, but it might be nice to get a hint before delivering your page what direction to weight that response in.
    - Groxx 14y ago
      What do you do when using a projector, and your DPI is something like 3? (serious question, I don't know how this is traditionally handled)
      - kijin 14y ago
        Something like device-pixel-ratio would be more useful than DPI. Raw DPI is meaningless unless you take distance into account, as is very clear in the case of the projector.
  - nextparadigms 14y ago
    Would this work better over WebP?
- dwnoiz 14y ago
  Check this way of doing it to avoid extra http requests. http://www.archer-group.com/2012/development/javascript-jquery-snippet-for-doing-retina-resolution-images http://www.archer-group.com/2012/development/javascript-jque...
  - caseyohara 14y ago
    That is certainly the ideal way to reduce the HTTP requests. The problem with this implementation is clear: without a 'src' attribute on your images, if your script fails then your site suddenly has no images at all. Not to mention the overhead of manually modifying all of your existing markup. Our goal with retina.js was to make it zero-config: no markup changes, no extra element attributes or flags.
    - oops 14y ago
      The Stripe gallery posted here the other day[1] used src-less img tags for lazy loading. It looks like they handle the script issue by having the normal img tag (with src attribute) nearby in a noscript block. Doesn't really do much to help with the overhead of modifying existing markup. Perhaps if you're already using something like image_tag in Rails, a retina_image_tag helper might not be too much of a stretch. 1: https://stripe.com/gallery https://stripe.com/gallery
ajross 14y ago
Regarding the first trick: the value really isn't very much. Storing 2x2 downsampled versions of an image at all levels is only a 33% increase in size. This is the mipmap trick, and can be done right now with nothing but a size check on the client. I don't think 33% is worth mucking with file formats over.
WiseWeasel 14y ago
Sounds complicated. I think we just need to do something similar to HTML5 'video' tags for the 'image' tag, where you simply put a list of file paths in the tag, and the browser selects the best one to request, only with different resolutions instead of different file encodings. We could have a standard notation like adding "_x2" to the double res image filenames, and the browser selects the appropriate one to load. Then, we can support current image formats in the way you envision. Also, anything you have vectorized should be in SVG format, including text in non-web fonts.
- archgrove 14y ago
  That's fine if a) you're willing to maintain two image sets and, b) the only two resolution images you need are "normal" and "2 x normal". The advantage of a progressive quality system is it scales to arbitrary requirements in a single resource. One file provides 1x, 2x, 3x, 3.14x etc.
  - beagle3 14y ago
    Storage is too cheap to care these days. When $110 buys you 2TB (this morning on Newegg), then doing it on the fly and caching the result has about 100x better ROI than trying to solve it with a smart format. Remember, we're speaking about still photos here, where high retina quality ones rarely take more than 500k each, so 2TB buys you room for 4,000,000 photos (and since you'll be caching the lower res 200K images, it actually buys you room for 10,000,000 million photos) That's not the case for videos - netflix, hulu and friends have multiple copies of each 1GB optimized for different devices. On-the-fly conversion is not possible, and the stream are significantly difference. For them, a reasonable, universally supported progressive video format will indeed make a huge difference
    - scott_s 14y ago
      I don't think archgrove's point was about storage, but about the extra friction of tracking N files instead of 1.

ned 14y ago

The idea of compound images is really interesting, although it would require an extra HTTP request to fetch the hight-DPI portion of the image. But maybe less of a problem over SPDY? The idea truncatable bitstreams is fascinating too. I'm not well versed in networking, but wouldn't the latency of a mobile network kill the benefit of this technique? e.g, by the time the server receives the "connection closed" signal, a large part of the extra data would have been sent, no?

archgrove 14y ago
It would need an extra HTTP request, yes. In practice for the PNG solution, one would be better off with sending the DPI in the request headers, so that the "correct" image is the only one sent. For JPEG 2000, the network characteristics are important, but I don't think it would be too bad on a mobile network. Low DPI devices might get a bit "too much", but it wouldn't be a problem - they can just throw it out (or incorporate more detail).
- jentulman 14y ago
  Headers do seem the best place for a non vendor specific standard to be set. I'd love to see something like viewport-dpi - the dpi viewport-max - the maximum possible pixel dimension and possibly viewport-current - the pixel dimensions at the time of the request Decent responsive design should deal with the differing viewport sizes, but it might be nice to get a hint before delivering your page what direction to weight that response in.
  - Groxx 14y ago
    What do you do when using a projector, and your DPI is something like 3? (serious question, I don't know how this is traditionally handled)
    - kijin 14y ago
      Something like device-pixel-ratio would be more useful than DPI. Raw DPI is meaningless unless you take distance into account, as is very clear in the case of the projector.
- nextparadigms 14y ago
  Would this work better over WebP?
dwnoiz 14y ago
Check this way of doing it to avoid extra http requests. http://www.archer-group.com/2012/development/javascript-jquery-snippet-for-doing-retina-resolution-images http://www.archer-group.com/2012/development/javascript-jque...
- caseyohara 14y ago
  That is certainly the ideal way to reduce the HTTP requests. The problem with this implementation is clear: without a 'src' attribute on your images, if your script fails then your site suddenly has no images at all. Not to mention the overhead of manually modifying all of your existing markup. Our goal with retina.js was to make it zero-config: no markup changes, no extra element attributes or flags.
  - oops 14y ago
    The Stripe gallery posted here the other day[1] used src-less img tags for lazy loading. It looks like they handle the script issue by having the normal img tag (with src attribute) nearby in a noscript block. Doesn't really do much to help with the overhead of modifying existing markup. Perhaps if you're already using something like image_tag in Rails, a retina_image_tag helper might not be too much of a stretch. 1: https://stripe.com/gallery https://stripe.com/gallery

ajross 14y ago

Regarding the first trick: the value really isn't very much. Storing 2x2 downsampled versions of an image at all levels is only a 33% increase in size. This is the mipmap trick, and can be done right now with nothing but a size check on the client. I don't think 33% is worth mucking with file formats over.

WiseWeasel 14y ago

Sounds complicated. I think we just need to do something similar to HTML5 'video' tags for the 'image' tag, where you simply put a list of file paths in the tag, and the browser selects the best one to request, only with different resolutions instead of different file encodings. We could have a standard notation like adding "_x2" to the double res image filenames, and the browser selects the appropriate one to load. Then, we can support current image formats in the way you envision. Also, anything you have vectorized should be in SVG format, including text in non-web fonts.

archgrove 14y ago
That's fine if a) you're willing to maintain two image sets and, b) the only two resolution images you need are "normal" and "2 x normal". The advantage of a progressive quality system is it scales to arbitrary requirements in a single resource. One file provides 1x, 2x, 3x, 3.14x etc.
- beagle3 14y ago
  Storage is too cheap to care these days. When $110 buys you 2TB (this morning on Newegg), then doing it on the fly and caching the result has about 100x better ROI than trying to solve it with a smart format. Remember, we're speaking about still photos here, where high retina quality ones rarely take more than 500k each, so 2TB buys you room for 4,000,000 photos (and since you'll be caching the lower res 200K images, it actually buys you room for 10,000,000 million photos) That's not the case for videos - netflix, hulu and friends have multiple copies of each 1GB optimized for different devices. On-the-fly conversion is not possible, and the stream are significantly difference. For them, a reasonable, universally supported progressive video format will indeed make a huge difference
  - scott_s 14y ago
    I don't think archgrove's point was about storage, but about the extra friction of tracking N files instead of 1.