9 ms·

Don’t use JPEG-XR on the Web

134 points by robin_reala 8y ago

PaulHoule 8y ago
From time to time I've run an analysis on the use of alternative image formats on the web and it never comes out to be worth it. New formats are not supported on all browsers so you wind up having to support the mainstream formats as well as one or more new formats. Rather than benefiting from reduced storage costs, your storage costs get multiplied. Performance is also a problem, particularly when people add "yet another polyfill" to get an image to work without native support. Once I get into eyeballing images closely I also end up questioning if the compression gains are real. For instance, one I read a comment on the article here https://calendar.perfplanet.com/2018/is-avif-the-future-of-images-on-the-web/ https://calendar.perfplanet.com/2018/is-avif-the-future-of-i... and looked closely I couldn't unsee that the AV1 had very different artifacts than the other compressed images. Not necessarily worse, but definitely different. Unlike the poster I liked the way the suit rendered but definitely there was a lot of blocking around edges that the poster interprets as "pixelation on the flag". Like many things, the costs of switching image formats are definite, but the benefits are questionable.
- cosmie 8y ago
  > From time to time I've run an analysis on the use of alternative image formats on the web and it never comes out to be worth it. I just checked the Google homepage, and even the New Year's Eve Google Doodle currently being shown is a good ol' GIF, rather than a modern format such as WebP. And this is with me using a desktop Chrome browser that definitively supports WebP. > Like many things, the costs of switching image formats are definite, but the benefits are questionable. One easy way to get some of the benefits without incurring the switching costs is to leverage Cloudflare Polish[1]. I've used it at a few places, and have had good experiences. Although the majority of gains came from optimizing images that were poorly optimized to begin with, rather than the incremental improvements that occurred from enabling WebP support. [1] https://support.cloudflare.com/hc/en-us/articles/360000607372-Using-Polish-to-compress-images-on-Cloudflare https://support.cloudflare.com/hc/en-us/articles/36000060737...
  - Avamander 8y ago
    I don't like WebP because it's either it or HW accelerated browsing - latter I like a tad more because it saves me a bit of battery juice on my laptop.
    - TazeTSchnitzel 8y ago
      There's the same problem with Google insisting on VP8/9 for YouTube in Chrome. If your Mac only supports h.264 in hardware you'll get better battery life in Safari.
      - xvilka 8y ago
        Obviously it is a problem with Apple, not Google. People should just push it more to do the job.
        TazeTSchnitzel 8y ago
        I don't think Google are free of responsibility. They are prioritising their lower bandwidth over the user's better power usage.
        vetinari 8y ago
        Apple has a bigger share of the blame. They purposely ignore any codec that is outside of their favorites, even if the hardware they ship does support it (i.e. Intel) or doesn't need any hardware support (Vorbis, Flac). On the other hand, they are quick to support any codec, that is problematic to implement elsewhere.
      - vosper 8y ago
        There's a Chrome extension that forces YouTube to stream h.264 https://chrome.google.com/webstore/detail/h264ify/aleakchihdccplidncghkekgioiakgal?hl=en https://chrome.google.com/webstore/detail/h264ify/aleakchihd...
      - acdha 8y ago
- pornel 8y ago
  In terms of compression, AV1 is a clear win. Note that artifacts that bother you are at about half the file size. You can increase the quality and still have an image that is both nicer and smaller than JPEG and WebP. However, as TFA mentions, better compression is currently a double-edged sword, because the CPU has become a bottleneck again: even top-of-the-line Android phones have weak Qualcomm CPUs, and sites use so much JavaScript that even high-end machines can struggle. So we may end up living with JPEG for a few more years. This format has been designed when CPUs had 25Mhz, and we've had over 25 years to optimize the implementations, so we get a lot of bang for the buck from it.
  - m-p-3 8y ago
    I hate that most websites are so heavy. I enjoyed browsing on my Kobo eReader but loading times of several seconds to a minutes makes it cumbersome. I limit my browsing to "lite" websites and HN suits that well. I also use the Pocket service because it's builtin and reduce the computational need required to read an article.
    - vanderZwan 8y ago
      > I hate that most websites are so heavy. Most of which has nothing to do with compression format, to be honest. No image compression format is going to save us from websites that use 2000x2000 sized photos for 200x200 author thumbnails.
      - jrockway 8y ago
        As far as I can tell, nobody has ever made it easy to fix this problem. I don't believe there is any webpack (etc.) plugin that will just recompress images for you. You have to do it manually. As a result, people don't do it, even those that have a proper "build process" for their website. Maybe I should write one, I guess.
        vanderZwan 8y ago
        Are you seriously suggesting that image resizing to match expected output size is an unsolved problem?
- mnutt 8y ago
  For a site with limited assets, people care more about bandwidth costs than storage costs and ideally your webserver can automatically transcode and serve the best format based on content negotiation. Except for JPEG2000, which Safari supports but does not include in its Accepts: header. So Akamai just sniffs the user-agent and serves JPEG2000 to anything that looks like Safari, whether they support it or not.
  - y4mi 8y ago
    Uuh, i thought the whole point of a static server was to not need an application that needs to prepare the content? It's supposed to be a dumb web server that accesses a folder structure or similar storage backed...
    - mnutt 8y ago
      Is that the point, though? I think the point is more that you tell the webserver "here is some content, I want you to serve it in the best way possible" and it figures out how to do it without any special configuration. In the same way I'd expect a "dumb" webserver to be able to negotiate gzip compression without having to point it at a directory of gzipped files. Edit: I agree that something like the pagespeed module rewriting html probably goes too far and html rewriting is fraught with weird edge cases, but content negotiation is something only a webserver (or CDN) can do.
      - Avamander 8y ago
        A directory of gzipped files is often incredibly useful, I've managed to reduce both latency and time it takes to load pages by just precompressing resources. It's more than incredibly useful on low-powered hardware like Raspberry Pis, some applications can be made to load in 3 seconds instead of 10.
        mnutt 8y ago
        That’s somewhat surprising that it makes that much of a difference, a Raspberry Pi 2 can gzip compress at around 20MB/s. Maybe there’s some inefficiencies in the webserver implementation, or just really big files?
- deleted 8y ago
  [deleted]
- pasta 8y ago
  This. The FLIF format still has a nice TODO list which containt: native browser support..
- bscphil 8y ago
  I work with video codecs a lot; to me the blocking on the AV1 picture looks like chroma subsampling rather than a compression artifact. It's much more noticeable on red than in other colors. Unfortunately, at least as of last time I checked, the AV1 still image converters were mostly only 4:2:0 and it's quite noticeable on some images. But this should go away entirely once they're used without subsampling. In other words: don't use that part of the picture to judge the quality of AV1.
  - mrob 8y ago
    It's definitely chroma upsampling artifacts. This is a problem with the decoder, not the encoder. The chroma channels are upscaled with nearest-neighbor filtering. I resampled them in ImageMagick with a Mitchell-Netravali cubic filter, which is reasonably cheap to calculate, and it looks much better: $ convert 400px-President_Barack_Obama.jpg.av1.mkv.png -colorspace YUV -channel 0 -separate Y.png $ convert 400px-President_Barack_Obama.jpg.av1.mkv.png -colorspace YUV -channel 1 -separate U.png $ convert 400px-President_Barack_Obama.jpg.av1.mkv.png -colorspace YUV -channel 2 -separate V.png $ convert U.png -filter point -resize 50% -filter Mitchell -resize 200% smoothed_U.png $ convert V.png -filter point -resize 50% -filter Mitchell -resize 200% smoothed_V.png $ convert Y.png smoothed_U.png smoothed_V.png -combine -set colorspace YUV -colorspace sRGB combined.png Result: https://imgur.com/a/AwoVmh1 https://imgur.com/a/AwoVmh1
- manigandham 8y ago
  This is best done at the edge/CDN layer now. Many of them support real-time optimization (along with resizing, etc) to send the best version to the client that's requesting it. Also recommend Cloudinary and Filestack for a completely managed assets system.
- btschaegg 8y ago
  > New formats are not supported on all browsers so you wind up having to support the mainstream formats as well as one or more new formats. Rather than benefiting from reduced storage costs, your storage costs get multiplied. There's a big thing that bugs me about that when it comes to the web: Software patents and licenses basically ruined wavelet-based image compression schemes on the web. Wavelets would have been ideal. Basically, the lossy compression would work about the same as the cosine transform in JPEG, but by using the discrete wavelet transform, you basically get thumbnailing built in: Imagine transmitting a tiny little thumbnail of your image. Then, you transmit the information used to derive a version 4x that size (2x width, 2x height) from that. Then, you transmit the data used to derive the next 4x layer. Rinse, repeat -- that's exactly how image data is organized after a wavelet transform. With some clever adaptions, this could have done away with so much cruft and complexity we have today (like image sets and @media queries for background images): Imagine just putting a 4K version of your background on the server and telling the browser how to render it (an <img> tag with a specified size, or background-size in css). The browser could then start loading the image until it realizes it has all the data for your resolution (say on your phone), and just terminate the transmission before the whole image is downloaded.
  - ksec 8y ago
    JPEG 2000 uses wavelet, and a few other I can't remember their names, none of them has better image / pref than the best ones on the market or soon to be released image format based on X Codec. So in reality much like Wavelet in video, it hasn't worked out well at all in real world implementation.
- ksec 8y ago
  > For instance, one I read a comment on the article here https://calendar.perfplanet.com/2018/is-avif-the-future-of-i.. https://calendar.perfplanet.com/2018/is-avif-the-future-of-i.... Jpeg was released the MPEG-1 ( H.261 ) era. Since then we have had H.262 / MPEG2, H.263 / Divx, H.264 /AVC, H.265 / HEVC, and the coming H.266 / VVC. If we consider the amount of improvement we got from Video Encoding, images hasn't changed much at all, we happen to fine tune the heck out of Jpeg. And I hardly call a new image format with 50% reduction bitrate compared to a nearly 30 years old tech a major breakthrough, and that is without mentioning the huge decoding resources requirement. There are many project that just improves on JPEG, Dropbox had a lossless compressor for JPEG, and Cant remember if Pik was originally based on jpeg as well. Unless there is a real major breakthrough that gives the current best JPG encoder quality at 25% or less its size, focusing on 200KB, 100KB, 50KB images size, ( i.e the new image will have to be 50KB, 25KB, 12.5KB ) while using within 2x decoding power. I am convinced our roadmap for Network bandwidth improvement in the next 3 - 5 years are going to make JPEG to last forever.
throwanem 8y ago
Apparently only IE and Edge support JPEG XR, so the format not being used on the web will happen on its own anyway.
- andreapaiola 8y ago
  Well... With Edge phasing to Blink noone will support JPEG XR
  - wereHamster 8y ago
    The Edge team could still decide to integrate their JPEG XR implementation into Blink. That would mean Chrome would gain support for JPEG XR.
    - 05 8y ago
      What makes you think the changes would be accepted into mainline?
      - CydeWeys 8y ago
        They don't have to be; Microsoft could maintain that as a separate fork/add-in to Blink. In theory it sounds like it should be highly modular, so that shouldn't be a huge ongoing maintenance burden. I don't know why they wouldn't just switch over to WebP in preference to that, though. It's better to have fewer of these formats around and the simplest thing to do is use the one that already has support.
        stordoff 8y ago
        > They don't have to be; Microsoft could maintain that as a separate fork/add-in to Blink Note that parent said "Chrome would gain support for JPEG XR", which I believe is what 05 is replying to.
    - andreapaiola 8y ago
      Could yes, probably not. Because very very few is using it.
SimeVidas 8y ago
It’s great to see a summary at the top of the article. However, the summary (and the article) leaves out a relevant question that immediately comes to mind: Are other image formats decoded off the main thread or is it just that decoding JPEG-XR is just more taxing compared to other formats, thus negating its advantages?
- JohnBooty 8y ago
  Yeah, this was my thought as well. I'm fairly certain the article may be poorly worded. Certainly standard JPGs are decoded on the CPU side as well, are they not? To the best of my knowledge, I don't think any browsers are doing JPG decoding in hardware; I think they use the GPU for primitives and compositing, etc. I would look forward to being corrected if I'm wrong!
  - pornel 8y ago
    The CPU portion of JPEG decoding is relatively cheap (just Huffman + RLE) and easy to parallelize (DCT blocks are independent). Conversion and upsampling of YCbCr can be done on the fly on the GPU, and this even saves memory compared to using raw RGB/RGBA bitmaps. Other formats use more expensive entropy coding, more block sizes, smoothing filters, and inter-block prediction which forces them to be decoded mostly serially.
    - dylan604 8y ago
      Isn't the JPEG instruction set built into the CPUs now anyways? I would be shocked to find out they are not seeing as MPEG2 has been part of the CPU since the Pentium days.
      - userbinator 8y ago
        There is no "JPEG instruction set" as such --- and what you may be referring to by MPEG2 is MMX, which is just a SIMD extension that certainly helps with JPEG and MPEG decoding, among other things.
mgamache 8y ago
Hardly surprising. JPEG is a very mature technology with deep support and optimizations going all the way to hardware support built into many (most?) cpus. If the real world savings are ~ 20%, it probably won't be worth it.
dfc 8y ago
The author says that JPEG-XRs are decoded on "the software-side on the CPU" a couple of times that it seems like it might be a term of art but it sounds very awkward to me. Is the TLDR actually "jpeg xr implementation is poorly optimized and negatively affects browser performance which decreased conversions."
k__ 8y ago
If the compression is really much better than with JPEG, there is probably a point where the connection is so slow, that JPEG-XR would be faster than JPEG, because the file-size would out-weight the decoding performance.
Proven 8y ago
Meh. You think you know the value users attribute to improved image quality and shorter battery life (let's not forget that most folks carry chargers or can charge at work and home). But you don't. Perhaps you shouldn't judge for others but merely notice this trade off.
subfay 8y ago
This piece gives a hint where Trivago as a company went with Holacracy. The lack of focus is so obvious and seems to let individual developers create some work which is not just poorly worded but also nonsense: Why should anyone test a format which is used by just two browsers with a very small market share and which will be never adopted by the company with the biggest market share (because Google has WebP). JPEG-XR is DOA and nobody should care and waste time on this (except the OP gone astray). I like that there is a tl;dr right at the beginning though.
- robin_reala 8y ago
  I think that’s unfair though. It’s not necessarily about percentages; potentially they could easily be seeing a few tens of millions of monthly visits from IE and Edge, and if putting in place an automated system to convert images to a (theoretically) more efficient system had increased conversions they’d be really happy for the payback. Thinking about it, assuming they’ve got a WebP output for Chrome already this could be as simple as adding one more build step with a different encoder, and one more negotiation rule on the CDN.
- jbreckmckye 8y ago
  I'm not sure I'd be quite so blunt, but I do agree with your argument in the main: this is so low on the list of priorities that I can't help but wonder if someone at Trivago thinks they've run out of any actual revenue-generating development work. It indicates to me that either the inmates are running the asylum - that devs are being given too much leeway to pursue personal interests - or (and?) that Trivago's day to day work is proving so dull that developers in some teams are inventing R&D work just to stay sane.
  - subfay 8y ago
    Yeah my tone was a bit to harsh and I didn't want to insult anyone but I am still wondering that nobody at Tribago gave him this feedback. However and hopwfully this useless piece of work leads him to some other more relevant stuff.
dahart 8y ago
> -1% negative impact on core business metrics including conversions in our test of JPEG-XRs [...] Let’s not use JPEG-XR on the Web. Is this an overreaction, or a subversive call to improve the decode perf of JPEG-XR? I'm certainly on the performance-is-a-feature bandwagon myself, but 1% ux impact is pretty small. And shouldn't we assume that decode perf will improve in the near future? Is there any reason to assume that this problem is permanent and unfixable?
- mediumdeviation 8y ago
  > but 1% impact is pretty small The best you can say here is that it’s within the margin of error, in which case the effect is so small it’s literally unmeasurable. In that case the null hypothesis still wins, and you should not switch. > Is there any reason to assume that this problem is permanent and unfixable? IE11 as tested in the article is not getting anything other than security patches. The users who use IE11 are probably also the users most affected by poor performance since they most likely have the oldest hardware.
  - dahart 8y ago
    Interesting response, thanks. Your comment gives me even more questions, though! > The best you can say here is that it’s within the margin of error What is preventing JPEG-XR from becoming faster and a net ux positive, tomorrow or in the future? Isn't the best I can say that today's results might be a bug or fluke or a quick and dirty implementation, and not a permanent problem? > IE11 as tested in the article is not getting anything other than security patches. The title and conclusion both said we should avoid JPEG-XR on the web, not that we should avoid it on IE11. You're right, the article focused on the problem with IE11. Is it already a net positive on Edge? (The article didn't say.) Should the shim to make it work in the old browser that fewer people use really override the potential benefits on the newer browser that more people are already using? Does it seem reasonable or unreasonable to make future web decisions depend on an old browser that's on life support today and has small and declining market share?
  - deleted 8y ago
    [deleted]
microcolonel 8y ago
This is a little bit wrong in that the other CPU-bound tasks during loading are generally in the main thread, whereas image decoding is usually on a different thread (so, depending on how many images you have, and how many CPUs, it may not increase overall page load time in reality). Now, overall time to decode still matters. If it's a gigantic hero image above the fold, then it could take a noticeable amount of time to decode, but it's probably still a win. With the only good JPEG 2000 encoder, its quality at small file sizes is easily better than webp, but in practice it seems virtually nobody uses JPEG 2000, because the open source encoders are not really that great (probably because the patents are a guessing game, although technically the known holders are willing to grant royalty-free licenses).
Yizahi 8y ago
At this point in time I suspect we do need to use anything not invented by Google, if only to slow down arrival of googlenet.
rhacker 8y ago
Going a little further than just JPEG-XR - definitely do performance tests for new releases. We had a bug that affected one page in our app. It made people cringe using the feature set that invoked it, but essentially something in angular was firing, and the thing it did caused a re-fire. It just kept looping when on these very specific pages in the app. I finally found it and killed it. But definitely don't go to production with a client CPU murdering bug. It can't be measured the same as network latency, etc... so you really need to use your apps. I was surprised to see a post from Trivago - I might have to book my next hotel through them.
jbreckmckye 8y ago
Exotic file formats, like exotic JavaScript language constructs, tend to perform poorly because they are not optimisation targets. And they are not optimisation targets because - obviously - no-one uses them. To my knowledge there is no particular intrinsic reason that JPEG-XR decoding cannot be moved off-thread. The solution to this problem is to start using these new technologies, so the browser vendors can prioritise work to make them performant. It is not to write articles that entrench that neglect and tie us to flawed, ancient technologies from the 1990s. It gets worse. If there's one kind of meme that survives a long time in the development world, it's performance memes. Rumours about poor perf invariably end up haunting technologies with FUD long after any issues have ceased to be relevant.
- inferiorhuman 8y ago
  Yep, it's kinda like Github and SSL/TLS. When they went SSL only it was a bit painful, but things have largely caught up and improved overall.
t0mbstone 8y ago
Regardless of the findings, I think it's funny that they think they were able to "statistically verify" a -1% impact on conversions... That's a hilarious thought to me. How can a 1% change in a sales funnel be considered statistically relevant?
Solar19 8y ago
I have several somewhat orthogonal thoughts on this: 1. This is exactly what I've feared and suspected about JPEG-XR, webp, and JPEG-2000 (which Safari ever so quietly supports). I especially wondered about JPEG-XR after reading how Microsoft had done some great things with GPU-based hardware acceleration for JPEG decoding in IE11: https://blogs.msdn.microsoft.com/ie/2013/09/12/using-hardware-to-decode-and-load-jpg-images-up-to-45-faster-in-internet-explorer-11/ https://blogs.msdn.microsoft.com/ie/2013/09/12/using-hardwar... I was puzzled by the fact that they never mention JPEG-XR and have never published a similar post about hardware acceleration for the more modern format. They've been incredibly lazy and unfocused on XR for years. It's effectively dead now. 2. The author of the don't use it post keeps saying it gets "software decoded". This implies that this is unusual. But it isn't. Does he think JPEGs are not software decoded? They are in Firefox, and probably every other browser but IE11 and Edge. Someone correct me if I'm wrong. Browser makers have been surprisingly inert on using GPUs for image decoding. libjpeg-turbo is used by Chrome and Firefox, and possibly Safari. It's pure CPU, albeit with some SIMD. That's "software decoding". Hardware acceleration normally means GPU, or a fixed function unit/ASIC like you see for some video codecs. 3. One percent? This was all about a -1% drop in some metric? What's the error margin? 4. He implies that there's a lot of JS in this SPA. This seems like a possible confound that won't necessarily apply to rigorous web developers who don't overload on JS (who are admittedly extremely rare in 2019). Sure, they showed that XR was using more CPU, but so do all post-JPEG formats. 5. Which leads to my last point. What about webp? He strangely never mentions it. It's like only JPEG and XR exist. webp performs very poorly in CPU efficiency compared to JPEG. What are trivago and Cloudinary seeing on that front? Is webp less taxing than XR?
- cflat 8y ago
  A few clarifying points: * the author specifically said -1% Conversion. If this were AWS, that would mean a daily reduction of $6million in revenue! * the author specifically says this is for IE users. This implies two things: a) webp is not an option and b) likely we are also talking about lower class of hardware (mom & dad). I would expect WebP had older hardware probably has the same kind of performance tax, but it's harder to identify because at least with IE/Edge there is an implied age-of-hardware

134 points by robin_reala 8y ago

From time to time I've run an analysis on the use of alternative image formats on the web and it never comes out to be worth it. New formats are not supported on all browsers so you wind up having to support the mainstream formats as well as one or more new formats. Rather than benefiting from reduced storage costs, your storage costs get multiplied. Performance is also a problem, particularly when people add "yet another polyfill" to get an image to work without native support. Once I get into eyeballing images closely I also end up questioning if the compression gains are real. For instance, one I read a comment on the article here https://calendar.perfplanet.com/2018/is-avif-the-future-of-images-on-the-web/ https://calendar.perfplanet.com/2018/is-avif-the-future-of-i... and looked closely I couldn't unsee that the AV1 had very different artifacts than the other compressed images. Not necessarily worse, but definitely different. Unlike the poster I liked the way the suit rendered but definitely there was a lot of blocking around edges that the poster interprets as "pixelation on the flag". Like many things, the costs of switching image formats are definite, but the benefits are questionable.

cosmie 8y ago
> From time to time I've run an analysis on the use of alternative image formats on the web and it never comes out to be worth it. I just checked the Google homepage, and even the New Year's Eve Google Doodle currently being shown is a good ol' GIF, rather than a modern format such as WebP. And this is with me using a desktop Chrome browser that definitively supports WebP. > Like many things, the costs of switching image formats are definite, but the benefits are questionable. One easy way to get some of the benefits without incurring the switching costs is to leverage Cloudflare Polish[1]. I've used it at a few places, and have had good experiences. Although the majority of gains came from optimizing images that were poorly optimized to begin with, rather than the incremental improvements that occurred from enabling WebP support. [1] https://support.cloudflare.com/hc/en-us/articles/360000607372-Using-Polish-to-compress-images-on-Cloudflare https://support.cloudflare.com/hc/en-us/articles/36000060737...
- Avamander 8y ago
  I don't like WebP because it's either it or HW accelerated browsing - latter I like a tad more because it saves me a bit of battery juice on my laptop.
  - TazeTSchnitzel 8y ago
    There's the same problem with Google insisting on VP8/9 for YouTube in Chrome. If your Mac only supports h.264 in hardware you'll get better battery life in Safari.
    - xvilka 8y ago
      Obviously it is a problem with Apple, not Google. People should just push it more to do the job.
      - TazeTSchnitzel 8y ago
        I don't think Google are free of responsibility. They are prioritising their lower bandwidth over the user's better power usage.
        vetinari 8y ago
        Apple has a bigger share of the blame. They purposely ignore any codec that is outside of their favorites, even if the hardware they ship does support it (i.e. Intel) or doesn't need any hardware support (Vorbis, Flac). On the other hand, they are quick to support any codec, that is problematic to implement elsewhere.
    - vosper 8y ago
      There's a Chrome extension that forces YouTube to stream h.264 https://chrome.google.com/webstore/detail/h264ify/aleakchihdccplidncghkekgioiakgal?hl=en https://chrome.google.com/webstore/detail/h264ify/aleakchihd...
    - acdha 8y ago
pornel 8y ago
In terms of compression, AV1 is a clear win. Note that artifacts that bother you are at about half the file size. You can increase the quality and still have an image that is both nicer and smaller than JPEG and WebP. However, as TFA mentions, better compression is currently a double-edged sword, because the CPU has become a bottleneck again: even top-of-the-line Android phones have weak Qualcomm CPUs, and sites use so much JavaScript that even high-end machines can struggle. So we may end up living with JPEG for a few more years. This format has been designed when CPUs had 25Mhz, and we've had over 25 years to optimize the implementations, so we get a lot of bang for the buck from it.
- m-p-3 8y ago
  I hate that most websites are so heavy. I enjoyed browsing on my Kobo eReader but loading times of several seconds to a minutes makes it cumbersome. I limit my browsing to "lite" websites and HN suits that well. I also use the Pocket service because it's builtin and reduce the computational need required to read an article.
  - vanderZwan 8y ago
    > I hate that most websites are so heavy. Most of which has nothing to do with compression format, to be honest. No image compression format is going to save us from websites that use 2000x2000 sized photos for 200x200 author thumbnails.
    - jrockway 8y ago
      As far as I can tell, nobody has ever made it easy to fix this problem. I don't believe there is any webpack (etc.) plugin that will just recompress images for you. You have to do it manually. As a result, people don't do it, even those that have a proper "build process" for their website. Maybe I should write one, I guess.
      - vanderZwan 8y ago
        Are you seriously suggesting that image resizing to match expected output size is an unsolved problem?
mnutt 8y ago
For a site with limited assets, people care more about bandwidth costs than storage costs and ideally your webserver can automatically transcode and serve the best format based on content negotiation. Except for JPEG2000, which Safari supports but does not include in its Accepts: header. So Akamai just sniffs the user-agent and serves JPEG2000 to anything that looks like Safari, whether they support it or not.
- y4mi 8y ago
  Uuh, i thought the whole point of a static server was to not need an application that needs to prepare the content? It's supposed to be a dumb web server that accesses a folder structure or similar storage backed...
  - mnutt 8y ago
    Is that the point, though? I think the point is more that you tell the webserver "here is some content, I want you to serve it in the best way possible" and it figures out how to do it without any special configuration. In the same way I'd expect a "dumb" webserver to be able to negotiate gzip compression without having to point it at a directory of gzipped files. Edit: I agree that something like the pagespeed module rewriting html probably goes too far and html rewriting is fraught with weird edge cases, but content negotiation is something only a webserver (or CDN) can do.
    - Avamander 8y ago
      A directory of gzipped files is often incredibly useful, I've managed to reduce both latency and time it takes to load pages by just precompressing resources. It's more than incredibly useful on low-powered hardware like Raspberry Pis, some applications can be made to load in 3 seconds instead of 10.
      - mnutt 8y ago
        That’s somewhat surprising that it makes that much of a difference, a Raspberry Pi 2 can gzip compress at around 20MB/s. Maybe there’s some inefficiencies in the webserver implementation, or just really big files?
deleted 8y ago
[deleted]
pasta 8y ago
This. The FLIF format still has a nice TODO list which containt: native browser support..
bscphil 8y ago
I work with video codecs a lot; to me the blocking on the AV1 picture looks like chroma subsampling rather than a compression artifact. It's much more noticeable on red than in other colors. Unfortunately, at least as of last time I checked, the AV1 still image converters were mostly only 4:2:0 and it's quite noticeable on some images. But this should go away entirely once they're used without subsampling. In other words: don't use that part of the picture to judge the quality of AV1.
- mrob 8y ago
  It's definitely chroma upsampling artifacts. This is a problem with the decoder, not the encoder. The chroma channels are upscaled with nearest-neighbor filtering. I resampled them in ImageMagick with a Mitchell-Netravali cubic filter, which is reasonably cheap to calculate, and it looks much better: $ convert 400px-President_Barack_Obama.jpg.av1.mkv.png -colorspace YUV -channel 0 -separate Y.png $ convert 400px-President_Barack_Obama.jpg.av1.mkv.png -colorspace YUV -channel 1 -separate U.png $ convert 400px-President_Barack_Obama.jpg.av1.mkv.png -colorspace YUV -channel 2 -separate V.png $ convert U.png -filter point -resize 50% -filter Mitchell -resize 200% smoothed_U.png $ convert V.png -filter point -resize 50% -filter Mitchell -resize 200% smoothed_V.png $ convert Y.png smoothed_U.png smoothed_V.png -combine -set colorspace YUV -colorspace sRGB combined.png Result: https://imgur.com/a/AwoVmh1 https://imgur.com/a/AwoVmh1
manigandham 8y ago
This is best done at the edge/CDN layer now. Many of them support real-time optimization (along with resizing, etc) to send the best version to the client that's requesting it. Also recommend Cloudinary and Filestack for a completely managed assets system.
btschaegg 8y ago
> New formats are not supported on all browsers so you wind up having to support the mainstream formats as well as one or more new formats. Rather than benefiting from reduced storage costs, your storage costs get multiplied. There's a big thing that bugs me about that when it comes to the web: Software patents and licenses basically ruined wavelet-based image compression schemes on the web. Wavelets would have been ideal. Basically, the lossy compression would work about the same as the cosine transform in JPEG, but by using the discrete wavelet transform, you basically get thumbnailing built in: Imagine transmitting a tiny little thumbnail of your image. Then, you transmit the information used to derive a version 4x that size (2x width, 2x height) from that. Then, you transmit the data used to derive the next 4x layer. Rinse, repeat -- that's exactly how image data is organized after a wavelet transform. With some clever adaptions, this could have done away with so much cruft and complexity we have today (like image sets and @media queries for background images): Imagine just putting a 4K version of your background on the server and telling the browser how to render it (an <img> tag with a specified size, or background-size in css). The browser could then start loading the image until it realizes it has all the data for your resolution (say on your phone), and just terminate the transmission before the whole image is downloaded.
- ksec 8y ago
  JPEG 2000 uses wavelet, and a few other I can't remember their names, none of them has better image / pref than the best ones on the market or soon to be released image format based on X Codec. So in reality much like Wavelet in video, it hasn't worked out well at all in real world implementation.
ksec 8y ago
> For instance, one I read a comment on the article here https://calendar.perfplanet.com/2018/is-avif-the-future-of-i.. https://calendar.perfplanet.com/2018/is-avif-the-future-of-i.... Jpeg was released the MPEG-1 ( H.261 ) era. Since then we have had H.262 / MPEG2, H.263 / Divx, H.264 /AVC, H.265 / HEVC, and the coming H.266 / VVC. If we consider the amount of improvement we got from Video Encoding, images hasn't changed much at all, we happen to fine tune the heck out of Jpeg. And I hardly call a new image format with 50% reduction bitrate compared to a nearly 30 years old tech a major breakthrough, and that is without mentioning the huge decoding resources requirement. There are many project that just improves on JPEG, Dropbox had a lossless compressor for JPEG, and Cant remember if Pik was originally based on jpeg as well. Unless there is a real major breakthrough that gives the current best JPG encoder quality at 25% or less its size, focusing on 200KB, 100KB, 50KB images size, ( i.e the new image will have to be 50KB, 25KB, 12.5KB ) while using within 2x decoding power. I am convinced our roadmap for Network bandwidth improvement in the next 3 - 5 years are going to make JPEG to last forever.

throwanem 8y ago

Apparently only IE and Edge support JPEG XR, so the format not being used on the web will happen on its own anyway.

andreapaiola 8y ago
Well... With Edge phasing to Blink noone will support JPEG XR
- wereHamster 8y ago
  The Edge team could still decide to integrate their JPEG XR implementation into Blink. That would mean Chrome would gain support for JPEG XR.
  - 05 8y ago
    What makes you think the changes would be accepted into mainline?
    - CydeWeys 8y ago
      They don't have to be; Microsoft could maintain that as a separate fork/add-in to Blink. In theory it sounds like it should be highly modular, so that shouldn't be a huge ongoing maintenance burden. I don't know why they wouldn't just switch over to WebP in preference to that, though. It's better to have fewer of these formats around and the simplest thing to do is use the one that already has support.
      - stordoff 8y ago
        > They don't have to be; Microsoft could maintain that as a separate fork/add-in to Blink Note that parent said "Chrome would gain support for JPEG XR", which I believe is what 05 is replying to.
  - andreapaiola 8y ago
    Could yes, probably not. Because very very few is using it.

SimeVidas 8y ago

It’s great to see a summary at the top of the article. However, the summary (and the article) leaves out a relevant question that immediately comes to mind: Are other image formats decoded off the main thread or is it just that decoding JPEG-XR is just more taxing compared to other formats, thus negating its advantages?

JohnBooty 8y ago
Yeah, this was my thought as well. I'm fairly certain the article may be poorly worded. Certainly standard JPGs are decoded on the CPU side as well, are they not? To the best of my knowledge, I don't think any browsers are doing JPG decoding in hardware; I think they use the GPU for primitives and compositing, etc. I would look forward to being corrected if I'm wrong!
- pornel 8y ago
  The CPU portion of JPEG decoding is relatively cheap (just Huffman + RLE) and easy to parallelize (DCT blocks are independent). Conversion and upsampling of YCbCr can be done on the fly on the GPU, and this even saves memory compared to using raw RGB/RGBA bitmaps. Other formats use more expensive entropy coding, more block sizes, smoothing filters, and inter-block prediction which forces them to be decoded mostly serially.
  - dylan604 8y ago
    Isn't the JPEG instruction set built into the CPUs now anyways? I would be shocked to find out they are not seeing as MPEG2 has been part of the CPU since the Pentium days.
    - userbinator 8y ago
      There is no "JPEG instruction set" as such --- and what you may be referring to by MPEG2 is MMX, which is just a SIMD extension that certainly helps with JPEG and MPEG decoding, among other things.

mgamache 8y ago

Hardly surprising. JPEG is a very mature technology with deep support and optimizations going all the way to hardware support built into many (most?) cpus. If the real world savings are ~ 20%, it probably won't be worth it.

dfc 8y ago

The author says that JPEG-XRs are decoded on "the software-side on the CPU" a couple of times that it seems like it might be a term of art but it sounds very awkward to me. Is the TLDR actually "jpeg xr implementation is poorly optimized and negatively affects browser performance which decreased conversions."

k__ 8y ago

If the compression is really much better than with JPEG, there is probably a point where the connection is so slow, that JPEG-XR would be faster than JPEG, because the file-size would out-weight the decoding performance.

Proven 8y ago

Meh. You think you know the value users attribute to improved image quality and shorter battery life (let's not forget that most folks carry chargers or can charge at work and home). But you don't. Perhaps you shouldn't judge for others but merely notice this trade off.

subfay 8y ago

This piece gives a hint where Trivago as a company went with Holacracy. The lack of focus is so obvious and seems to let individual developers create some work which is not just poorly worded but also nonsense: Why should anyone test a format which is used by just two browsers with a very small market share and which will be never adopted by the company with the biggest market share (because Google has WebP). JPEG-XR is DOA and nobody should care and waste time on this (except the OP gone astray). I like that there is a tl;dr right at the beginning though.

robin_reala 8y ago
I think that’s unfair though. It’s not necessarily about percentages; potentially they could easily be seeing a few tens of millions of monthly visits from IE and Edge, and if putting in place an automated system to convert images to a (theoretically) more efficient system had increased conversions they’d be really happy for the payback. Thinking about it, assuming they’ve got a WebP output for Chrome already this could be as simple as adding one more build step with a different encoder, and one more negotiation rule on the CDN.
jbreckmckye 8y ago
I'm not sure I'd be quite so blunt, but I do agree with your argument in the main: this is so low on the list of priorities that I can't help but wonder if someone at Trivago thinks they've run out of any actual revenue-generating development work. It indicates to me that either the inmates are running the asylum - that devs are being given too much leeway to pursue personal interests - or (and?) that Trivago's day to day work is proving so dull that developers in some teams are inventing R&D work just to stay sane.
- subfay 8y ago
  Yeah my tone was a bit to harsh and I didn't want to insult anyone but I am still wondering that nobody at Tribago gave him this feedback. However and hopwfully this useless piece of work leads him to some other more relevant stuff.

dahart 8y ago

> -1% negative impact on core business metrics including conversions in our test of JPEG-XRs [...] Let’s not use JPEG-XR on the Web. Is this an overreaction, or a subversive call to improve the decode perf of JPEG-XR? I'm certainly on the performance-is-a-feature bandwagon myself, but 1% ux impact is pretty small. And shouldn't we assume that decode perf will improve in the near future? Is there any reason to assume that this problem is permanent and unfixable?

mediumdeviation 8y ago
> but 1% impact is pretty small The best you can say here is that it’s within the margin of error, in which case the effect is so small it’s literally unmeasurable. In that case the null hypothesis still wins, and you should not switch. > Is there any reason to assume that this problem is permanent and unfixable? IE11 as tested in the article is not getting anything other than security patches. The users who use IE11 are probably also the users most affected by poor performance since they most likely have the oldest hardware.
- dahart 8y ago
  Interesting response, thanks. Your comment gives me even more questions, though! > The best you can say here is that it’s within the margin of error What is preventing JPEG-XR from becoming faster and a net ux positive, tomorrow or in the future? Isn't the best I can say that today's results might be a bug or fluke or a quick and dirty implementation, and not a permanent problem? > IE11 as tested in the article is not getting anything other than security patches. The title and conclusion both said we should avoid JPEG-XR on the web, not that we should avoid it on IE11. You're right, the article focused on the problem with IE11. Is it already a net positive on Edge? (The article didn't say.) Should the shim to make it work in the old browser that fewer people use really override the potential benefits on the newer browser that more people are already using? Does it seem reasonable or unreasonable to make future web decisions depend on an old browser that's on life support today and has small and declining market share?
- deleted 8y ago
  [deleted]

microcolonel 8y ago

This is a little bit wrong in that the other CPU-bound tasks during loading are generally in the main thread, whereas image decoding is usually on a different thread (so, depending on how many images you have, and how many CPUs, it may not increase overall page load time in reality). Now, overall time to decode still matters. If it's a gigantic hero image above the fold, then it could take a noticeable amount of time to decode, but it's probably still a win. With the only good JPEG 2000 encoder, its quality at small file sizes is easily better than webp, but in practice it seems virtually nobody uses JPEG 2000, because the open source encoders are not really that great (probably because the patents are a guessing game, although technically the known holders are willing to grant royalty-free licenses).

Yizahi 8y ago

At this point in time I suspect we do need to use anything not invented by Google, if only to slow down arrival of googlenet.

rhacker 8y ago

Going a little further than just JPEG-XR - definitely do performance tests for new releases. We had a bug that affected one page in our app. It made people cringe using the feature set that invoked it, but essentially something in angular was firing, and the thing it did caused a re-fire. It just kept looping when on these very specific pages in the app. I finally found it and killed it. But definitely don't go to production with a client CPU murdering bug. It can't be measured the same as network latency, etc... so you really need to use your apps. I was surprised to see a post from Trivago - I might have to book my next hotel through them.

jbreckmckye 8y ago

Exotic file formats, like exotic JavaScript language constructs, tend to perform poorly because they are not optimisation targets. And they are not optimisation targets because - obviously - no-one uses them. To my knowledge there is no particular intrinsic reason that JPEG-XR decoding cannot be moved off-thread. The solution to this problem is to start using these new technologies, so the browser vendors can prioritise work to make them performant. It is not to write articles that entrench that neglect and tie us to flawed, ancient technologies from the 1990s. It gets worse. If there's one kind of meme that survives a long time in the development world, it's performance memes. Rumours about poor perf invariably end up haunting technologies with FUD long after any issues have ceased to be relevant.

inferiorhuman 8y ago
Yep, it's kinda like Github and SSL/TLS. When they went SSL only it was a bit painful, but things have largely caught up and improved overall.

t0mbstone 8y ago

Regardless of the findings, I think it's funny that they think they were able to "statistically verify" a -1% impact on conversions... That's a hilarious thought to me. How can a 1% change in a sales funnel be considered statistically relevant?

Solar19 8y ago

I have several somewhat orthogonal thoughts on this: 1. This is exactly what I've feared and suspected about JPEG-XR, webp, and JPEG-2000 (which Safari ever so quietly supports). I especially wondered about JPEG-XR after reading how Microsoft had done some great things with GPU-based hardware acceleration for JPEG decoding in IE11: https://blogs.msdn.microsoft.com/ie/2013/09/12/using-hardware-to-decode-and-load-jpg-images-up-to-45-faster-in-internet-explorer-11/ https://blogs.msdn.microsoft.com/ie/2013/09/12/using-hardwar... I was puzzled by the fact that they never mention JPEG-XR and have never published a similar post about hardware acceleration for the more modern format. They've been incredibly lazy and unfocused on XR for years. It's effectively dead now. 2. The author of the don't use it post keeps saying it gets "software decoded". This implies that this is unusual. But it isn't. Does he think JPEGs are not software decoded? They are in Firefox, and probably every other browser but IE11 and Edge. Someone correct me if I'm wrong. Browser makers have been surprisingly inert on using GPUs for image decoding. libjpeg-turbo is used by Chrome and Firefox, and possibly Safari. It's pure CPU, albeit with some SIMD. That's "software decoding". Hardware acceleration normally means GPU, or a fixed function unit/ASIC like you see for some video codecs. 3. One percent? This was all about a -1% drop in some metric? What's the error margin? 4. He implies that there's a lot of JS in this SPA. This seems like a possible confound that won't necessarily apply to rigorous web developers who don't overload on JS (who are admittedly extremely rare in 2019). Sure, they showed that XR was using more CPU, but so do all post-JPEG formats. 5. Which leads to my last point. What about webp? He strangely never mentions it. It's like only JPEG and XR exist. webp performs very poorly in CPU efficiency compared to JPEG. What are trivago and Cloudinary seeing on that front? Is webp less taxing than XR?

cflat 8y ago
A few clarifying points: * the author specifically said -1% Conversion. If this were AWS, that would mean a daily reduction of $6million in revenue! * the author specifically says this is for IE users. This implies two things: a) webp is not an option and b) likely we are also talking about lower class of hardware (mom & dad). I would expect WebP had older hardware probably has the same kind of performance tax, but it's harder to identify because at least with IE/Edge there is an implied age-of-hardware