14 ms·
Zopfli Optimization: Literally Free Bandwidth
- jMyles 11y ago> it's about as close as it gets to literally free bandwidth in our line of work. Listen, I'm all about inventive ways to lighten the yoke of static media on the web today. But, in two important ways, this is not "literally free bandwidth": 1) The weaker: Despite the tone of obviousness in this article, it acknowledges that the choice of which technology to use is not made for you: there are edge cases where other methodologies are indeed superior. So, far from being free, these sorts of solution do have a time cost. 2) The stronger: We live in a world where, on a great day, the user's realized downstream bandwidth is 20% their LAN connection; their upstream 5% or less. Connecting to a next-door neighbor via a conventional web application served through a typical corporate ISP probably means pushing packets a thousand miles or more, only for them to come back into our community. Complicating this issue: our name service and certificate distribution are implemented in a way that is reasonably called "incorrect." Our ISPs have a "speak when spoken to mentality" about connectivity, and competition is rare. A solution bragging "literally free bandwidth" needs to service this concern - let me transfer a piece of media to a next door neighbor utilizing the other 95% of my network interface upstream capacity. That I'll call free bandwidth.
- cbsmith 11y agoBoth scenarios are free bandwidth.
- arcameron 11y agoRegarding the avatar images: Why not a <div> with a border-radius & background color? Seems you could achieve the same thing without another HTTP request (1 for each unique avatar), no need to zopfli 45,000 unique files.
- bunkat 11y agoI was confused by this as well, you could also support more colors this way. Since it's just a circle with a letter, I don't think there would be concerns with crispness of the icon.
- kbuck 11y agoThese are default (placeholder) avatars. Users can upload their own avatar image as well. They're serving PNGs so they don't have to deal with the case where users using default avatars get HTML output whereas users who have uploaded an avatar get an image inserted.
- cfcef 11y agoSee the comments: lots of tweaking and unreliability and not guaranteed to work in emails, apparently.
- codinghorror 11y agoAlso font and alignment issues cross browser can be brutal. There is a lot to be said for a tiny image that works everywhere, on any device.
- qopp 11y agoWhat about an svg?
- other_herbert 11y agoThat wont work in email... I'm generating a usage graph (svg) that is emailed and the graph will not display in the client.. Must be opened in a browser
- ZeroGravitas 11y agoSeems like you could use an SVG avatar for clients that can handle it and fallback to PNG for email.
- Reedx 11y agoThat's what I was thinking too, but they really want it to be pixel perfect in every browser, email, etc. They have a good in-depth discussion about all the options and pros/cons of each approach here: https://meta.discourse.org/t/optimizing-letter-avatar-rendering/33082 https://meta.discourse.org/t/optimizing-letter-avatar-render...
- legulere 11y agoAnd you could save even more bandwidth if you changed the PNG standard to also allow Brotli compression. Even more probably if you created a better file format. That's not free anymore, but it's technologically easily possible to drastically reduce the amount of resources we use. What holds us back is that it's hard to get other people to do stuff like supporting new file formats or even have a better output in their image manipulation program.
- eridius 11y agoIf you change the PNG standard, it's no longer PNG. If existing PNG decoders can't decode it, then all you've done is invented a new format that looks very similar to PNG.
- legulere 11y agoSomething similar was made with WOFF. Version 1 used deflate, version 2 uses brotli. You could now argue semantics wether WOFF2 still is WOFF, but that doesn't matter at all. The point I wanted to get across here is that it's a relatively easy change
- viraptor 11y agoWe can't even use jpg2000 everywhere yet. Maybe in another 15 years we can upgrade png...
- laarc 11y agojpg2000 is unfortunately encumbered by patents. I was going to use it for a certain project a long time ago, but had to abandon the idea due to that. It's a shame, because many of the ideas in jpg2000 are very valuable for the world.
- Houshalter 11y agoSome of these compression patents are expiring now. I don't know if they were ever enforced to begin with. The wikipedia page is confusing, but suggests that the JPEG committee got the relevant patent holders to agree to let the patents be used freely.
- roeme 11y agoSomewhat OT; As a swiss, the omitted ö is really beginning to bug me. It's Zöpfli. Gopferteckel. (The second word is a somewhat soft cuss word - But don't try to use it as a non-native). Also, you can't "zopfli" something - it's a noun! You "zöpf" - or, since we're in the alemannic german space; "zöpfle". /rant /vent
- m_st 11y agoJa das git mier o so.
- sschueller 11y agoGenau so mit Brötli.
- mech4bg 11y agoHow do you feel when someone badly appropriates a word like this and it gets really big, like Uber? :) I understand why the umlaut gets dropped, but it super bugs me too - especially as it's as easy as dropping an 'e' next to the letter if you don't have an easy way to add an umlaut. Keeping things slightly off-topic for a moment - I love Swiss German. I lived in Bavaria and speak reasonably fluent German, so I can understand bits of it, but it's mostly incomprehensible to me, but in a delightful way. Its 'sing-song' nature and fun pronunciation are the best. I also found in Switzerland people were far more willing to tolerate and respond to my German than in Germany (where people tend to switch to English), which I very much appreciated when I was still learning the language.
- _ak 11y agoGiven that zopfli was created in the Google office in Zürich by non-Swiss people, blame it on your fellow Swiss citizens for lack of intervention.
- jrcii 11y agoYou Swiss and your fancy ö's
- dikaiosune 11y agoDoesn't this happen in every language that borrows words? Accent/diacritic marks appear and disappear to suit the speakers/writers of the "guest" language. Also regarding our US-centric naming conventions wouldn't it be very unconventional to include a non-ASCII character in the name of an imported library or called function?
- eridius 11y agoAre the reported Zopfli numbers for PNGs achieved by recompressing the original, or by recompressing the output of PNGout? The wikipedia page for PNGout says > PNGOUT also performs automatic bit depth, color, and palette reduction where appropriate. Assuming the Zopfli numbers were created by recompressing the original, I wonder if there's any further savings to be had by recompressing the output of PNGout? Alternatively, PNGcrush can also do the same sort of lossless bit depth and palette reduction, so I'd be curious about the combination of PNGcrush + Zopfli as well.
- danielvf 11y agoThat doesn't matter - both systems decompress the losses PNGs before recompressing them.
- HeyImAlex 11y agoIt does kind of matter, because compression isn't the only form of optimization that can be done. Consider a 32 bit png that is actually bilevel with no transparency. If zopfli just recompresses, PNGOut will still make smaller output because it does bit depth reductions.
- danielvf 11y agoThe zopfliPNG program does do bit depth reduction if that's possible. It even tests to see if a given small image that would be possible to bit depth reduce would actually be smaller by it keeping 32 bit, since the added pallet overhead may cost more than the gain of a smaller pixel storage cost.
- monochromatic 11y agoZopfli is a general purpose compressor. It's not unpacking the png data before compressing it.
- danielvf 11y agoIt does decompress the source PNG and build a new file from the pixels. You can see the source here. https://github.com/google/zopfli/blob/master/src/zopflipng/zopflipng_lib.cc https://github.com/google/zopfli/blob/master/src/zopflipng/z...
- callumjones 11y agoYou could probably have a worker system that eventually produces a Zopfli compressed file. User uploads a provided PNG, you perform the quickest compression compression but then queue up a Zopfli compression. Up front you're only returning the less compressed file but after time you begin serving up the lesser bandwidth file. If the uploaded file or associated post is deleted then you can wipe it from the queue.
- brianwawok 11y agoOr maybe don't even Zopfli it until it gets 100 some hits. Cost some amount of cents of CPU time to do the compress... unless maybe you made the uploaded so it.
- codinghorror 11y agoYeah the smart thing to do is schedule "important" user submitted images for heavier recompression, but even then you might get in trouble for very large PNGs, and need to gate based on dimensions. For huge PNGs I am not sure the 80x - 160x time penalty is tenable, you might be talking 30 minutes in some cases.
- jzelinskie 11y agoHis example use case seems a little contrived. We have a similar default avatar at Quay.io, but we handle it all client-side with CSS. There's some more free bandwidth!
- quadrature 11y agoa little, but images are important if you want to ensure consistency across different platforms.
- HeyImAlex 11y agoIf anyone is interested in PNG optimization in general, a wrote an article about it a while back. The deflate step is only one part of it! http://heyimalex.com/journal/png-optimization http://heyimalex.com/journal/png-optimization
- millstone 11y ago> It's a smaller file to send over the wire, and the smaller the file, the faster the decompression. Can someone elaborate on this? Why do smaller files decompress faster? > However, remember that decompression is still the same speed, and totally safe Wait, what? Didn't we just establish that it's faster to decompress?
- ndesaulniers 11y ago> Why do smaller files decompress faster? The same reason why "no code" is faster than optimized code. ;)
- millstone 11y agoThe trivial identity "compressor" is fastest to decompress, since it requires no work. Likewise PNG has literal blocks, which are longer but faster to decompress. So is it actually true that Zopfli-compressed files decompress faster?
- deleted 11y ago[deleted]
- arjie 11y agoThe first faster refers to total time spent decompressing being lower. The second time speed is mentioned he's saying the rate is the same. With constant rate, smaller loads are completed faster. And the rate is constant but the load is smaller.
- millstone 11y agoDecompression speed is normally measured in bytes per second of output, not input. Since the output is the same, two algorithms that decompress at the same rate take the same time, regardless of compression ratio. Googling finds a benchmark [1] which shows Zopfli decompression rate as intermediate between deflate 1 and deflate 9. On two our of three tests, Zopfli produces smaller files but is slower to decompress than deflate 9. So "the smaller the file, the faster the decompression" isn't correct. [1] PDF: https://cran.r-project.org/web/packages/brotli/vignettes/brotli-2015-09-22.pdf https://cran.r-project.org/web/packages/brotli/vignettes/bro...
- deleted 11y ago[deleted]
- sandGorgon 11y agothe lions share of image creation these days happens on the mobile. and I cant seem to find a Zopfli library for android. I wonder why - is it too heavy duty for a mobile CPU ?
- Nutmog 11y agoPhotos maybe, but who makes images that would suit PNG on mobile?
- Qwertious 11y agoSounds like there should be a system to JIT images with this.
- ck2 11y agoBatch files which go though all the tools and find the smallest sizes for various images and .gz files http://css-ig.net/tools/ http://css-ig.net/tools/ Personally I've found this tool is faster and does a better job than most others and it's free: http://psydk.org/pngoptimizer http://psydk.org/pngoptimizer
- Rygu 11y agoIt's not even just about bandwidth. You're reducing page load time, and therefore increasing revenue. http://www.fastcompany.com/1825005/how-one-second-could-cost-amazon-16-billion-sales http://www.fastcompany.com/1825005/how-one-second-could-cost... If you're a designer/developer on Mac I would strongly recommend ImageOptim (https://imageoptim.com/ https://imageoptim.com/). It supports Zopfli and has a simple drag-n-drop user interface.
- vanderZwan 11y agoOn Linux there is Trimage: http://trimage.org/ http://trimage.org/
- leni536 11y ago> ~250 color schemes Theoretically one could use one of the indexed PNG formats and only change the palette. I don't think that those avatar images use too much number of colors (even with anti-aliasing) so 8 bit indexed PNG should be more than enough.
- jonsneyers 11y agoThe original PNG of that PBF comic is 671,012 bytes. ZopfliPNG crunches it down to 585,117 bytes. Not bad, but if you use newer image formats, you can do better. Lossless WebP brings it down to 429,696 bytes (using -lossless -m 6 -q 100) FLIF with default options (which means interlaced for progressive decoding) takes it down to 322,858 bytes. Non-interlaced FLIF reduces it further to 302,551 bytes.
- manigandham 11y agoThis is interesting - but really don't we have better things like FLIF that should be gaining developer momentum first? That would solve basically everything wrong with today's decades old formats.
- bartvk 11y agoJust checked; on my MacBook it installs with a quick $ brew install zopfli That's assuming you have Homebrew installed, if not: http://brew.sh http://brew.sh
- jibsen 11y agoI don't think tools like pngquant should be so easily dismissed. With the pixel density of todays monitors, the lossy changes they introduce can be very hard to see (which may be surprising for those of us who remember the pixelated horrors of Floyd-Steinberg from a couple of decades ago). For some image types, lossy png has the huge advantage over jpg at the same file size that they have no jpg artifacts. 671.012 original 584.677 zopflipng -m 580.180 zopflipng -m --lossy_transparent 576.637 pngwolf --max-stagnate-time=0 --max-time=300 --normalize-alpha --strip-optional 190.598 pngquant --speed 1 179.638 pngquant + pngwolf