14 ms·

WebP: The WebPage Compression Format

552 points by Kubuxu 2y ago

butz 2y ago
Dropping google fonts should improve page load time a bit too, considering those are loaded from remote server that requires additional handshake.
- kevindamm 2y ago
  ..but if enough other sites are also using that font then it may already be available locally.
  - pornel 2y ago
    This has stopped working many years ago. Every top-level domain now has its own private cache of all other domains. You likely have dozens of copies of Google Fonts, each in a separate silo, with absolutely zero reuse between websites. This is because a global cache use to work like a cookie, and has been used for tracking.
    - kevindamm 2y ago
      ah, I had forgotten about that, you're right. well at least you don't have to download it more than once for the site, but first impressions matter yeah
    - madeofpalk 2y ago
      Where "many years ago" is... 11 years ago for Safari! https://bugs.webkit.org/show_bug.cgi?id=110269 https://bugs.webkit.org/show_bug.cgi?id=110269
  - SushiHippie 2y ago
    No, because of cache partitioning, which has been a thing for a while. https://developer.chrome.com/blog/http-cache-partitioning https://developer.chrome.com/blog/http-cache-partitioning
next_xibalba 2y ago
If only we hadn't lost Jan Sloot's Digital Coding System [1], we'd be able to transmit GB in milliseconds across the web! [1] https://en.wikipedia.org/wiki/Sloot_Digital_Coding_System https://en.wikipedia.org/wiki/Sloot_Digital_Coding_System
- supriyo-biswas 2y ago
  This claim itself is probably a hoax and not relevant to the article at hand; but these days with text-to-image models and browser support, you could probably do something like <img prompt="..."> and have the browser render something that matches the description, similar to the "cookbook" analogy used in the Wikipedia article.
  - Lorin 2y ago
    That's an interesting concept, although it would generate a ton of bogomips since each client has to generate the image themselves instead of one time on the server. You'd also want "seed" and "engine" attributes to ensure all visitors see the same result.
    - LeoPanthera 2y ago
      You could at least push the work closer to the edge, by having genAI servers on each LAN, and in each ISP, similar to the idea of a caching web proxy before HTTPS rendered them impossible.
      - lucianbr 2y ago
        Push the work closer to the edge, and multiply it by quite a lot. Generate each image many times. Why would we want this? Seems like the opposite of caching in a sense.
        sroussey 2y ago
        If you are reading a web page on mars and bandwidth is more precious than processing power, then <img prompt=“…”> might make sense. Not so much for us on earth however.
niceguy4 2y ago
Not to side track the conversation but to side track the conversation, has there been many other major WebP exploits like the serious one in the past?
- deleted 2y ago
  [deleted]
pjmlp 2y ago
[flagged]
- solardev 2y ago
  Actually, the article is about using that same image compression format to compress web pages (text and HTML) instead, which sometimes works even better than GZIP. Brotli still wins for most cases, though. It's a confusing title, though. I thought what you did at first, too.
  - pjmlp 2y ago
    Indeed...
- afavour 2y ago
  Apparently we don’t read articles before commenting on them
  - pjmlp 2y ago
    We do, doesn't change the fact of giving another meaning to WebP, or its hack.
    - afavour 2y ago
      Of course it does. The title is a play on using WebP to compress web page content, it’s not a real name.
    - Dylan16807 2y ago
      Well if that's your criticism then I don't know why you linked the wikipedia page for WebP. The article explains the image format, so that link comes across as a mistaken reaction to the title. Also I disagree that it really gives "another meaning".
- croes 2y ago
  The article is about hack to use the WebP from your link as a web page compression format.
BugsJustFindMe 2y ago
> the longest post on my site, takes 92 KiB instead of 37 KiB. This amounts to an unnecessary 2.5x increase in load time Sure, if you ignore latency. In reality it's an unnecessary 0.001% increase in load time because that size increase isn't enough to matter vs the round trip time. And the time you save transmitting 55 fewer KiB is probably less than the time lost to decompression. :p While fun, I would expect this specific scenario to actually be worse for the user experience not better. Speed will be a complete wash and compatibility will be worse.
- jsnell 2y ago
  That size difference is large enough to make a difference in the number of round trips required (should be roughly one fewer roundtrip with any sensible modern value for the initial congestion window). Won't be a 2.5x difference, but also not 0.001%.
  - BugsJustFindMe 2y ago
    You don't need a new roundtrip for every packet. That would be devastating for throughput. One vs two vs three file packets get acked as a batch either way, not serially. Also when you get to the end, you then see > The actual savings here are moderate: the original is 88 KiB with gzip, and the WebP one is 83 KiB with gzip. In contrast, Brotli would provide 69 KiB. At 69 KiB you're still over the default TCP packet max, which means both cases transmit the same number of packets, one just has a bunch of extra overhead added for the extra JavaScript fetch, load, and execute. The time saved here is going to be negligible at best anyway, but there looks to be actually negative because we're burning time without reducing the number of needed packets at all.
    - jsnell 2y ago
      Those numbers are for a different page. For the original page, the article quotes 44 kB with this method vs. 92 kB for gzip. > At 69 KiB you're still over the default TCP packet max, which means both cases transmit the same number of packets, What? No, they absolutely don't transmit the same number of packets. Did you mean some other word?
98469056 2y ago
While peeking at the source, I noticed that the doctype declaration is missing a space. It currently reads <!doctypehtml>, but it should be <!doctype html>
- BugsJustFindMe 2y ago
  Maybe their javascript adds it back in :)
- palsecam 2y ago
  `<!doctype html>` can be minified into `<!doctypehtml>`. It’s, strictly speaking, invalid HTML, but it still successfully triggers standards mode. See https://GitHub.com/kangax/html-minifier/pull/970 https://GitHub.com/kangax/html-minifier/pull/970 / https://HTML.spec.WHATWG.org/multipage/parsing.html#parse-error-missing-whitespace-before-doctype-name https://HTML.spec.WHATWG.org/multipage/parsing.html#parse-er... (I too use that trick on https://FreeSolitaire.win https://FreeSolitaire.win)
  - saagarjha 2y ago
    Why would you do this?
    - palsecam 2y ago
      To have FreeSolitaire.win homepage be only 20.7 kB over the wire. That’s for the whole game: graphics are inline SVGs, JS & CSS are embedded in <script> and <style> elements.
      - saagarjha 2y ago
        Right but surely skipping that space wasn't that much of a win
      - lifthrasiir 2y ago
        To be clear, you can minimize much further than what you already have. For example you haven't fully minified SVG, which can be also replaced with just symbol characters. Saving a single space from `<!doctype html>` is insignificant compared to that.
gkbrk 2y ago
> Why readPixels is not subject to anti-fingerprinting is beyond me. It does not sprinkle hardly visible typos all over the page, so that works for me. > keep the styling and the top of the page (about 8 KiB uncompressed) in the gzipped HTML and only compress the content below the viewport with WebP Ah, that explains why the article suddenly cut off after a random sentence, with an empty page that follows. I'm using LibreWolf which disables WebGL, and I use Chromium for random web games that need WebGL. The article worked just fine with WebGL enabled, neat technique to be honest.
- niutech 2y ago
  It isn't neat as long as it doesn't work with all modern web browsers (even with fingerprinting protection) and doesn't have a fallback for older browsers. WWW should be universally accessible and progressively enhanced, starting with plain HTML.
  - afavour 2y ago
    It isn’t a serious proposal. It’s a creative hack that no one, author included is suggesting should be used in production.
    - account42 2y ago
      The author is using the hack in "production".
      - afavour 2y ago
        Debatable. I consider a personal blog to be fair game for experimentation, it’s not a paid for project and has no customer.
  - kjhcvkek77 2y ago
    This philosophy hands your content on a silver platter to ai companies, so they can rake in money while giving nothing back to the author.
Retr0id 2y ago
I've used this trick before! Oddly enough I can't remember what I used it for (perhaps just to see if I could), and I commented on it here: https://gist.github.com/gasman/2560551?permalink_comment_id=4431196#gistcomment-4431196 https://gist.github.com/gasman/2560551?permalink_comment_id=... Edit: I found my prototype from way back, I guess I was just testing heh: https://retr0.id/stuff/bee_movie.webp.html https://retr0.id/stuff/bee_movie.webp.html
- lucb1e 2y ago
  That page breaks my mouse gestures add-on! (Or, I guess we don't have add-ons anymore but rather something like script injections that we call extensions, yay...) Interesting approach to first deliver 'garbage' and then append a bit of JS to transform it back into a page. The inner security nerd in me wonders if this might open up attacks if you would have some kind of user-supplied data, such as a comment form. One could probably find a sequence of bytes to comment that will, after compression, turn into a script tag, positioned (running) before yours would?
  - Retr0id 2y ago
    Yeah that's plausible, you definitely don't want any kind of untrusted data in the input. Something I wanted to do but clearly never got around to, was figuring out how to put an open-comment sequence (<!--) in a header somewhere, so that most of the garbage gets commented out
- lifthrasiir 2y ago
  In my experience WebP didn't work well for the general case where this technique is actually useful (i.e. data less than 10 KB), because the most additions of WebP lossless to PNG are about modelling and not encoding, while this textual compression would only use the encoding part of WebP.
raggi 2y ago
Chromies got in the way of it for a very long time, but zstd is now coming to the web too, as it’s finally landed in chrome - now we’ve gotta get safari onboard
- CharlesW 2y ago
  Looks like it's on the To Do list, at least: https://webkit.org/standards-positions/#position-168 https://webkit.org/standards-positions/#position-168
- mananaysiempre 2y ago
  I’d love to go all-Zstandard, but in this particular case, as far as I know, Brotli and Zstandard are basically on par at identical values of decompressor memory consumption.
  - simondotau 2y ago
    Realistically, everything should just support everything. There’s no reason why every (full featured) web server and every (full featured) web browser couldn’t support all compelling data compression algorithms. Unfortunately we live in a world where Google decides to rip JPEG-XL support out of Chrome for seemingly no reason other than spite. If the reason was a lack of maturity in the underlying library, fine, but that wasn’t the reason they offered.
    - madeofpalk 2y ago
      > There’s no reason Of course, there is - and it's really boring. Prioritisation, and maintenance. It's a big pain to add, say, 100 compressions formats and support them indefinitely, especially with little differentiation between them. Once we agree on what the upper bound of useless formats is, we can start to negotiate what the lower limit is.
      - raggi 2y ago
        It’s not prioritization of the code - it’s relatively little implementation and low maintenance but security is critical here and everyone is very afraid of compressors because gzip, jpeg and various others were pretty bad. Zstd, unlike lz4 before it (at least early on) has a good amount of tests and fuzzing. The implementation could probably be safer (with fairly substantial effort) but having test and fuzzing coverage is a huge step forward over prior industry norms
csjh 2y ago
I think the most surprising part here is the gzipped-base64'd-compressed data almost entirely removes the base64 overhead.
- deleted 2y ago
  [deleted]
- zamadatix 2y ago
  It feels somewhat intuitive since (as the article notes) the Huffman encoding stage effectively "reverses" the original base64 overhead issue that an 8 bit (256 choices) index is used for 6 bits (64 choices) of actual characters. A useful compression algorithm which _didn't_ do this sort of thing would be very surprising as it would mean it doesn't notice simple patterns in the data but somehow compresses things anyways.
  - ranger_danger 2y ago
    how does it affect error correction though?
    - zamadatix 2y ago
      Neither base64 nor the gzipped version have error correction as implemented. The extra overhead bits in base64 come from selecting only a subset of printable characters, not by adding redundancy to the useful bits.
galaxyLogic 2y ago
Is there a tool or some other way to easily encode a JPG image so it can be embedded into HTML? I know there is something like that, but is it easy? Could it be made easier?
- throwanem 2y ago
  You can convert it to base64 and inline it anywhere an image URL is accepted, eg <img src="data:image/jpeg;base64,abc123..." /> (Double-check the exact syntax and the MIME type before you use it; it's been a few years since I have, and this example is from perhaps imperfect memory.)
  - galaxyLogic 2y ago
    Thanks
TacticalCoder 2y ago
I loved that (encoding stuff in webp) but my takeaway from the figures in the article is this: brotli is so good I'll host from somewhere where I can serve brotli (when and if the client supports brotli ofc).
niutech 2y ago
This page is broken at least on Sailfish OS browser, there is a long empty space after the paragraph: > Alright, so we’re dealing with 92 KiB for gzip vs 37 + 71 KiB for Brotli. Umm… That said, the overhead of gzip vs brotli HTML compression is nothing compared with amount of JS/images/video current websites use.
- mediumsmart 2y ago
  same on orion and safari and librewolf - is this a chrome page?
  - simmonmt 2y ago
    A different comment says librewolf disables webgl by default, breaking OP's decompression. Is that what you're seeing?
  - zamadatix 2y ago
    Works fine on Safari and Firefox for me.
- txtsd 2y ago
  Same on Mull
Dibby053 2y ago
I didn't know canvas anti-fingerprinting was so rudimentary. I don't think it increases uniqueness (the noise is different every run) but bypassing it seems trivial: run the thing n times and take the mode. With so little noise, 4 or 5 times should be more than enough.
- hedora 2y ago
  The article says it’s predictable within a given client, so your trick wouldn’t work. So, just use the anti-fingerprint noise as a cookie, I guess?
  - Dibby053 2y ago
    Huh, it seems it's just my browser that resets the noise every run. I opened the page in Firefox like the article suggests and I get a different pattern per site and session. That prevents using the noise as a supercookie, I think, if its pattern changes every time cookies are deleted.
    - sunaookami 2y ago
      It is different per site and per session, yes.
toddmorey 2y ago
"I hope you see where I’m going with this and are yelling 'Oh why the fuck' right now." I love reading blogpost like these.
efilife 2y ago
[flagged]
- bowmessage 2y ago
  [flagged]
  - efilife 2y ago
    So still a woman? And her pronouns are she/her, not they/them.
    - bowmessage 2y ago
      Did i say otherwise?
      - efilife 2y ago
        I don't see the point of pointing out the MTF stuff then
butz 2y ago
I wonder what is the difference in CPU usage on client side for WebP variant vs standard HTML? Are you causing more battery drain on visitor devices?
- lucb1e 2y ago
  It depends. Quite often (this is how you can tell I live in Germany) mobile data switches to "you're at the EDGE of the network range" mode¹ and transferring a few KB means keeping the screen on and radio active for a couple of minutes. If the page is now 45KB instead of 95KB, that's a significant reduction in battery drain! Under normal circumstances you're probably very right ¹ Now I wonder if the makers foresaw how their protocol name might sound to us now
bawolff 2y ago
They did all this and didn't even measure time to first paint? What is the point of doing this sort of thing if you dont even test how much faster or slower it made the page to load?
- tampontim 2y ago
  [flagged]
lxgr 2y ago
From the linked Github issue giving the rationale why Brotli is not available in the CompressionStream API: > As far as I know, browsers are only shipping the decompression dictionary. Brotli has a separate dictionary needed for compression, which would significantly increase the size of the browser. How can the decompression dictionary be smaller than the compression one? Does the latter contain something like a space-time tradeoff in the form of precalculated most efficient representations of given input substrings or something similar?
- zamadatix 2y ago
  Perhaps I'm reading past some of the surrounding context but that doesn't actually say the problem is about the relative sizes, just that the compression dictionary isn't already in browsers while the decompression dictionary already is. It's a bit disappointing you can't use Brotli in the DecompressionStream() interface just because it may or may not be available in the CompressionStream() interface though.
  - lxgr 2y ago
    I'm actually not convinced that there are two different dictionaries at all. The Brotli RFC only talks about a static dictionary, not a separate encoding vs. a decoding dictionary. My suspicion is that this is a confusion of the (runtime) sliding window, which limits maximum required memory on the decoder's side to 16 MB, with the actual shared static dictionary (which needs to be present in the decoder only, as far as I can tell; the encoder can use it, and if it does, it would be the same one the decoder has as well).
    - zamadatix 2y ago
      IIRC from working with Brotli before it's not that the it's truly a "different dictionary" but rather more like a "reverse view" of what is ultimately the same dictionary mappings. On one hand it seems a bit silly to worry about ~100 KB in browser for what will probably, on average, save more than that in upload/download the first time it is used. On the other hand "it's just a few hundred KB" each release for a few hundred releases ends up being a lot of cruft you can't remove without breaking old stuff. On the third hand coming out of our head... it's not like Chrome has been against shipping more for functionality for features they'd like to impose on users even if users don't actually want them anyways so what are small ones users can actually benefit from against that.
gildas 2y ago
In the same vein, you can package HTML pages as self-extracting ZIP files with SingleFile [1]. You can even include a PNG image to produce files compatible with HTML, ZIP and PNG [2], and for example display the PNG image in the HTML page [3]. [1] https://github.com/gildas-lormeau/SingleFile?tab=readme-ov-file#file-format-comparison https://github.com/gildas-lormeau/SingleFile?tab=readme-ov-f... [2] https://github.com/gildas-lormeau/Polyglot-HTML-ZIP-PNG https://github.com/gildas-lormeau/Polyglot-HTML-ZIP-PNG [3] https://github.com/gildas-lormeau/Polyglot-HTML-ZIP-PNG/raw/main/demo.png.zip.html https://github.com/gildas-lormeau/Polyglot-HTML-ZIP-PNG/raw/...
01095634045 2y ago
[flagged]
astrostl 2y ago
Things I seek in an image format: (1) compatibility (2) features WebP still seems far behind on (1) to me so I don't care about the rest. I hope it gets there, though, because folks like this seem pretty enthusiastic about (2).
- mistrial9 2y ago
  agree - webp has a lib on Linux but somehow, standard image viewers just do not read it, so -> "FAIL"
  - sgbeal 2y ago
    > webp has a lib on Linux but somehow, standard image viewers just do not read it That may apply to old "LTS" Linuxes, but not any relatively recent one. Xviewer and gimp immediately come to mind as supporting it and i haven't had a graphics viewer on Linux _not_ be able to view webp in at least 3 or 4 years.
- jfoster 2y ago
  The compatibility gap on WebP is already quite small. Every significant web browser now supports it. Almost all image tools & viewers do as well. Lossy WebP comes out a lot smaller than JPEG. It's definitely worth taking the saving.
somishere 2y ago
Lots of nice tricks in here, definitely fun! Only minor nitpick is that it departs fairly rapidly from the lede ... which espouses the dual virtues of an accessible and js-optional reading experience ;)
bogzz 2y ago
Still waiting on a webp encoder to be added to the Go stdlib...
kopirgan 2y ago
19 year old and look at the list of stuff she's done! Perhaps started coding in the womb?! Amazing.
Pesthuf 2y ago
It's impressive how close this is to Brotli even though brotli has this massive pre-shared dictionary. Is the actual compression algorithm used by it just worse, or does the dictionary just matter much less than I think?
- lifthrasiir 2y ago
  Pre-shared dictionary is most effective for the small size that can't reach the stationary distribution required for typical compressors. I don't know the exact threshold, but my best guess is around 1--10 KB.
ajsnigrutin 2y ago
So... 60kB less of transfer... and how much slower is it on an eg. galaxy s8 phone my mom has, because of all the shenanigans done to save those 60kB?
ranger_danger 2y ago
>ensure it works without JavaScript enabled >manually decompress it in JavaScript >Brotli decompressor in WASM the irony seems lost
- purplesyringa 2y ago
  A nojs version is compiled separately and is linked via a meta refresh. Not ideal, and I could add some safeguards for people without webgl, but I'm not an idiot.
divbzero 2y ago
> Typically, Brotli is better than gzip, and gzip is better than nothing. gzip is so cheap everyone enables it by default, but Brotli is way slower. Note that way slower applies to speed of compression, not decompression. So Brotli is a good bet if you can precompress. > Annoyingly, I host my blog on GitHub pages, which doesn’t support Brotli. If your users all use modern browsers and you host static pages through a service like Cloudflare or CloudFront that supports custom HTTP headers, you can implement your own Brotli support by precompressing the static files with Brotli and adding a Content-Encoding: br HTTP header. This is kind of cheating because you are ignoring proper content negotiation with Accept-Encoding, but I’ve done it successfully for sites with targeted user bases.
lifthrasiir 2y ago
It is actually possible to use Brotli directly in the web browser... with caveats of course. I believe my 2022 submission to JS1024 [1] is the first ever demonstration of this concept, and I also have a proof-of-concept code for the arbitrary compression (which sadly didn't work for the original size-coding purpose though). The main caveat is that you are effectively limited to the ASCII character, and that it is highly sensitive to the rendering stack for the obvious reason---it no longer seems to function in Firefox right now. [1] https://js1024.fun/demos/2022/18/readme https://js1024.fun/demos/2022/18/readme [2] https://gist.github.com/lifthrasiir/1c7f9c5a421ad39c1af19a9c4f060743 https://gist.github.com/lifthrasiir/1c7f9c5a421ad39c1af19a9c...
- purplesyringa 2y ago
  This technique is amazing and way cooler than my post. Kudos!
- zamadatix 2y ago
  The key note for understanding the approach (without diving into how to understand it's actually wrangled): > The only possibility is to use the WOFF2 font file format which Brotli was originally designed for, but you need to make a whole font file to leverage this. This got more complicated recently by the fact that modern browsers sanitize font files, typically by the OpenType Sanitizer (OTS), as it is very insecure to put untrusted font files directly to the system. Therefore we need to make an WOFF2 file that is sane enough to be accepted by OTS _and_ has a desired byte sequence inside which can be somehow extracted. After lots of failed experiments, I settled on the glyph widths ("advance") which get encoded in a sequence of two-byte signed integers with almost no other restrictions. Fantastic idea!
- lifthrasiir 2y ago
  Correction: It still works on Firefox, I just forgot that its zoom factor should be exactly 100% on Firefox to function. :-)
jfoster 2y ago
I work on Batch Compress (https://batchcompress.com/en https://batchcompress.com/en) and recently added WebP support, then made it the default soon after. As far as I know, it was already making the smallest JPEGs out of any of the web compression tools, but WebP was coming out only ~50% of the size of the JPEGs. It was an easy decision to make WebP the default not too long after adding support for it. Quite a lot of people use the site, so I was anticipating some complaints after making WebP the default, but it's been about a month and so far there has been only one complaint/enquiry about WebP. It seems that almost all tools & browsers now support WebP. I've only encountered one website recently where uploading a WebP image wasn't handled correctly and blocked the next step. Almost everything supports it well these days.
- pornel 2y ago
  Whenever WebP gives you file size savings bigger than 15%-20% compared to a JPEG, the savings are coming from quality degradation, not from improved compression. If you compress and optimize JPEG well, it shouldn't be far behind WebP. You can always reduce file size of a JPEG by making a WebP that looks almost the same, but you can also do that by recompressing a JPEG to a JPEG that looks almost the same. That's just a property of all lossy codecs, and the fact that file size grows exponentially with quality, so people are always surprised how even tiny almost invisible quality degradation can change the file sizes substantially.
- account42 2y ago
  > but WebP was coming out only ~50% of the size of the JPEGs Based on which quality comparison metric? WebP has a history of atrocious defaults that murder detail in dark areas.
  - jfoster 2y ago
    Nothing technically objective, just the size that a typical photo can be reduced to without looking bad. It really depends on what you're after, right? If preserving every detail matters to you, lossless is what you want. That's not going to create a good web experience for most users, though.
Jamie9912 2y ago
Why don't they make zstd images surely that would beat webp
- sgbeal 2y ago
  > Why don't they make zstd images surely that would beat webp zstd is a general-purpose compressor. By and large (and i'm unaware of any exceptions), specialized/format-specific compression (like png, wepb, etc.) will compress better than a general-purpose compressor because format-specific compressors can take advantage of quirks of the format which a general-purpose solution cannot. Also, format-specific ones are often lossy (or conditionally so), enabling them to trade lower fidelity for better compression, something a general-purpose compressor cannot do.
phh 2y ago
> A real-world web page compressed with WebP? Oh, how about the one you’re reading right now? Unless you use an old browser or have JavaScript turned off, WebP compresses this page starting from the “Fool me twice” section. If you haven’t noticed this, I’m happy the trick is working :-) Well it didn't work in Materialistic (I guess their webview disable js), and the failure mode is really not comfortable.
spintin 2y ago
[dead]
cobbal 2y ago
I would love to try reading the lossy version.
kardos 2y ago
On the fingerprinting noise: this sounds like a job for FEC [1]. It would increase the size but allow using the Canvas API. I don't know if this would solve the flicker though (not a front end expert here) Also, it's a long shot, but could the combo of FEC (+size) and lossy compression (-size) be a net win? [1] https://en.m.wikipedia.org/wiki/Error_correction_code https://en.m.wikipedia.org/wiki/Error_correction_code
michaelbrave 2y ago
I personally don't much care for the format, if I save an image and it ends up WebP then I have to convert it before I can edit or use it in any meaningful way since it's not supported in anything other than web browsers. It's just giving me extra steps to have to do.
- ezfe 2y ago
  Takes 2 seconds (literally, on macOS it's in the right click menu) to convert and it's smaller so not really a problem
  - account42 2y ago
    Conversion means additional quality loss unless you are converting to a lossy format. Better solution would be to use reasonable tools that are not stuck in the stone age and can open WebP directly.
- hot_gril 2y ago
  Ironically, even Google's products like Slides don't support webp images. But if/when it gets more support, I guess it's fine. I can tolerate a new format once every 20 years. .webm can go away, though.
  - account42 2y ago
    WebP is already deprecated from a compression ratio standpoint - it's entirely surpassed by all of AVIF, HEIF, and JXL. Too bad that none of these has broad support (and the best of them, JXL, is being blocked by all browser makers). WebM on the other hand still has a reason to exist unfortunately: patents on H.264.
    - qingcharles 2y ago
      JXL is supported in Safari, and Mozilla just put out a post saying they'll add support if someone will write a Rust decoder, and (bizarrely) Google has agreed to supply one. So there is a small chance Google will reverse the removal from Chrome.
rrrix1 2y ago
I very much enjoyed reading this. Quite clever! But... > Annoyingly, I host my blog on GitHub pages, which doesn’t support Brotli. Is the glaringly obvious solution to this not as obvious as I think it is? TFA went through a lot of round-about work to get (some) Brotli compression. Very impressive Yak Shave! If you're married to the idea of a Git-based automatically published web site, you could at least replicate your code and site to Gitlab Pages, which has supported precompressed Brotli since 2019. Or use one of Cloudflare's free tier services. There's a variety of ways to solve this problem before the first byte is sent to the client. Far too much of the world's source code already depends exclusively on Github. I find it distasteful to also have the small web do the same while blindly accepting an inferior experience and worse technology.
- purplesyringa 2y ago
  The solution's obvious, but if I followed it, I wouldn't have a fun topic to discuss or fake internet points to boast about, huh? :) I'll probably switch to Cloudflare Pages someday when I have time to do that.
DaleCurtis 2y ago
What a fun excursion :) You can also use the ImageDecoder API: https://developer.mozilla.org/en-US/docs/Web/API/ImageDecoder https://developer.mozilla.org/en-US/docs/Web/API/ImageDecode... and VideoFrame.copyTo: https://developer.mozilla.org/en-US/docs/Web/API/VideoFrame/copyTo https://developer.mozilla.org/en-US/docs/Web/API/VideoFrame/... to skip canvas entirely.
- purplesyringa 2y ago
  It's unfortunately Chromium-only for now, and I wanted to keep code simple. I've got a PoC lying around with VideoFrame and whatnot, but I thought this would be better for a post.

552 points by Kubuxu 2y ago

butz 2y ago

Dropping google fonts should improve page load time a bit too, considering those are loaded from remote server that requires additional handshake.

kevindamm 2y ago
..but if enough other sites are also using that font then it may already be available locally.
- pornel 2y ago
  This has stopped working many years ago. Every top-level domain now has its own private cache of all other domains. You likely have dozens of copies of Google Fonts, each in a separate silo, with absolutely zero reuse between websites. This is because a global cache use to work like a cookie, and has been used for tracking.
  - kevindamm 2y ago
    ah, I had forgotten about that, you're right. well at least you don't have to download it more than once for the site, but first impressions matter yeah
  - madeofpalk 2y ago
    Where "many years ago" is... 11 years ago for Safari! https://bugs.webkit.org/show_bug.cgi?id=110269 https://bugs.webkit.org/show_bug.cgi?id=110269
- SushiHippie 2y ago
  No, because of cache partitioning, which has been a thing for a while. https://developer.chrome.com/blog/http-cache-partitioning https://developer.chrome.com/blog/http-cache-partitioning

next_xibalba 2y ago

If only we hadn't lost Jan Sloot's Digital Coding System [1], we'd be able to transmit GB in milliseconds across the web! [1] https://en.wikipedia.org/wiki/Sloot_Digital_Coding_System https://en.wikipedia.org/wiki/Sloot_Digital_Coding_System

supriyo-biswas 2y ago
This claim itself is probably a hoax and not relevant to the article at hand; but these days with text-to-image models and browser support, you could probably do something like <img prompt="..."> and have the browser render something that matches the description, similar to the "cookbook" analogy used in the Wikipedia article.
- Lorin 2y ago
  That's an interesting concept, although it would generate a ton of bogomips since each client has to generate the image themselves instead of one time on the server. You'd also want "seed" and "engine" attributes to ensure all visitors see the same result.
  - LeoPanthera 2y ago
    You could at least push the work closer to the edge, by having genAI servers on each LAN, and in each ISP, similar to the idea of a caching web proxy before HTTPS rendered them impossible.
    - lucianbr 2y ago
      Push the work closer to the edge, and multiply it by quite a lot. Generate each image many times. Why would we want this? Seems like the opposite of caching in a sense.
      - sroussey 2y ago
        If you are reading a web page on mars and bandwidth is more precious than processing power, then <img prompt=“…”> might make sense. Not so much for us on earth however.

niceguy4 2y ago

Not to side track the conversation but to side track the conversation, has there been many other major WebP exploits like the serious one in the past?

deleted 2y ago
[deleted]

pjmlp 2y ago

[flagged]

solardev 2y ago
Actually, the article is about using that same image compression format to compress web pages (text and HTML) instead, which sometimes works even better than GZIP. Brotli still wins for most cases, though. It's a confusing title, though. I thought what you did at first, too.
- pjmlp 2y ago
  Indeed...
afavour 2y ago
Apparently we don’t read articles before commenting on them
- pjmlp 2y ago
  We do, doesn't change the fact of giving another meaning to WebP, or its hack.
  - afavour 2y ago
    Of course it does. The title is a play on using WebP to compress web page content, it’s not a real name.
  - Dylan16807 2y ago
    Well if that's your criticism then I don't know why you linked the wikipedia page for WebP. The article explains the image format, so that link comes across as a mistaken reaction to the title. Also I disagree that it really gives "another meaning".
croes 2y ago
The article is about hack to use the WebP from your link as a web page compression format.

BugsJustFindMe 2y ago

> the longest post on my site, takes 92 KiB instead of 37 KiB. This amounts to an unnecessary 2.5x increase in load time Sure, if you ignore latency. In reality it's an unnecessary 0.001% increase in load time because that size increase isn't enough to matter vs the round trip time. And the time you save transmitting 55 fewer KiB is probably less than the time lost to decompression. :p While fun, I would expect this specific scenario to actually be worse for the user experience not better. Speed will be a complete wash and compatibility will be worse.

jsnell 2y ago
That size difference is large enough to make a difference in the number of round trips required (should be roughly one fewer roundtrip with any sensible modern value for the initial congestion window). Won't be a 2.5x difference, but also not 0.001%.
- BugsJustFindMe 2y ago
  You don't need a new roundtrip for every packet. That would be devastating for throughput. One vs two vs three file packets get acked as a batch either way, not serially. Also when you get to the end, you then see > The actual savings here are moderate: the original is 88 KiB with gzip, and the WebP one is 83 KiB with gzip. In contrast, Brotli would provide 69 KiB. At 69 KiB you're still over the default TCP packet max, which means both cases transmit the same number of packets, one just has a bunch of extra overhead added for the extra JavaScript fetch, load, and execute. The time saved here is going to be negligible at best anyway, but there looks to be actually negative because we're burning time without reducing the number of needed packets at all.
  - jsnell 2y ago
    Those numbers are for a different page. For the original page, the article quotes 44 kB with this method vs. 92 kB for gzip. > At 69 KiB you're still over the default TCP packet max, which means both cases transmit the same number of packets, What? No, they absolutely don't transmit the same number of packets. Did you mean some other word?

98469056 2y ago

While peeking at the source, I noticed that the doctype declaration is missing a space. It currently reads <!doctypehtml>, but it should be <!doctype html>

BugsJustFindMe 2y ago
Maybe their javascript adds it back in :)
palsecam 2y ago
`<!doctype html>` can be minified into `<!doctypehtml>`. It’s, strictly speaking, invalid HTML, but it still successfully triggers standards mode. See https://GitHub.com/kangax/html-minifier/pull/970 https://GitHub.com/kangax/html-minifier/pull/970 / https://HTML.spec.WHATWG.org/multipage/parsing.html#parse-error-missing-whitespace-before-doctype-name https://HTML.spec.WHATWG.org/multipage/parsing.html#parse-er... (I too use that trick on https://FreeSolitaire.win https://FreeSolitaire.win)
- saagarjha 2y ago
  Why would you do this?
  - palsecam 2y ago
    To have FreeSolitaire.win homepage be only 20.7 kB over the wire. That’s for the whole game: graphics are inline SVGs, JS & CSS are embedded in <script> and <style> elements.
    - saagarjha 2y ago
      Right but surely skipping that space wasn't that much of a win
    - lifthrasiir 2y ago
      To be clear, you can minimize much further than what you already have. For example you haven't fully minified SVG, which can be also replaced with just symbol characters. Saving a single space from `<!doctype html>` is insignificant compared to that.

gkbrk 2y ago

> Why readPixels is not subject to anti-fingerprinting is beyond me. It does not sprinkle hardly visible typos all over the page, so that works for me. > keep the styling and the top of the page (about 8 KiB uncompressed) in the gzipped HTML and only compress the content below the viewport with WebP Ah, that explains why the article suddenly cut off after a random sentence, with an empty page that follows. I'm using LibreWolf which disables WebGL, and I use Chromium for random web games that need WebGL. The article worked just fine with WebGL enabled, neat technique to be honest.

niutech 2y ago
It isn't neat as long as it doesn't work with all modern web browsers (even with fingerprinting protection) and doesn't have a fallback for older browsers. WWW should be universally accessible and progressively enhanced, starting with plain HTML.
- afavour 2y ago
  It isn’t a serious proposal. It’s a creative hack that no one, author included is suggesting should be used in production.
  - account42 2y ago
    The author is using the hack in "production".
    - afavour 2y ago
      Debatable. I consider a personal blog to be fair game for experimentation, it’s not a paid for project and has no customer.
- kjhcvkek77 2y ago
  This philosophy hands your content on a silver platter to ai companies, so they can rake in money while giving nothing back to the author.

Retr0id 2y ago

I've used this trick before! Oddly enough I can't remember what I used it for (perhaps just to see if I could), and I commented on it here: https://gist.github.com/gasman/2560551?permalink_comment_id=4431196#gistcomment-4431196 https://gist.github.com/gasman/2560551?permalink_comment_id=... Edit: I found my prototype from way back, I guess I was just testing heh: https://retr0.id/stuff/bee_movie.webp.html https://retr0.id/stuff/bee_movie.webp.html

lucb1e 2y ago
That page breaks my mouse gestures add-on! (Or, I guess we don't have add-ons anymore but rather something like script injections that we call extensions, yay...) Interesting approach to first deliver 'garbage' and then append a bit of JS to transform it back into a page. The inner security nerd in me wonders if this might open up attacks if you would have some kind of user-supplied data, such as a comment form. One could probably find a sequence of bytes to comment that will, after compression, turn into a script tag, positioned (running) before yours would?
- Retr0id 2y ago
  Yeah that's plausible, you definitely don't want any kind of untrusted data in the input. Something I wanted to do but clearly never got around to, was figuring out how to put an open-comment sequence (<!--) in a header somewhere, so that most of the garbage gets commented out
lifthrasiir 2y ago
In my experience WebP didn't work well for the general case where this technique is actually useful (i.e. data less than 10 KB), because the most additions of WebP lossless to PNG are about modelling and not encoding, while this textual compression would only use the encoding part of WebP.

raggi 2y ago

Chromies got in the way of it for a very long time, but zstd is now coming to the web too, as it’s finally landed in chrome - now we’ve gotta get safari onboard

CharlesW 2y ago
Looks like it's on the To Do list, at least: https://webkit.org/standards-positions/#position-168 https://webkit.org/standards-positions/#position-168
mananaysiempre 2y ago
I’d love to go all-Zstandard, but in this particular case, as far as I know, Brotli and Zstandard are basically on par at identical values of decompressor memory consumption.
- simondotau 2y ago
  Realistically, everything should just support everything. There’s no reason why every (full featured) web server and every (full featured) web browser couldn’t support all compelling data compression algorithms. Unfortunately we live in a world where Google decides to rip JPEG-XL support out of Chrome for seemingly no reason other than spite. If the reason was a lack of maturity in the underlying library, fine, but that wasn’t the reason they offered.
  - madeofpalk 2y ago
    > There’s no reason Of course, there is - and it's really boring. Prioritisation, and maintenance. It's a big pain to add, say, 100 compressions formats and support them indefinitely, especially with little differentiation between them. Once we agree on what the upper bound of useless formats is, we can start to negotiate what the lower limit is.
    - raggi 2y ago
      It’s not prioritization of the code - it’s relatively little implementation and low maintenance but security is critical here and everyone is very afraid of compressors because gzip, jpeg and various others were pretty bad. Zstd, unlike lz4 before it (at least early on) has a good amount of tests and fuzzing. The implementation could probably be safer (with fairly substantial effort) but having test and fuzzing coverage is a huge step forward over prior industry norms

csjh 2y ago

I think the most surprising part here is the gzipped-base64'd-compressed data almost entirely removes the base64 overhead.

deleted 2y ago
[deleted]
zamadatix 2y ago
It feels somewhat intuitive since (as the article notes) the Huffman encoding stage effectively "reverses" the original base64 overhead issue that an 8 bit (256 choices) index is used for 6 bits (64 choices) of actual characters. A useful compression algorithm which _didn't_ do this sort of thing would be very surprising as it would mean it doesn't notice simple patterns in the data but somehow compresses things anyways.
- ranger_danger 2y ago
  how does it affect error correction though?
  - zamadatix 2y ago
    Neither base64 nor the gzipped version have error correction as implemented. The extra overhead bits in base64 come from selecting only a subset of printable characters, not by adding redundancy to the useful bits.

galaxyLogic 2y ago

Is there a tool or some other way to easily encode a JPG image so it can be embedded into HTML? I know there is something like that, but is it easy? Could it be made easier?

throwanem 2y ago
You can convert it to base64 and inline it anywhere an image URL is accepted, eg <img src="data:image/jpeg;base64,abc123..." /> (Double-check the exact syntax and the MIME type before you use it; it's been a few years since I have, and this example is from perhaps imperfect memory.)
- galaxyLogic 2y ago
  Thanks

TacticalCoder 2y ago

I loved that (encoding stuff in webp) but my takeaway from the figures in the article is this: brotli is so good I'll host from somewhere where I can serve brotli (when and if the client supports brotli ofc).

niutech 2y ago

This page is broken at least on Sailfish OS browser, there is a long empty space after the paragraph: > Alright, so we’re dealing with 92 KiB for gzip vs 37 + 71 KiB for Brotli. Umm… That said, the overhead of gzip vs brotli HTML compression is nothing compared with amount of JS/images/video current websites use.

mediumsmart 2y ago
same on orion and safari and librewolf - is this a chrome page?
- simmonmt 2y ago
  A different comment says librewolf disables webgl by default, breaking OP's decompression. Is that what you're seeing?
- zamadatix 2y ago
  Works fine on Safari and Firefox for me.
txtsd 2y ago
Same on Mull

Dibby053 2y ago

I didn't know canvas anti-fingerprinting was so rudimentary. I don't think it increases uniqueness (the noise is different every run) but bypassing it seems trivial: run the thing n times and take the mode. With so little noise, 4 or 5 times should be more than enough.

hedora 2y ago
The article says it’s predictable within a given client, so your trick wouldn’t work. So, just use the anti-fingerprint noise as a cookie, I guess?
- Dibby053 2y ago
  Huh, it seems it's just my browser that resets the noise every run. I opened the page in Firefox like the article suggests and I get a different pattern per site and session. That prevents using the noise as a supercookie, I think, if its pattern changes every time cookies are deleted.
  - sunaookami 2y ago
    It is different per site and per session, yes.

toddmorey 2y ago

"I hope you see where I’m going with this and are yelling 'Oh why the fuck' right now." I love reading blogpost like these.

efilife 2y ago

[flagged]

bowmessage 2y ago
[flagged]
- efilife 2y ago
  So still a woman? And her pronouns are she/her, not they/them.
  - bowmessage 2y ago
    Did i say otherwise?
    - efilife 2y ago
      I don't see the point of pointing out the MTF stuff then

butz 2y ago

I wonder what is the difference in CPU usage on client side for WebP variant vs standard HTML? Are you causing more battery drain on visitor devices?

lucb1e 2y ago
It depends. Quite often (this is how you can tell I live in Germany) mobile data switches to "you're at the EDGE of the network range" mode¹ and transferring a few KB means keeping the screen on and radio active for a couple of minutes. If the page is now 45KB instead of 95KB, that's a significant reduction in battery drain! Under normal circumstances you're probably very right ¹ Now I wonder if the makers foresaw how their protocol name might sound to us now

bawolff 2y ago

They did all this and didn't even measure time to first paint? What is the point of doing this sort of thing if you dont even test how much faster or slower it made the page to load?

tampontim 2y ago
[flagged]

lxgr 2y ago

From the linked Github issue giving the rationale why Brotli is not available in the CompressionStream API: > As far as I know, browsers are only shipping the decompression dictionary. Brotli has a separate dictionary needed for compression, which would significantly increase the size of the browser. How can the decompression dictionary be smaller than the compression one? Does the latter contain something like a space-time tradeoff in the form of precalculated most efficient representations of given input substrings or something similar?

zamadatix 2y ago
Perhaps I'm reading past some of the surrounding context but that doesn't actually say the problem is about the relative sizes, just that the compression dictionary isn't already in browsers while the decompression dictionary already is. It's a bit disappointing you can't use Brotli in the DecompressionStream() interface just because it may or may not be available in the CompressionStream() interface though.
- lxgr 2y ago
  I'm actually not convinced that there are two different dictionaries at all. The Brotli RFC only talks about a static dictionary, not a separate encoding vs. a decoding dictionary. My suspicion is that this is a confusion of the (runtime) sliding window, which limits maximum required memory on the decoder's side to 16 MB, with the actual shared static dictionary (which needs to be present in the decoder only, as far as I can tell; the encoder can use it, and if it does, it would be the same one the decoder has as well).
  - zamadatix 2y ago
    IIRC from working with Brotli before it's not that the it's truly a "different dictionary" but rather more like a "reverse view" of what is ultimately the same dictionary mappings. On one hand it seems a bit silly to worry about ~100 KB in browser for what will probably, on average, save more than that in upload/download the first time it is used. On the other hand "it's just a few hundred KB" each release for a few hundred releases ends up being a lot of cruft you can't remove without breaking old stuff. On the third hand coming out of our head... it's not like Chrome has been against shipping more for functionality for features they'd like to impose on users even if users don't actually want them anyways so what are small ones users can actually benefit from against that.

gildas 2y ago

In the same vein, you can package HTML pages as self-extracting ZIP files with SingleFile [1]. You can even include a PNG image to produce files compatible with HTML, ZIP and PNG [2], and for example display the PNG image in the HTML page [3]. [1] https://github.com/gildas-lormeau/SingleFile?tab=readme-ov-file#file-format-comparison https://github.com/gildas-lormeau/SingleFile?tab=readme-ov-f... [2] https://github.com/gildas-lormeau/Polyglot-HTML-ZIP-PNG https://github.com/gildas-lormeau/Polyglot-HTML-ZIP-PNG [3] https://github.com/gildas-lormeau/Polyglot-HTML-ZIP-PNG/raw/main/demo.png.zip.html https://github.com/gildas-lormeau/Polyglot-HTML-ZIP-PNG/raw/...

01095634045 2y ago

[flagged]

astrostl 2y ago

Things I seek in an image format: (1) compatibility (2) features WebP still seems far behind on (1) to me so I don't care about the rest. I hope it gets there, though, because folks like this seem pretty enthusiastic about (2).

mistrial9 2y ago
agree - webp has a lib on Linux but somehow, standard image viewers just do not read it, so -> "FAIL"
- sgbeal 2y ago
  > webp has a lib on Linux but somehow, standard image viewers just do not read it That may apply to old "LTS" Linuxes, but not any relatively recent one. Xviewer and gimp immediately come to mind as supporting it and i haven't had a graphics viewer on Linux _not_ be able to view webp in at least 3 or 4 years.
jfoster 2y ago
The compatibility gap on WebP is already quite small. Every significant web browser now supports it. Almost all image tools & viewers do as well. Lossy WebP comes out a lot smaller than JPEG. It's definitely worth taking the saving.

somishere 2y ago

Lots of nice tricks in here, definitely fun! Only minor nitpick is that it departs fairly rapidly from the lede ... which espouses the dual virtues of an accessible and js-optional reading experience ;)

bogzz 2y ago

Still waiting on a webp encoder to be added to the Go stdlib...

kopirgan 2y ago

19 year old and look at the list of stuff she's done! Perhaps started coding in the womb?! Amazing.

Pesthuf 2y ago

It's impressive how close this is to Brotli even though brotli has this massive pre-shared dictionary. Is the actual compression algorithm used by it just worse, or does the dictionary just matter much less than I think?

lifthrasiir 2y ago
Pre-shared dictionary is most effective for the small size that can't reach the stationary distribution required for typical compressors. I don't know the exact threshold, but my best guess is around 1--10 KB.

ajsnigrutin 2y ago

So... 60kB less of transfer... and how much slower is it on an eg. galaxy s8 phone my mom has, because of all the shenanigans done to save those 60kB?

ranger_danger 2y ago

>ensure it works without JavaScript enabled >manually decompress it in JavaScript >Brotli decompressor in WASM the irony seems lost

purplesyringa 2y ago
A nojs version is compiled separately and is linked via a meta refresh. Not ideal, and I could add some safeguards for people without webgl, but I'm not an idiot.

divbzero 2y ago

> Typically, Brotli is better than gzip, and gzip is better than nothing. gzip is so cheap everyone enables it by default, but Brotli is way slower. Note that way slower applies to speed of compression, not decompression. So Brotli is a good bet if you can precompress. > Annoyingly, I host my blog on GitHub pages, which doesn’t support Brotli. If your users all use modern browsers and you host static pages through a service like Cloudflare or CloudFront that supports custom HTTP headers, you can implement your own Brotli support by precompressing the static files with Brotli and adding a Content-Encoding: br HTTP header. This is kind of cheating because you are ignoring proper content negotiation with Accept-Encoding, but I’ve done it successfully for sites with targeted user bases.

lifthrasiir 2y ago

It is actually possible to use Brotli directly in the web browser... with caveats of course. I believe my 2022 submission to JS1024 [1] is the first ever demonstration of this concept, and I also have a proof-of-concept code for the arbitrary compression (which sadly didn't work for the original size-coding purpose though). The main caveat is that you are effectively limited to the ASCII character, and that it is highly sensitive to the rendering stack for the obvious reason---it no longer seems to function in Firefox right now. [1] https://js1024.fun/demos/2022/18/readme https://js1024.fun/demos/2022/18/readme [2] https://gist.github.com/lifthrasiir/1c7f9c5a421ad39c1af19a9c4f060743 https://gist.github.com/lifthrasiir/1c7f9c5a421ad39c1af19a9c...

purplesyringa 2y ago
This technique is amazing and way cooler than my post. Kudos!
zamadatix 2y ago
The key note for understanding the approach (without diving into how to understand it's actually wrangled): > The only possibility is to use the WOFF2 font file format which Brotli was originally designed for, but you need to make a whole font file to leverage this. This got more complicated recently by the fact that modern browsers sanitize font files, typically by the OpenType Sanitizer (OTS), as it is very insecure to put untrusted font files directly to the system. Therefore we need to make an WOFF2 file that is sane enough to be accepted by OTS _and_ has a desired byte sequence inside which can be somehow extracted. After lots of failed experiments, I settled on the glyph widths ("advance") which get encoded in a sequence of two-byte signed integers with almost no other restrictions. Fantastic idea!
lifthrasiir 2y ago
Correction: It still works on Firefox, I just forgot that its zoom factor should be exactly 100% on Firefox to function. :-)

jfoster 2y ago

I work on Batch Compress (https://batchcompress.com/en https://batchcompress.com/en) and recently added WebP support, then made it the default soon after. As far as I know, it was already making the smallest JPEGs out of any of the web compression tools, but WebP was coming out only ~50% of the size of the JPEGs. It was an easy decision to make WebP the default not too long after adding support for it. Quite a lot of people use the site, so I was anticipating some complaints after making WebP the default, but it's been about a month and so far there has been only one complaint/enquiry about WebP. It seems that almost all tools & browsers now support WebP. I've only encountered one website recently where uploading a WebP image wasn't handled correctly and blocked the next step. Almost everything supports it well these days.

pornel 2y ago
Whenever WebP gives you file size savings bigger than 15%-20% compared to a JPEG, the savings are coming from quality degradation, not from improved compression. If you compress and optimize JPEG well, it shouldn't be far behind WebP. You can always reduce file size of a JPEG by making a WebP that looks almost the same, but you can also do that by recompressing a JPEG to a JPEG that looks almost the same. That's just a property of all lossy codecs, and the fact that file size grows exponentially with quality, so people are always surprised how even tiny almost invisible quality degradation can change the file sizes substantially.
account42 2y ago
> but WebP was coming out only ~50% of the size of the JPEGs Based on which quality comparison metric? WebP has a history of atrocious defaults that murder detail in dark areas.
- jfoster 2y ago
  Nothing technically objective, just the size that a typical photo can be reduced to without looking bad. It really depends on what you're after, right? If preserving every detail matters to you, lossless is what you want. That's not going to create a good web experience for most users, though.

Jamie9912 2y ago

Why don't they make zstd images surely that would beat webp

sgbeal 2y ago
> Why don't they make zstd images surely that would beat webp zstd is a general-purpose compressor. By and large (and i'm unaware of any exceptions), specialized/format-specific compression (like png, wepb, etc.) will compress better than a general-purpose compressor because format-specific compressors can take advantage of quirks of the format which a general-purpose solution cannot. Also, format-specific ones are often lossy (or conditionally so), enabling them to trade lower fidelity for better compression, something a general-purpose compressor cannot do.

phh 2y ago

> A real-world web page compressed with WebP? Oh, how about the one you’re reading right now? Unless you use an old browser or have JavaScript turned off, WebP compresses this page starting from the “Fool me twice” section. If you haven’t noticed this, I’m happy the trick is working :-) Well it didn't work in Materialistic (I guess their webview disable js), and the failure mode is really not comfortable.

spintin 2y ago

[dead]

cobbal 2y ago

I would love to try reading the lossy version.

kardos 2y ago

On the fingerprinting noise: this sounds like a job for FEC [1]. It would increase the size but allow using the Canvas API. I don't know if this would solve the flicker though (not a front end expert here) Also, it's a long shot, but could the combo of FEC (+size) and lossy compression (-size) be a net win? [1] https://en.m.wikipedia.org/wiki/Error_correction_code https://en.m.wikipedia.org/wiki/Error_correction_code

michaelbrave 2y ago

I personally don't much care for the format, if I save an image and it ends up WebP then I have to convert it before I can edit or use it in any meaningful way since it's not supported in anything other than web browsers. It's just giving me extra steps to have to do.

ezfe 2y ago
Takes 2 seconds (literally, on macOS it's in the right click menu) to convert and it's smaller so not really a problem
- account42 2y ago
  Conversion means additional quality loss unless you are converting to a lossy format. Better solution would be to use reasonable tools that are not stuck in the stone age and can open WebP directly.
hot_gril 2y ago
Ironically, even Google's products like Slides don't support webp images. But if/when it gets more support, I guess it's fine. I can tolerate a new format once every 20 years. .webm can go away, though.
- account42 2y ago
  WebP is already deprecated from a compression ratio standpoint - it's entirely surpassed by all of AVIF, HEIF, and JXL. Too bad that none of these has broad support (and the best of them, JXL, is being blocked by all browser makers). WebM on the other hand still has a reason to exist unfortunately: patents on H.264.
  - qingcharles 2y ago
    JXL is supported in Safari, and Mozilla just put out a post saying they'll add support if someone will write a Rust decoder, and (bizarrely) Google has agreed to supply one. So there is a small chance Google will reverse the removal from Chrome.

rrrix1 2y ago

I very much enjoyed reading this. Quite clever! But... > Annoyingly, I host my blog on GitHub pages, which doesn’t support Brotli. Is the glaringly obvious solution to this not as obvious as I think it is? TFA went through a lot of round-about work to get (some) Brotli compression. Very impressive Yak Shave! If you're married to the idea of a Git-based automatically published web site, you could at least replicate your code and site to Gitlab Pages, which has supported precompressed Brotli since 2019. Or use one of Cloudflare's free tier services. There's a variety of ways to solve this problem before the first byte is sent to the client. Far too much of the world's source code already depends exclusively on Github. I find it distasteful to also have the small web do the same while blindly accepting an inferior experience and worse technology.

purplesyringa 2y ago
The solution's obvious, but if I followed it, I wouldn't have a fun topic to discuss or fake internet points to boast about, huh? :) I'll probably switch to Cloudflare Pages someday when I have time to do that.

DaleCurtis 2y ago

What a fun excursion :) You can also use the ImageDecoder API: https://developer.mozilla.org/en-US/docs/Web/API/ImageDecoder https://developer.mozilla.org/en-US/docs/Web/API/ImageDecode... and VideoFrame.copyTo: https://developer.mozilla.org/en-US/docs/Web/API/VideoFrame/copyTo https://developer.mozilla.org/en-US/docs/Web/API/VideoFrame/... to skip canvas entirely.

purplesyringa 2y ago
It's unfortunately Chromium-only for now, and I wanted to keep code simple. I've got a PoC lying around with VideoFrame and whatnot, but I thought this would be better for a post.