6 ms·
Minify and Gzip (2022)
- matthewfelgate 3y agoGood point. Has anyone got a simple explanation on how zip programs work, and what types of text they optimised best, and how to optimise for this?
- tyingq 3y agoThis is a reasonable explanation of DEFLATE, which the underlying thing being talked about in the article: https://www.zlib.net/feldspar.html https://www.zlib.net/feldspar.html
- julian37 3y agoClosure Compiler follows the same line of thinking: https://github.com/google/closure-compiler/wiki/FAQ#closure-compiler-inlined-all-my-strings-which-made-my-code-size-bigger-why-did-it-do-that https://github.com/google/closure-compiler/wiki/FAQ#closure-...
- Traubenfuchs 3y agoThere is also an online tool / API for this, which can be quite helpful for non npm using vanilla.js workflows. https://closure-compiler.appspot.com/ https://closure-compiler.appspot.com/ Should be set to: // @language_in UNSTABLE // @language_out ECMASCRIPT_NEXT for it to work as expected.
- palsecam 3y agoYou’re right, however for offline but non npm workflow, you can also use the Closure Compiler JAR, grabbed for instance from Maven: https://mvnrepository.com/artifact/com.google.javascript/closure-compiler https://mvnrepository.com/artifact/com.google.javascript/clo... $ wget https://repo1.maven.org/maven2/com/google/javascript/closure-compiler/v20230502/closure-compiler-v20230502.jar […] $ java -jar closure-compiler-v20230502.jar The compiler is waiting for input via stdin. The Closure Compiler still is a Java project. It just gets primarily distributed via NPM these days.
- fsdsadlfkj 3y agoMy naive guess to what's going on is gzip here is set on tame settings, such that when it sees the same codeblock twice it shrinks it in a more space-efficient less cpu-efficient way. When it sees it just once, it doesn't. However it happens, I think if you're being real, it would take A LOT of these micro-optimizations multiplied by A LOT of cache misses to add up to anything important. What I got out of this is, unless you're looking at the specific bytes in the network tab every time you edit your code, just minify/gzip and forget about it.
- r1ch 3y agoI wonder if we're at a point yet where gzip's slowness is becoming a bottleneck. I wish zstd support in the browser was prioritized a bit higher, brotli does have similar/ better compression due to the dictionary but speed is also important.
- TillE 3y agoTo really exploit zstd you'd want all major browser vendors to agree on some standard, updateable dictionaries for HTML/CSS/etc.
- theandrewbailey 3y agoZstandard Compression and the application/zstd Media Type https://datatracker.ietf.org/doc/html/rfc8478 https://datatracker.ietf.org/doc/html/rfc8478
- arthur2e5 3y agoSlowness on compression or decompression? The compression end, which is slower, can be plausibility mitigated with analogues of gzip_static. And as the brotli people will insist on telling you, it's good even without that dictionary: https://news.ycombinator.com/item?id=27163981 https://news.ycombinator.com/item?id=27163981
- arp242 3y agoA few years ago I did some measurements of popular projects (Wikipedia, WordPress, default React, jQuery, few things like that) comparing minify, minify+gzip, and just gzip, and found found that most minification beyond "removing comments" gives very little benefit even for poor 2G connections. Especially things like removing as much whitespace as possible or replacing true/false with !0/!1 just don't really save a lot of space. The only two I found to make a significant difference were removing comments, and shortening identifiers (and usually with most of the savings being in "removing comments", depending on the amount of comments and code style).
- jackmott 3y agohow does minify affect the times to decompress and load the css I wonder?
- catlifeonmars 3y agoMy money is on much less than the time it would take to send the extra bytes over the wire. But I suppose measurement is key :D
- metalrain 3y agoUnless client combines decompress and CSS parsing, they would at least allocate memory for the whitespace before parsing and then skipping them. It might be better to have slightly bigger payload on the wire to save time after decompress.
- Gordonjcp 3y agoA lot of it reminds me of the olden days of Sinclair ZX81s with 1kB of memory, of which roughly half was free once you drew on the screen and defined a few variables. We used to write things like "LET A=COS PI" instead of "LET A=1", or "SIN PI" instead of 0, because keywords were represented as a single byte and numeric constants as five bytes so two keywords represented a useful saving of three bytes. Back then it made a huge difference but now that modern computers have hundreds or even thousands of kilobytes of RAM free it's a bit pointless.
- didntcheck 3y agoIt would be interesting to benchmark the CPU performance of a full page load and paint for each version. At the end of the day the language parser will see the decompressed text, so the minified version should reduce the time spent in that stage. However AFAIK a basic minify, that only compresses whitespace and identifiers, would only speed up the lexer, which I expect is the fastest part anyway. But reducing the number of actual CSS rules (as the author experiments with manually) may make a measurable difference fwiw I don't bother minifying anything on my personal site. I realise that complex sites can have large enough payloads to make it worth it, especially for Javascript, but for the typical developer blog it's likely overengineering in the extreme. If I were to apply any postprocessor it would probably be a prettifier, for the HTML, since the template substitution results in mismatched indents. But then again anyone looking at the DOM is likely to be doing it in the dev tools, which already renders a projectional view, rather then the original source
- tiffanyh 3y agoCloudflare use to only perform compression level 4 (level 9 is most compressed), because they found while the file they served was Bigger - the amount to time to compress/decompress was longer for Level 9 … resulting in perceived slower loading/speeds of end user. https://blog.cloudflare.com/this-is-brotli-from-origin/#:~:text=Since%202017%2C%20Cloudflare%20has%20been,if%20the%20origin%20supported%20Brotli https://blog.cloudflare.com/this-is-brotli-from-origin/#:~:t....
- stabbles 3y agoYou're linking to an article about brotli. I don't think gzip is affected by it. If anything, its decompression should be faster given fewer bytes to process. The usual reason to stick to gzip level 4 is that compression gets extremely slow at marginal savings. I would expect that cloudflare caches compressed data so they pay the compression cost only once.
- kevingadd 3y agoAt cloudflare's scale I can imagine that for many websites, compressed data falls out of cache frequently and then the file has to get compressed again on next request. For example if you're hosting a website that contains a ton of different news articles or user-submitted stories that's probably tens of thousands of separate blobs of text, and each one might only get read a few dozen times a day. The odds are much lower that the individual content is going to be in cache on the particular edge node that handles the request. The CDN is still useful in this case of course, both for ddos mitigation and because it can cache the common stuff like your website's stylesheets.
- grose 3y agoFor my "fun" projects I never minify the CSS or JS because I miss the days when you could View Source on a website and easily dig into how it works. If you're a NoScript user, go ahead and read the source and decide for yourself if you want to allow it. Much easier to maintain without all the moving parts of the JS build ecosystem as well.
- mbork_pl 3y agoI'll be starting a completely serious side project in a few days where I'm going to not minify anything (especially JS) so that user will be able to hit C-u and check for themselves nothing nefarious is going on privacy-wise. Also, no cookies at all, and the whole site will be just a bunch of static html files, two fonts, one css file, one small vanilla JS file and some images where appropriate. I hope to get load times well below one second (especially once the assets are cached).
- catlifeonmars 3y agoFWIW if you ship source maps, then you can have the best of both worlds.
- kevingadd 3y agoDo those work in View Source? Don't you have to load the site and inspect it in a debugger for source maps to kick in, which doesn't happen in the noscript scenario?
- zamadatix 3y agoI don't think view source will load source maps but you can still open the f12 debugger and load source maps (even automatically) with noscript enabled. The debugger isn't treated as a normal webpage, rather part of the browser itself.
- WorldMaker 3y agoIt's mostly just kind of funny that "View Source" diverged so much from the F12 debugger. For most of its history Spartan Edge had caught and fixed that mistake and View Source just dropped you into the F12 debugger. It's almost sad that Chromium Edge again shows you a different unrelated view. Firefox has also seemed to wander back and forth between View Source just reuses the full Dev Tools and View Source is a disconnected view from Dev Tools for whatever reason. At the moment the big difference flag seems to be if you install/use Firefox Developer or not. (Allegedly, one of the reasons is that some non-technical users have strange reasons that they View Source from time to time and they get upset if it opens Dev Tools because Dev Tools is just too confusing. Mac Safari has that interesting dance to ask it to opt in to showing Dev Tools. I seem to recall IE11 did something similar that if you ever opened Dev Tools for more than a few minutes it then took over for View Source, but otherwise used the older and simpler View Source from ancient IE history. I'm surprised that if the reason is "it confuses non-technical reasons" there isn't an opt-in flag in Chrome or Chromium Edge. The "Firefox Developer Edition" is a separate SKU is also an interesting way to approach that.)
- idoubtit 3y agoIf you control the deploying process and the web server (at least with Nginx), you can disable gzip for the small files (like the article has), and avoid on-the-fly compressing for the larger files. location /js/small { gzip off; } location /js/ { gzip on; # Use the .gz file if it exists gzip_static on; }
- palsecam 3y agoNB: there is also nginx `gzip_min_length` directive to avoid compressing small files https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_min_length https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzi...
- Theodores 3y agoThere are better ways to use media queries. In your CSS use a variable for the property that changes, and put the fallback in it too. In that way the next person to work on the code can see what changes. In your media queries only have CSS variables, as locally scoped as possible, and name them after the property with -- at the front. On this way it is self evident what the media queries are about. For example: p { color: var(--color, black); } @query { p { color: chocolate; } } And have the queries after the CSS for legibility. By writing CSS this way, your source will be compact and easy to maintain. Because of this you won't need to mangle it with CSS mangling tools or bother with CSS minifiers. Your production time will speed up immensely. Also, style the elements, then add classes and IDs if you have to. Stay away from div's, use all the semantic elements with CSS grid and you will get much better results.
- deleted 3y ago[deleted]
- runiq 3y agoAh, the end-to-end principle in action. :) https://en.wikipedia.org/wiki/End-to-end_principle https://en.wikipedia.org/wiki/End-to-end_principle
- franciscop 3y agoI made a tiny jQuery library replacement a while back and I found similar things to the author; you should not be measuring the raw text representation, but the final gzip size, every time you make an optimization. I go through a list of examples that I found where it does or it does not matter whether you "optimize" it for size or not. I also theorize (I do not have enough knowledge to verify my claims though) why this is happening: https://medium.com/@fpresencia/understanding-gzip-size-836c74b66c0b https://medium.com/@fpresencia/understanding-gzip-size-836c7...
- rzwitserloot 3y agoI (thought) I saw where the article was going before it got there, but, the notion that the 'optimization' actually __increases__ gzipped sizes surprised me (I thought the conclusion was going to drive at 'it makes no difference', or 'only a byte or two saved'). I don't have a site that sees a ton of traffic, and I know what some of my direct competitors spend on IAAS (and how long their pages take to load). I'm orders of magnitude below it - hence, I do not spend much time investigating optimizations. I can name a list as long as my leg as to where I bet I can win quite a bit on optimizing for IAAS cost or UX speed. Business-wise I decided it's not (currently!) worth investigating. One of the more intriguing ones on my list is that I __do not minify anything__. My reasoning is that post-gzip, who cares. It's such a tiny difference. Yes, turning e.g. class names or whatnot in the CSS, or local identifiers in javascript, into short letter sequences means even post-gzip it'll be smaller, but by so little. The reduction in file size when e.g. eliminating all unnecessary whitespace is really, really tiny. For any site that isn't trying to optimize for 'millions of hits a day!' levels of traffic, why _do_ you minimize all that stuff? It makes the developer experience considerably shittier and the gain seems a few orders of magnitude too small to even bother with it.
- williamdclt 3y agoYou make decent points, but for this one: > It makes the developer experience considerably shittier and the gain seems a few orders of magnitude too small to even bother with it. Does it? It's very rare that I interact with the minimised code
- ravenstine 3y agoWouldn't that have to be due to your setup not minifying code during development? That's typical to a lot of web frameworks, if not nearly all of them. I think the OP may be referring to situations where production needs to be debugged, in which case navigating production code usually becomes a massive pain in the rear.
- zamadatix 3y agoSource maps generally solve most of the pain automatically. They aren't perfect, but production bugs don't mean you need to look at minified code often. That said, if one can get away with not needing to do more work and still have a performing site then how little the difference is remains irrelevant.
- GuB-42 3y agoCompression aware minifiers are definitely a thing. Instead of trying to make the code as small as possible, they will try to make it as redundant as possible. For example by reusing names. For example "message="hello";print(message)" can be minified into "hello="hello";print(hello)", which may compress better than "a="hello";print(a)".
- jerrre 3y agoI like the idea, but just tested your specific example with https://facia.dev/tools/compress-decompress/gzip-compress/ https://facia.dev/tools/compress-decompress/gzip-compress/ and the second version compresses better, as my intuition already thought, without knowing how gzip works, because you still need a kind of identifier for repeated strings, and how much shorter than 1 byte can they be.
- tomjakubowski 3y agoOP's suggestion may work better on real life examples of source code (which would be longer and include a lot more different kinds of repeated strings)
- aidenn0 3y agoI ran some tests, and with zstd, brotli, and gzip, single-byte identifiers win on even slightly more complicated examples. If you have more than N identifiers (where N is the valid number of single-byte identifiers) then using short strings from the brotli dictionary seems to be at worst break-even for brotli. Also, there seems to be a 40-byte minimum size on zstd; Making the original example significantly smaller (or larger with repeated patterns) all yielded 40 byte files. I had to change it to e.g. print(hello);foo(hello); bar(hello) to get any difference between using "x" and "hello" as an identifier.
- Alifatisk 3y agoIs anyone compressing brotli these days? I switched from gzip to brotli quite easy in my Rails app.
- mhitza 3y agoI use it in a symfony project, and I compress assets beforehand in my deployment pipeline into static br suffixed assets. Then Apache is configured to serve these files if present instead of compressing on the fly.
- seenthat 3y agoThis minify/gzip size effect is a well known quirk to developers of javascript minifiers. The minifier's symbol mangling algorithm often has a more pronounced effect than does advanced AST optimization. This website has real life data on the matter for popular libraries: * https://github.com/privatenumber/minification-benchmarks https://github.com/privatenumber/minification-benchmarks Compare the trophies indicating smallest size for Minified versus Minzipped (gzip). Generally the smallest minified size yields the smallest minified+gzip size, but there are some notable anomolies outside the range of statistical noise. It is not practical for a javascript minifier to take a compression algorithm into account - it would blow up the minify timings exponentially.
- breakingrules 3y ago[dead]
- zelphirkalt 3y agoNow consider all the time, developer nerves, and compute (installation of dependencies and processing files), missed learning opportunity (people wanting to know how things work and looking at the source of a website), that have been wasted in setting up "minifiers" unnecessarily in frontend projects, "because it will be smaller" or "because I've someone else recommending it".
- AshleysBrain 3y agoI remember finding a similar thing while working on minifying JavaScript. If you have code like func("hello world"); func("hello world"); Then naively it seems that can be optimized to this: let s = "hello world"; func(s); func(s); However once compressed, the second result is usually larger! It's basically because the second example adds the `let s =` part which is new unique content it has to store. So if you want to minify JavaScript to a shorter version uncompressed, it's good to transform the first to the second. However if you want to minify JavaScript to the smallest compressed size, it's actually better to do the reverse transform and turn the second case in to the first! But then you get in to tradeoffs with parse time with a longer input, especially with very long strings - so I'm not sure any of today's minifiers actually do that. (Edit - turns out Closure Compiler does: https://github.com/google/closure-compiler/wiki/FAQ#closure-compiler-inlined-all-my-strings-which-made-my-code-size-bigger-why-did-it-do-that https://github.com/google/closure-compiler/wiki/FAQ#closure-...)
- astrobe_ 3y agoProgramming and compression are related, see Kolmogorov complexity [1]. It's then no surprise that when you do the "compression" yourself, it is almost like your are compressing twice, which often results in inflation because of the compression dictionary. [1] https://en.wikipedia.org/wiki/Kolmogorov_complexity https://en.wikipedia.org/wiki/Kolmogorov_complexity
- deleted 3y ago[deleted]
- csswizardry 3y agoI’m very late to the party, but I wrote about this back in early 2016! https://csswizardry.com/2016/02/mixins-better-for-performance/ https://csswizardry.com/2016/02/mixins-better-for-performanc...