3 ms·
> I want to know why a company as big as Amazon was still using gzip in 2022.... The short answer to "Why still using gzip in 2022" is almost always answered b
by Twirrim 4y ago
> I want to know why a company as big as Amazon was still using gzip in 2022....
The short answer to "Why still using gzip in 2022" is almost always answered by "Because it made no business or financial sense to spend head count on it".
Amazon is usually fairly smart about the ways it spends head count, particularly on things that could notably reduce costs. Managers/directors/VPs/SVPs obsess over what the value is from various work, and routinely adjust priorities and reallocate head count. They track not just what is happening within individual services, but also the overall business strategies, what's on the horizon etc.
One simple example that most folks outside the industry won't be seeing is that every major cloud has been working their collective arses off for well over a year on meeting JWCC contract needs. There's a lot of work involved in that contract, because unsurprisingly it's really hard to build and run an entirely air-gapped cloud region, but the payoff on being selected as a vendor is phenomenal. Far more than saving an additional 30% of storage, even at AWS S3 scale. _It's not the only major business and engineering initiative_ that will be taking place across AWS.
I've got a long list of "We should do x, y, z" that applies to my service in the cloud I work for, a number of which will see notable performance improvements. They make zero sense allocating head count to, though, because I've got at least a dozen other more important things going on from a business roadmap perspective to get solved that will make us way more money and/or save way more engineering resources down the line, be it automation stuff, or new features.
What does happen, though, is that list is kept in mind any time new business priorities come about. If there is any way bits on the list can be tied in to something the business wants, it'll get hooked in to it. The business gets what it wants, the service gets what I want, everyone is happy.
(To repeat something others have pointed out, there's also zero indication of timeline, it could have happened any time in the last several years that zstd has been a thing, they may well have switched to it almost as soon as Facebook removed that ridiculous "You can't sue us" clause)