A gross underestimation, IMO. When I was in big data, fewer than 5% of data written was ever touched again, and only a single digit number of our large customer

A gross underestimation, IMO. When I was in big data, fewer than 5% of data written was ever touched again, and only a single digit number of our large customers (out of tens of thousands) actually made real use of their "big data", and created most of the load. That's the trouble with "checkbox driven development" - 10 years ago you were required to have a "big data strategy" for anyone to take you seriously, even if your strategy boiled down to just ETL-ing a bunch of crap you're never going to need into the cloud and never touching it again. Now I'm in AI, and the same thing is happening to AI. It's great if you're selling shovels, so to speak, but not so great if you plan on selling them for an extended period of time. This, by the way, has implications on storage systems design. You want something that's cheap yet dense to encode, potentially at the slight expense of decode speed. Normally people really lose sleep about decode speed first and foremost, which, while important, does not minimize the overall resource bill.