3 ms·
I’m reminded of Erik Naggum’s epic rant about XML (I’ve misplaced the link): as the cost of compute and storage and network move relative to one another decade
by benreesman 4y ago
I’m reminded of Erik Naggum’s epic rant about XML (I’ve misplaced the link): as the cost of compute and storage and network move relative to one another decade over decade, some things go from bad to good idea or vice versa.
These days, your storage-adjacent nodes can burn compute on the fly, you have to try hard to pay for rack-adjacent 10 gig, Snappy and zstd are so fast that SREs at Google and FB turn them on or off under service owners to shave the margins.
You can afford getting out of and back into a custom format for your domain, multiple times.
It’s going to be mmap’d ndarrays before it hits the compute stuff, and maybe libraries could be stronger there, but you can always throw away column names and putting them back is hard.