4 ms·
If the big tech firms vanished, and instead there would only be many tech dwarfs, we still would have big data centers that needed the same capacity.
by stabbles 6y ago
If the big tech firms vanished, and instead there would only be many tech dwarfs, we still would have big data centers that needed the same capacity.
- mottosso 6y agoThis rings true. The title should really be "Tech consumption in Sweden needs half a nuclear plant". What else is new?
- mathgladiator 6y agoWouldn't we need more due to more overhead due to less economies of scale?
- Avamander 6y agoVery likely. Public transport vs. a car type of thing I suspect.
- sargun 6y agoYeah, I think the best example is S3. Only when you have to hat much data with a Pareto access distribution can you do things like oversubscribe I/O a ton, or do clever encoding / compression schemes that wouldn’t make sense at the scales of an individual organization.
- sargun 6y agoThere’s also trickle down economics at work here. The rest of the industry got brotli and zstd for free. Every dwarf benefits.
- paxys 6y agoIt really depends on what the capacity is being used for. From my experience, everything user facing (including all web serving, storage etc.) comprises a very tiny fraction of overall costs. The real $$$ (both input and output) is from big data/analytics/ML pipelines. Start a company that doesn't do any of that, and you'll have an infinitely better energy footprint.
- jojobas 6y agoWould the tech dwarfs choose to store all the data they ever came across, forever?
- colinmhayes 6y agoThey'd be stupid not to
- mulmen 6y agoThat's not true. Storing data has costs. And not just hard drives. There are organizational risks in keeping data forever. Analytics could be needlessly complex with excessive data retention. Just store the data you need.
- nemothekid 6y agoEven if they didn't, I'm willing to bet that most datacenters are dominated by compute, not storage.
- sargun 6y agoIt’s likely we would need (much) more. If you look the efficiency of google and facebook they’re significantly better than many smaller organizations when it comes to utilization and PUE. Part of this is because they can afford to throw engineers at the problem, and a 1% savings in something like search easily makes up for an engineer’s salary. Depending on that 1% gain, it could also be applied to other BUs.