3 ms·
Anyone have an updated estimate for how much storage could be fit in a shipping container and its bandwidth equivalent? [https://news.ycombinator.com/item?id=1
by lazide 3y ago
Anyone have an updated estimate for how much storage could be fit in a shipping container and its bandwidth equivalent?
[https://news.ycombinator.com/item?id=1437634#:~:text=Conservatively%20speaking%2C%20you%20co...%20%7C%20Hacker%20News&text=file%20sharing%20network-,Or%20a%20shipping%20container%20filled%20with%20hard%20drives.,3.5%22%202%20TB%20hard%20drives https://news.ycombinator.com/item?id=1437634#:~:text=Conserv....] says 60PB, which for 20 TB drives would be approx 3000.
A petabyte is still a lot of data!
In my experience the bigger issue is that much large data sets are rarely truly static, and inventorying/tracking/validating is a massive undertaking all on it’s own.
- voidUpdate 3y agoObligatory XKCD: https://what-if.xkcd.com/31/ https://what-if.xkcd.com/31/
- lazide 3y agoAs a side note since XKXD leaves this out and having gotten involved in implementing systems using physical storage for transport in the past, physical copying to/from media is a major issue in performance and capital costs. Most large physical hard drives max out around 300-400MB/s transfer rate, so it typically takes several days to fill a single drive. And you can’t ship anything until all the media is written. So either you have a ton of online, parallel drives (which dramatically increases capex, and decreases available storage due to physical space, power, and cooling issues) or you go really slow. Even having 300 parallel full speed drive interfaces for instance, it would take approximately a year to fill those 3000 drives with data, and approx. another year to read all the data off. Which dramatically changes all the calculations.