4 ms·
Amazon storing more than 449 billion objects in S3
- charlesju 15y agoI am calling it here. 1 trillion objects by 2013.
- bdr 15y agoThat's a linear extrapolation from the last two data points on the graph, but it seems to be growing super-linearly.
- jeffbarr 15y agoI work for Amazon (and wrote the blog post referenced by GigaOm), and I certainly won't speculate (or place a bet). However, note that the growth from 262 billion to 449 billion objects took place in 6 months.
- cperciva 15y agoI'm calling 1T+ objects for Q2 2012. The S3 object count is growing by between 150% and 200% every year.
- moe 15y agoI'm curious what their storage capacity is. Assuming an average object size of 100kb that would amount to 45 petabytes (plus redundancy). But I have no idea what their average object size is...
- diolpah 15y agoWe have a few million objects stored in S3 and our average object size is only ~40k. Plenty of people use cloudfront as a streaming CDN, though. Those assets would be far larger, obviously.
- cemerick 15y agoAt what point does S3 break? Very happy user of S3 and much of the rest of AWS here, but I can't help but ask (mostly rhetorically). S3 is presumably the largest system of its kind ever; it's not clear to me what its failure modes, thresholds, or weak links might be as it continues to grow. Anything from a breakdown in the dynamo architecture to hard infrastructure limits to failures in operations and management strike me as plausible. What will we see first: data loss, increases in latency, repeated catastrophic outages, or "other"?
- deleted 15y ago[deleted]
- deleted 15y ago[deleted]
- cemerick 15y agoGetting some feedback/insight on this topic seemed interesting enough to me that I wrote a slightly-expanded post separately: http://cemerick.com/2011/07/20/at-what-point-does-s3-break/ http://cemerick.com/2011/07/20/at-what-point-does-s3-break/
- alexro 15y agoIf DropBox continue to grow, so will S3
- saurik 15y agoPrevious link to the actual source (posted here by an Amazon employee): http://news.ycombinator.com/item?id=2783595 http://news.ycombinator.com/item?id=2783595