2 ms·

So, why do Instagram and Netflix continue to use S3 for petabytes of video and image data? In Netflix' case, this doesn't even count their vast replicated CDN

So, why do Instagram and Netflix continue to use S3 for petabytes of video and image data? In Netflix' case, this doesn't even count their vast replicated CDN infrastructure between your device and S3. The math is never this simple. Firstly I assume you mean 700 TB, not GB. You might as well look at reduced-redundancy S3 which brings the annual cost down to $194,220. With that you still have multiple WAN links, excellent throughput, multiple data centers, and no labour for dealing with failed drives. And if we're truly trying to do backup / archival, where download throughput doesn't matter, let's look at Glacier, even if we do 50000 monthly uploads and retrieve 20% of the 700TB monthly, we're down to $96,630. Most sizable businesses also aren't buying bargain basement drive arrays from SuperMicro, they're buying EMC, NetApp, 3PAR, or Hitachi. As with anything S3 is not perfect, but it really depends on your use case if your own data center or rented equipment is appropriate. For many large businesses, Amazon makes more sense.

skuhn 12y ago
> In Netflix' case, this doesn't even count their vast replicated CDN infrastructure between your device and S3. There is part of your answer. They only pay S3 outbound on cache misses, not on cache hits. That will lower their S3 bandwidth bill to 1/50th or 1/100th of the full cost. Also, Netflix doesn't pay list prices for AWS, and they made a call that they wanted to be fully cloud based early in the streaming service's life. Maybe they would do it differently today, but that ship has obviously sailed for them. On the other hand, lots of large AWS hosted services do discover that they need to move off in order to get their costs right; I have quite a bit of first hand experience there. > Glacier Costs and time to retrieve data from Glacier are quite different, and I would only use it for data that I basically never expect to access. A service like Backblaze couldn't use Glacier in reality, despite being a service for backups. > Bargain basement drive arrays On the contrary, I think that building it yourself with basic hardware building blocks is much more common at large scale Internet companies. A less technical or more appliance driven company might choose NetApp or EMC, but I don't think it's the right move for somewhere with some in-house expertise. It's the right move if you want more of a black box that you hope will work without much internal effort (hint: it won't work out that way).

skuhn 12y ago

> In Netflix' case, this doesn't even count their vast replicated CDN infrastructure between your device and S3. There is part of your answer. They only pay S3 outbound on cache misses, not on cache hits. That will lower their S3 bandwidth bill to 1/50th or 1/100th of the full cost. Also, Netflix doesn't pay list prices for AWS, and they made a call that they wanted to be fully cloud based early in the streaming service's life. Maybe they would do it differently today, but that ship has obviously sailed for them. On the other hand, lots of large AWS hosted services do discover that they need to move off in order to get their costs right; I have quite a bit of first hand experience there. > Glacier Costs and time to retrieve data from Glacier are quite different, and I would only use it for data that I basically never expect to access. A service like Backblaze couldn't use Glacier in reality, despite being a service for backups. > Bargain basement drive arrays On the contrary, I think that building it yourself with basic hardware building blocks is much more common at large scale Internet companies. A less technical or more appliance driven company might choose NetApp or EMC, but I don't think it's the right move for somewhere with some in-house expertise. It's the right move if you want more of a black box that you hope will work without much internal effort (hint: it won't work out that way).