5 ms·

Anyone else find this surprisingly low? I'd imagine your typical web service holds a few thousand objects in S3 for images etc, then backups and anything else.

Anyone else find this surprisingly low? I'd imagine your typical web service holds a few thousand objects in S3 for images etc, then backups and anything else. Then you have your big players like netflix, dropbox etc that use the service. Who store data for tens of millions of customers...

ceejayoz 14y ago
Netflix is unlikely to store customer data on S3. They use it for movie files IIRC, but customer data would go in a proper database of some sort. Dropbox does use S3, but I think you're probably doing the fairly standard human mistake of not realizing just how big a trillion is. Dropbox hit 100M users in November of 2012, so for Dropbox to use up two trillion objects each user would need to have 20,000 of them. Dropbox does deduplication, has lots of inactive/minimal users, etc., so they're probably a percent or two of S3 objects.
- saurik 14y ago
  Netflix stores most of their user data in Cassandra (at the AWS conference last year I attended a talk by them, and specifically asked after some of the things they were storing). (At one point, the stuff I was storing/logging for Cydia actually represented over a percent, maybe it was even over two percent, of all objects in S3; now I'm between 0.1% and 0.5%.)
charlesju 14y ago
I think you are desensitized to what a trillion means. http://demonocracy.info/infographics/usa/us_debt/us_debt.html http://demonocracy.info/infographics/usa/us_debt/us_debt.htm...
- NewAccnt2 14y ago
  Or ~1 year of money at the current rate of money being printed by the fed ($85 Billion / month)
fooyc 14y ago
1.1 Million Requests/Second seems especially low for Amazon. The average server can serve more than 1,000 static objects / second easily.
- amock 14y ago
  There is more to S3 than just serving static objects off of the local drive. Maintaining the integrity of the data and ensuring that the data sent out is consistent is not a trivial task. A constantly changing map with 2 trillion keys is a hard problem on its own. Also, serving 1000 1MiB objects per second is not the same as serving 1000 1KiB objects per second so it's hard to say how many resources just the serving portion consumes.
  - fooyc 14y ago
    I'm not saying it's simple nor easy. I was assuming that the number of requests referred to read requests, and I guess that the system is designed in a way that makes read requests very cheap, maybe even cheaper than reading a file on a usual filesystem, at least for hot data. Just because it's huge and complex doesn't mean it's slow and that requests are expensive.
- kondro 14y ago
  If you ignore the fact that these aren't actually static objects and require a lot more computation to work out where they are and where they need to go. 1.1M RPS is the amount they actually serve, not how much they can serve. Just because your single server can serve more than 1,000 static objects/second (in fact, that number should be much higher), it doesn't mean you need to.
  - fooyc 14y ago
    > If you ignore the fact that these aren't actually static objects It depends what you call a static object. If the content of your object is stored somewhere and you can just send it without transformations, it's some kind of static object. Now, looking at a single S3 bucket as a key-value store, with some kind of routing mapping an object's URL to a set of shards each containing the object, one could argue it's serving static objects. > require a lot more computation to work out where they are and where they need to go I hope not. I would bet it's not very far from serving a file from the filesystem. There may be a lot of i/o contension though. > in fact, that number should be much higher Yes, very probably, and that only makes the 1.1M RPS number seem even less impressive - for amazon. > 1.1M RPS is the amount they actually serve, not how much they can serve My whole point was this number of requests seem low for amazon, not that they couldn't handle more.

ceejayoz 14y ago

Netflix is unlikely to store customer data on S3. They use it for movie files IIRC, but customer data would go in a proper database of some sort. Dropbox does use S3, but I think you're probably doing the fairly standard human mistake of not realizing just how big a trillion is. Dropbox hit 100M users in November of 2012, so for Dropbox to use up two trillion objects each user would need to have 20,000 of them. Dropbox does deduplication, has lots of inactive/minimal users, etc., so they're probably a percent or two of S3 objects.

saurik 14y ago
Netflix stores most of their user data in Cassandra (at the AWS conference last year I attended a talk by them, and specifically asked after some of the things they were storing). (At one point, the stuff I was storing/logging for Cydia actually represented over a percent, maybe it was even over two percent, of all objects in S3; now I'm between 0.1% and 0.5%.)

charlesju 14y ago

I think you are desensitized to what a trillion means. http://demonocracy.info/infographics/usa/us_debt/us_debt.html http://demonocracy.info/infographics/usa/us_debt/us_debt.htm...

NewAccnt2 14y ago
Or ~1 year of money at the current rate of money being printed by the fed ($85 Billion / month)

fooyc 14y ago

1.1 Million Requests/Second seems especially low for Amazon. The average server can serve more than 1,000 static objects / second easily.

amock 14y ago
There is more to S3 than just serving static objects off of the local drive. Maintaining the integrity of the data and ensuring that the data sent out is consistent is not a trivial task. A constantly changing map with 2 trillion keys is a hard problem on its own. Also, serving 1000 1MiB objects per second is not the same as serving 1000 1KiB objects per second so it's hard to say how many resources just the serving portion consumes.
- fooyc 14y ago
  I'm not saying it's simple nor easy. I was assuming that the number of requests referred to read requests, and I guess that the system is designed in a way that makes read requests very cheap, maybe even cheaper than reading a file on a usual filesystem, at least for hot data. Just because it's huge and complex doesn't mean it's slow and that requests are expensive.
kondro 14y ago
If you ignore the fact that these aren't actually static objects and require a lot more computation to work out where they are and where they need to go. 1.1M RPS is the amount they actually serve, not how much they can serve. Just because your single server can serve more than 1,000 static objects/second (in fact, that number should be much higher), it doesn't mean you need to.
- fooyc 14y ago
  > If you ignore the fact that these aren't actually static objects It depends what you call a static object. If the content of your object is stored somewhere and you can just send it without transformations, it's some kind of static object. Now, looking at a single S3 bucket as a key-value store, with some kind of routing mapping an object's URL to a set of shards each containing the object, one could argue it's serving static objects. > require a lot more computation to work out where they are and where they need to go I hope not. I would bet it's not very far from serving a file from the filesystem. There may be a lot of i/o contension though. > in fact, that number should be much higher Yes, very probably, and that only makes the 1.1M RPS number seem even less impressive - for amazon. > 1.1M RPS is the amount they actually serve, not how much they can serve My whole point was this number of requests seem low for amazon, not that they couldn't handle more.