3 ms·

I am also experimenting a serverless architecture on top of AWS. Just like you, I tried to directly use S3 + cognito, which is great. The only problem I see is

I am also experimenting a serverless architecture on top of AWS. Just like you, I tried to directly use S3 + cognito, which is great. The only problem I see is that it is not possible to limit the space used by a bucket and, therefore, it could be used as infinite storage by malicious clients. I've checked out with AWS people [1] and asked on stackoverflow [2], but it seems not possible yet. They suggested me to use lambda to delete unwanted content after it was uploaded, which surely works, but is a waste of bandwidth and of lambda time. Did you find the same problems? Any other solution? Beside, it looks like a great approach, but it surely ties you to AWS way more than other approaches. [1] https://twitter.com/davide_vernizzi/status/620994200490930176 https://twitter.com/davide_vernizzi/status/62099420049093017... [2] http://stackoverflow.com/questions/31412335/can-i-limit-the-size-of-an-object-put-into-s3-via-the-javascript-api http://stackoverflow.com/questions/31412335/can-i-limit-the-...

jpatters 11y ago
I've been working on something like this as well and the biggest problem that I have found is that there is a hard limit of 100 S3 buckets per aws account. Which, unfortunately makes it impossible to use S3 in the manner I had intended.
- tigroferoce 11y ago
  AWS has hard limits, but usually can be easily overcome just by asking to their support. We increased many times the hard limits on EC2 instances and EIP.
  - pas256 11y ago
    That's true, but the S3 bucket limit, just like the VPC security group limit, is fixed at 100, even if you are Netflix. --- Edit: Wow, what a difference a day makes.
  - mwarkentin 11y ago
    They actually just changed this yesterday: http://aws.amazon.com/about-aws/whats-new/2015/08/amazon-s3-introduces-new-usability-enhancements/ http://aws.amazon.com/about-aws/whats-new/2015/08/amazon-s3-... > Bucket Limit Increase: You can now increase your Amazon S3 bucket limit per AWS account. All AWS accounts have a default bucket limit of 100 buckets, and starting today you can now request additional buckets by visiting AWS Service Limits.
- connorleech 11y ago
  look into namespacing your buckets. a bucket is not the same as a table. aws works in massive scale production and can suit all kinds of needs. the good folks at the aws popup loft in SF set me straight on this one
  - jpatters 11y ago
    Unfortunately namespacing won't work for us. We need the ability to point a domain name at a bucket. Can't do that with a folder inside a bucket. If I am incorrect on this, it would be great if you could set me straight too.

jpatters 11y ago

I've been working on something like this as well and the biggest problem that I have found is that there is a hard limit of 100 S3 buckets per aws account. Which, unfortunately makes it impossible to use S3 in the manner I had intended.

tigroferoce 11y ago
AWS has hard limits, but usually can be easily overcome just by asking to their support. We increased many times the hard limits on EC2 instances and EIP.
- pas256 11y ago
  That's true, but the S3 bucket limit, just like the VPC security group limit, is fixed at 100, even if you are Netflix. --- Edit: Wow, what a difference a day makes.
- mwarkentin 11y ago
  They actually just changed this yesterday: http://aws.amazon.com/about-aws/whats-new/2015/08/amazon-s3-introduces-new-usability-enhancements/ http://aws.amazon.com/about-aws/whats-new/2015/08/amazon-s3-... > Bucket Limit Increase: You can now increase your Amazon S3 bucket limit per AWS account. All AWS accounts have a default bucket limit of 100 buckets, and starting today you can now request additional buckets by visiting AWS Service Limits.
connorleech 11y ago
look into namespacing your buckets. a bucket is not the same as a table. aws works in massive scale production and can suit all kinds of needs. the good folks at the aws popup loft in SF set me straight on this one
- jpatters 11y ago
  Unfortunately namespacing won't work for us. We need the ability to point a domain name at a bucket. Can't do that with a folder inside a bucket. If I am incorrect on this, it would be great if you could set me straight too.