4 ms·
> You will hit limits to how fast you can generate those crypto signatures for s3 Presigned URLs are HMAC-SHA256: https://docs.aws.amazon.com/AmazonS3/latest/
by willglynn 7y ago
> You will hit limits to how fast you can generate those crypto signatures for s3
Presigned URLs are HMAC-SHA256:
https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html#example-signature-calculations https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-heade...
I just measured GetObjectRequest.Presign() at 1063 ns. That's a million signatures per second of my 2015 MacBook Pro using the stock AWS SDK for Go _on one core_. This is plenty fast already, and it's guaranteed to be running in parallel on better hardware in production. There's no way signature generation is a bottleneck in any real application.
> and a limit to how much can serve
I've gotten hundreds of gigabits per second between S3 and EC2 without doing anything special. What problems are you referring to?
> at a certain scale you will want to use a cdn
So use a CDN too. CloudFront fits there neatly, and it can even sign its own requests to S3:
https://aws.amazon.com/blogs/networking-and-content-delivery/amazon-s3-amazon-cloudfront-a-match-made-in-the-cloud/ https://aws.amazon.com/blogs/networking-and-content-delivery...
None of those criticisms are valid reasons to avoid S3. S3 is an excellent solution for many common problems.
Having said that, S3 has genuine drawbacks for certain workloads. (For example, the long tail on time to first byte is sometimes a dealbreaker.) In my opinion the biggest limitation is budgetary: typically one will spend all their money on AWS data transfer fees long before hitting S3 service limits.
- grogenaut 7y agoTons of tiny files is the problem where the bytes to sign ratio is low. And I didn't say to avoid s3. I work for a subsiary of AWS I want you to use it a ton. Let's say your serving up a bunch of thumbnails, maybe 1024 per user per load. Or in my scenario 25k files. 25k signs per user is 40 users per second per server signing constantly.