3 ms·
Does S3 really have that high of a latency? So high that —— if you run a static file several in an EC2, would that be faster than S3?
by winter_blue 1y ago
Does S3 really have that high of a latency? So high that —— if you run a static file several in an EC2, would that be faster than S3?
- reese_john 1y agoS3 has a low-latency offering[0] which promises single digit millisecond latency, I’m surprised not to see it mentioned. [0]: https://aws.amazon.com/s3/storage-classes/express-one-zone/ https://aws.amazon.com/s3/storage-classes/express-one-zone/
- huntaub 1y agoThese are, effectively, different use cases. You want to use (and pay for) Express One Zone in situations in which you need the same object reused from multiple instances repeatedly, while it looks like this on-disk or in-memory cache is for when you may want the same file repeatedly used from the same instance.
- manquer 1y agoIs it the same instance ? Rising wave (and similar tools )are designed to run in production on a lot of distributed compute nodes for processing data , serving/streaming queries and running control panes . Even for any single query it will likely run on multiple nodes with distributed workers gathering and processing data from storage layer, that is whole idea behind MapReduce after all.
- artursapek 1y agoAlso, aren't most people putting Cloudfront in front of S3 anyway?
- hobofan 1y agoFor CDN use-cases yes, but not for DB storage-compute separation use-cases as described here.
- huntaub 1y agoYes, definitely. S3 has a time to first byte of 50-150ms (depending on how lucky you are). If you're serving from memory that goes to ~0, and if you're serving from disk, that goes to 0.2-1ms. It will depend on your needs though, since some use cases won't want to trade off the scalability of S3's ability to serve arbitrary amounts of throughput.
- manquer 1y agoIn that case you run the proxy service load balanced to get desired throughput or run a sidecar/process in each compute instance where data is needed . You are limited anyway by the network capacity of the instance you are fetching the data from .