> See https://www.nature.com/articles/s41592-021-01326-w https://www.nature.com/articles/s41592-021-01326-w figure 1 a and b for a demonstration of the distinct

> See https://www.nature.com/articles/s41592-021-01326-w https://www.nature.com/articles/s41592-021-01326-w figure 1 a and b for a demonstration of the distinct time-to-first-byte for uncached data (local POSIX, S3, and nginx/http). Looking at your figure, almost a second to download a 128KB chunk with HDF5 seems extremely high. How many requests are being made here? Is the HDF5 metadata being re-read from scratch for each iteration of the benchmark? It might be interesting to trace log each http request being made and its timing during an iteration of the benchmark. Using AWS CloudShell I see range requests for the last 128KB of a 512MB S3 file without authentication taking about 25ms when reusing a connection and 35ms when not. > I'm trying to find a way to accelerate that TTFB for a dataset that lives in S3 and we don't know which parts could be cached before a user shows up and clicks on a dataset. If latency from S3 turns out to be the problem I'm not sure there's any alternative but paying for the space your data uses on fast storage. The cheapest option on AWS will likely be EBS SSD volumes. That runs to $0.08/GB/month or $8000/month for 100TB (4x more than S3) before IOPS charges. You could try they EBS HDD volumes but they do not advertise latency figures and are still 2x S3 pricing.