4 ms·
So the primary reason for the multiple second response times is that many sequential megabytes have to be read from an HD in order to return an often quite tiny
by johansch 9y ago
So the primary reason for the multiple second response times is that many sequential megabytes have to be read from an HD in order to return an often quite tiny HTTP response data for a particular {url, timestamp}? That would explain the nearly constantly (slow) response time.
(I was puzzled why it didn't seem to vary much depending on the time of day, etc.)
- bnewbold 9y agoWell, I oversimplified a bit. The "tarball" (WARC or ARC file) is a single large file with individually compressed HTTP response objects concatenated together. The index stores the byte offset and length (compressed) into this file, so an HTTP/1.1 range request can pull only the data needed. So in theory it's efficient. I don't work on that team directly and haven't looked in to exactly what the performance latency bottleneck is, sorry for the misleading response.
- johansch 9y agoOk. Well, I do hope you work on improving the response times. I think it would do wonders for the popularity of the wayback machine (and in the end, financial contributions).