Performance is fun! One aspect is that our data centers are in California, with no CDN. If you are on the other side of the world, you will have higher round-t

Performance is fun! One aspect is that our data centers are in California, with no CDN. If you are on the other side of the world, you will have higher round-trip latency on every request, for all services. Another is layers of caching. Popular or recently requested Wayback content is more likely to be in either an explicit cache (eg, redis), or implicitly in kernel page caches across all layers of the request. Every wayback replay request hits several layers of index indexes (sorted by domain, path, and timestamp), which are huge and thus actually served from spinning disk over HTTP (!). This includes a timeline summary for the primary document, to display the banner. Then the actual raw records are fetched from another spinning disk over HTTP. This may result in one or more layers of internal redirect (additional fetches) if there was a "revisit" (identical HTTP body content, same URL, different timestamp). Then finally the record is re-written for replay (for HTML, CSS, Javascript, etc, unless the raw record was requested). Some pages will have many sub-resources, so this process is repeated many times, but that is the same as page load and you can see which resources are slow or not. As mentioned in the video, depending on where we are in the network hardware upgrade lifecycle, sometimes outbound bandwidth is tight also, which slows down transfer. And of course most of these services operate without a ton of overhead, so if there is a spike in traffic everything will slow down a bit. There is a lot of multi-tenancy-like situations also, so if there is a very popular zip file or Flash game getting served from the same storage disk as the WARC file holding a wayback resource, the replay for that specific resource will be slow due to disk I/O contention. If you are curious about why a specific HTML wayback replay was slow, you can look in the source code of the re-written document and see some timing numbers. Several organizations run large web archives that operate similarly to web.archive.org, and have described cost/benefit trade offs for different components. Eg, National Library of Australia has an alternative CDX index called OutbackCDX, which uses RocksDB on SSDs. I believe other folks store WARC files in S3 or S3-like object storage systems. The Wayback Machine is somewhat unique in the amount of (read) traffic it gets, the heterogeneity of archived content (from several crawlers, in older ARC as well as WARC), volume of live crawling ("save paper now" results show up pretty fast in the main site, which is black magic), running on "boring" general purpose hardware, and deep integration with our general purpose storage cluster. Note: I work at IA but not on the Wayback system