5 ms·

Well, yeah. Disk cache can take hundreds of MS to retrieve, even on modern SSDs. I had a handful of oddly heated discussions with an architect about this exact

by oblak 4y ago

Well, yeah. Disk cache can take hundreds of MS to retrieve, even on modern SSDs. I had a handful of oddly heated discussions with an architect about this exact thing at my previous job. Showing him the network tab did not because he had read articles and was well informed about these things.

TheDudeMan 4y ago
According to this, it can take 10ms (not 100s of ms) and only for a very large read (like 32MB). https://i.gzn.jp/img/2021/06/07/reading-from-external-memory/nvme-single-read.png https://i.gzn.jp/img/2021/06/07/reading-from-external-memory... Full article: https://gigazine.net/gsc_news/en/20210607-reading-from-external-memory/ https://gigazine.net/gsc_news/en/20210607-reading-from-exter...
- TheDudeMan 4y ago
  (Anyone who is not running their OS and temp space on NVME should not expect good performance. Such a configuration has been very cheap for several years now.)
  - zinekeller 4y ago
    > Such a configuration has been very cheap for several years now. This is a very weird comment, considering that a) it's cheaper than yesteryear but SATA SDD (or even modern magnetic HDDs) are still sold and are in active use and b) ignores phones completely, where a large number of sites would have mobile-dominated visitors and can't just switch to an NVMe-like performance even for those with large disposable incomes (because at the end of the day even with UFS phones are still slower than NVMe latency-wise).
- deleted 4y ago
  [deleted]
- Retr0id 4y ago
  This assumes it only takes a single disk read to locate and retreive an object from the cache (which is unlikely to be the case).
  - TheDudeMan 4y ago
    OK, one small read plus one huge read would top-out at about 10.1ms, according to that graph.
    - Retr0id 4y ago
      Also a big assumption (which we could verify by looking at the relevant implementations, but I'm not going to)
  - staticassertion 4y ago
    Why is that unlikely to be the case?
    - Retr0id 4y ago
      There's going to be at least one level of indirection, and at least some of those indirections are going to be on-disk.
staticassertion 4y ago
The issue has nothing to do with disk speed. If you had read the article you'd see a very nice chart that shows the vast majority of cache hits returning in under 2 or 3ms.
divbzero 4y ago
I wish I had a clearer memory or record of this, but I think I’ve also ~100ms for browser cache retrieval on an SSD. Has anyone else observed this and have an explanation? A sibling comment points out that SSD read latency should be ~10ms at most so the limitation must be in the software? OP mentioned specifically that “there have been bugs in Chromium with request prioritisation, where cached resources were delayed while the browser fetched higher priority requests over the network” and that “Chrome actively throttles requests, including those to cached resources, to reduce I/O contention”. I wonder if there are also other limitations with how browsers retrieve from cache.
- staticassertion 4y ago
  > Has anyone else observed this and have an explanation? Yes that is the subject of this post. https://simonhearne.com/2020/network-faster-than-cache/#the-obvious-bottleneck https://simonhearne.com/2020/network-faster-than-cache/#the-...
  - divbzero 4y ago
    The graphs in OP show that cache latency is mostly ~10ms for desktop browsers. ~100ms would still be an outlier.
    - staticassertion 4y ago
      The Y axis of the chart that I linked, entitled 'Cache Retrieval Time by Count of Cached Assets', shows latency well above 100ms.
      - divbzero 4y ago
        Thanks. Switching the metric from Average to Max does show that the cache retrieval time can reach ~100ms even when cached resource count is low.
OJFord 4y ago
Why are you measuring latency in mega Siemens?
kccqzy 4y ago
At a previous job I worked on serving data from SSDs. I wasn't really involved in configuring the hardware but I believe they were good quality enterprise-grade SSDs. My experience was that a random read (which could be a small number of underlying reads) from mmap()'ed files from those SSDs took between 100 and 200 microseconds. That's far from your figure of hundreds of milliseconds. Of course 200 microseconds still isn't fast. That translates to serving 5000 requests per second, leaving the CPU almost completely idle. Another odd fact was that we in fact did have to implement our own semaphores and throttling to limit concurrent reads from SSDs.

by oblak 4y ago

TheDudeMan 4y ago

According to this, it can take 10ms (not 100s of ms) and only for a very large read (like 32MB). https://i.gzn.jp/img/2021/06/07/reading-from-external-memory/nvme-single-read.png https://i.gzn.jp/img/2021/06/07/reading-from-external-memory... Full article: https://gigazine.net/gsc_news/en/20210607-reading-from-external-memory/ https://gigazine.net/gsc_news/en/20210607-reading-from-exter...

TheDudeMan 4y ago
(Anyone who is not running their OS and temp space on NVME should not expect good performance. Such a configuration has been very cheap for several years now.)
- zinekeller 4y ago
  > Such a configuration has been very cheap for several years now. This is a very weird comment, considering that a) it's cheaper than yesteryear but SATA SDD (or even modern magnetic HDDs) are still sold and are in active use and b) ignores phones completely, where a large number of sites would have mobile-dominated visitors and can't just switch to an NVMe-like performance even for those with large disposable incomes (because at the end of the day even with UFS phones are still slower than NVMe latency-wise).
deleted 4y ago
[deleted]
Retr0id 4y ago
This assumes it only takes a single disk read to locate and retreive an object from the cache (which is unlikely to be the case).
- TheDudeMan 4y ago
  OK, one small read plus one huge read would top-out at about 10.1ms, according to that graph.
  - Retr0id 4y ago
    Also a big assumption (which we could verify by looking at the relevant implementations, but I'm not going to)
- staticassertion 4y ago
  Why is that unlikely to be the case?
  - Retr0id 4y ago
    There's going to be at least one level of indirection, and at least some of those indirections are going to be on-disk.

staticassertion 4y ago

The issue has nothing to do with disk speed. If you had read the article you'd see a very nice chart that shows the vast majority of cache hits returning in under 2 or 3ms.

divbzero 4y ago

I wish I had a clearer memory or record of this, but I think I’ve also ~100ms for browser cache retrieval on an SSD. Has anyone else observed this and have an explanation? A sibling comment points out that SSD read latency should be ~10ms at most so the limitation must be in the software? OP mentioned specifically that “there have been bugs in Chromium with request prioritisation, where cached resources were delayed while the browser fetched higher priority requests over the network” and that “Chrome actively throttles requests, including those to cached resources, to reduce I/O contention”. I wonder if there are also other limitations with how browsers retrieve from cache.

staticassertion 4y ago
> Has anyone else observed this and have an explanation? Yes that is the subject of this post. https://simonhearne.com/2020/network-faster-than-cache/#the-obvious-bottleneck https://simonhearne.com/2020/network-faster-than-cache/#the-...
- divbzero 4y ago
  The graphs in OP show that cache latency is mostly ~10ms for desktop browsers. ~100ms would still be an outlier.
  - staticassertion 4y ago
    The Y axis of the chart that I linked, entitled 'Cache Retrieval Time by Count of Cached Assets', shows latency well above 100ms.
    - divbzero 4y ago
      Thanks. Switching the metric from Average to Max does show that the cache retrieval time can reach ~100ms even when cached resource count is low.

OJFord 4y ago

Why are you measuring latency in mega Siemens?

kccqzy 4y ago

At a previous job I worked on serving data from SSDs. I wasn't really involved in configuring the hardware but I believe they were good quality enterprise-grade SSDs. My experience was that a random read (which could be a small number of underlying reads) from mmap()'ed files from those SSDs took between 100 and 200 microseconds. That's far from your figure of hundreds of milliseconds. Of course 200 microseconds still isn't fast. That translates to serving 5000 requests per second, leaving the CPU almost completely idle. Another odd fact was that we in fact did have to implement our own semaphores and throttling to limit concurrent reads from SSDs.