12 ms·

When network is faster than browser cache (2020)

331 points by harporoeder 4y ago

deleted 4y ago
[deleted]
hgazx 4y ago
When I had a very old and slow hard disk I ran my browsers without disk cache precisely because of this reason.
dllthomas 4y ago
... oh, that cache.
- femiagbabiaka 4y ago
  I had the same reaction. It's a good title.
oblak 4y ago
Well, yeah. Disk cache can take hundreds of MS to retrieve, even on modern SSDs. I had a handful of oddly heated discussions with an architect about this exact thing at my previous job. Showing him the network tab did not because he had read articles and was well informed about these things.
- TheDudeMan 4y ago
  According to this, it can take 10ms (not 100s of ms) and only for a very large read (like 32MB). https://i.gzn.jp/img/2021/06/07/reading-from-external-memory/nvme-single-read.png https://i.gzn.jp/img/2021/06/07/reading-from-external-memory... Full article: https://gigazine.net/gsc_news/en/20210607-reading-from-external-memory/ https://gigazine.net/gsc_news/en/20210607-reading-from-exter...
  - TheDudeMan 4y ago
    (Anyone who is not running their OS and temp space on NVME should not expect good performance. Such a configuration has been very cheap for several years now.)
    - zinekeller 4y ago
      > Such a configuration has been very cheap for several years now. This is a very weird comment, considering that a) it's cheaper than yesteryear but SATA SDD (or even modern magnetic HDDs) are still sold and are in active use and b) ignores phones completely, where a large number of sites would have mobile-dominated visitors and can't just switch to an NVMe-like performance even for those with large disposable incomes (because at the end of the day even with UFS phones are still slower than NVMe latency-wise).
  - deleted 4y ago
    [deleted]
  - Retr0id 4y ago
    This assumes it only takes a single disk read to locate and retreive an object from the cache (which is unlikely to be the case).
    - TheDudeMan 4y ago
      OK, one small read plus one huge read would top-out at about 10.1ms, according to that graph.
      - Retr0id 4y ago
        Also a big assumption (which we could verify by looking at the relevant implementations, but I'm not going to)
    - staticassertion 4y ago
      Why is that unlikely to be the case?
      - Retr0id 4y ago
        There's going to be at least one level of indirection, and at least some of those indirections are going to be on-disk.
- staticassertion 4y ago
  The issue has nothing to do with disk speed. If you had read the article you'd see a very nice chart that shows the vast majority of cache hits returning in under 2 or 3ms.
- divbzero 4y ago
  I wish I had a clearer memory or record of this, but I think I’ve also ~100ms for browser cache retrieval on an SSD. Has anyone else observed this and have an explanation? A sibling comment points out that SSD read latency should be ~10ms at most so the limitation must be in the software? OP mentioned specifically that “there have been bugs in Chromium with request prioritisation, where cached resources were delayed while the browser fetched higher priority requests over the network” and that “Chrome actively throttles requests, including those to cached resources, to reduce I/O contention”. I wonder if there are also other limitations with how browsers retrieve from cache.
  - staticassertion 4y ago
    > Has anyone else observed this and have an explanation? Yes that is the subject of this post. https://simonhearne.com/2020/network-faster-than-cache/#the-obvious-bottleneck https://simonhearne.com/2020/network-faster-than-cache/#the-...
    - divbzero 4y ago
      The graphs in OP show that cache latency is mostly ~10ms for desktop browsers. ~100ms would still be an outlier.
      - staticassertion 4y ago
        The Y axis of the chart that I linked, entitled 'Cache Retrieval Time by Count of Cached Assets', shows latency well above 100ms.
        divbzero 4y ago
        Thanks. Switching the metric from Average to Max does show that the cache retrieval time can reach ~100ms even when cached resource count is low.
- OJFord 4y ago
  Why are you measuring latency in mega Siemens?
- kccqzy 4y ago
  At a previous job I worked on serving data from SSDs. I wasn't really involved in configuring the hardware but I believe they were good quality enterprise-grade SSDs. My experience was that a random read (which could be a small number of underlying reads) from mmap()'ed files from those SSDs took between 100 and 200 microseconds. That's far from your figure of hundreds of milliseconds. Of course 200 microseconds still isn't fast. That translates to serving 5000 requests per second, leaving the CPU almost completely idle. Another odd fact was that we in fact did have to implement our own semaphores and throttling to limit concurrent reads from SSDs.
TheDudeMan 4y ago
Seems like a badly-designed cache.
- charcircuit 4y ago
  When you make the request the server the server will have to look up the image from its own "cache" before having to send it back to you. The client's cache would have to be not only slower than it's ping, but slower than it's ping + the server's "cache."
- staticassertion 4y ago
  The issue has nothing to do with the cache itself. It's about the throttling behavior.
  - gowld 4y ago
    The throttling is part of the cache design. It's browser cache, not a multi-client cache in the OS.
    - staticassertion 4y ago
      That doesn't sound right based on my reading of this document. If I'm wrong please do correct me, I don't know a ton about the internals here. https://docs.google.com/document/d/1Aa7OKFRdtmn4IFzgHYfqeqk5lnyTiv4jkwCzMoAlrTU/edit#heading=h.zdlavartcrrw https://docs.google.com/document/d/1Aa7OKFRdtmn4IFzgHYfqeqk5...
cwoolfe 4y ago
So is the takeaway that data in the RAM of some server connected by fast network is sometimes "closer" in retrieval time than that same data on a local SSD?
- staticassertion 4y ago
  Not really. The issue is with throttling.
- philsnow 4y ago
  Back in ~2003 I had bought a new motherboard + cpu (a Duron 800MHz IIRC) but as a poor college kid, only had enough money left over for 128MB of RAM.. but the system I was replacing had ~768MB. I made a ~640MB ramdisk on the old system and mounted it on the new system as a network block device, and the result was much, much faster than local swap (this was before consumer SSDs though). [0] "nbd" / https://en.wikipedia.org/wiki/Network_block_device https://en.wikipedia.org/wiki/Network_block_device ; this driver is of course still in the kernel; you could do this today with an anemic raspberry pi if you wanted
  - adamius 4y ago
    Now I'm imagining a rack of raspis acting as one giant ram swap drive over nbd. This could work for a given value of "work". cost of pi vs stick of ram. A kv storage as well perhaps. Then again, whats a TB worth on just one xeon server? Probably cheaper... or not?
    - jhartwig 4y ago
      Have you seen the cost of Pis lately :)
      - adamius 4y ago
        Didn't say much about availability either :P
staticassertion 4y ago
Since apparently no one is willing to read this excellent article, which even comes with fun sliders and charts... > It turns out that Chrome actively throttles requests, including those to cached resources, to reduce I/O contention. This generally improves performance, but will mean that pages with a large number of cached resources will see a slower retrieval time for each resource.
- cogman10 4y ago
  Seems like a shortcut that shouldn't be. I can understand throttling network requests, but disk requests? The only reason to do that would be for power savings (you don't want to push the CPU into a higher state as it loads up the data).
  - sieabahlpark 4y ago
  - staticassertion 4y ago
    I imagine it's just that the cache is in an awkward place where the throttling logic is unaware of the cache and the cache is not able to preempt the throttling logic.
    - cogman10 4y ago
      I'd guess the same thing is going on. Definitely FEELS like a code structure problem.
  - fnordpiglet 4y ago
    Depends on if you value latency for the user. Saying I try both and choose the one coming first hurts no one but the servers not being protected by a client cache. But there’s absolutely no reason to believe a client has a cache that masks requests. AFAIK there’s no standard that says clients use caches for parsimony and not exclusively latency. As a matter of fact I think this is a good idea if it ever takes time to consult the cache, and the trade off is more bandwidth consumption which we are awash in. If you care that much run a caching proxy and use that and you’ll get the same effect of the client side cache masking requests. But I would say it’s superior because it always uses the local cache first and doesn’t waste user time on the edge condition in their cache coherency. It comes from Netscape which famously convinced everyone that it’s one of the hardest problems. That leads to the final benefit, the cache doesn’t have to cohere. If it’s too expensive at that moment to cohere and query then I can use the network source. Again the only downside is the network bandwidth is more consistently user. I would be hard pressed to believe most Firefox users already are grossly bandwidth over provisioned, and the amount of a fraction of a cable line a web browser loading from the cache no one could even notice that.
agumonkey 4y ago
What's the storage capacity of internet cables ?
- moralestapia 4y ago
  On the order of petabytes.
- arccy 4y ago
  https://github.com/yarrick/pingfs https://github.com/yarrick/pingfs
- LeonenTheDK 4y ago
  Explored here: http://tom7.org/harder/ http://tom7.org/harder/ There's a video and a paper linked on this page with the information on this absurdity.
  - agumonkey 4y ago
    oh of course, how could I forget this :)
r1ch 4y ago
For me this is very noticeable whenever I open a new Chrome tab. It takes 3+ seconds for the icons of the recently visited sites to appear, whatever cache is used for the favicons is extremely slow. Thankfully the disk cache for other resources runs at a more normal speed.
gowld 4y ago
The article shows a lot of data about cache speed, but I don't see a comparison to cacheless network.
sattoshi 4y ago
> This seemed odd to me, surely using a cached response would always be faster than making the whole request again! Well it turns out that in some cases, the network is faster than the cache. Did I miss a follow up on this, or did it remain unanswered as to what the benefit of racing against the network is? The post basically says that sometimes the cache is slower because of throttling or bugs, but mostly bugs. Why is Firefox sending an extra request instead of figuring out what is slowing down the cache? It seems like an overly expensive mitigation…
- 8n4vidtmkvmk 4y ago
  only reason i can think of is slow mechanical hard drive vs fast network. or if its based on etag and it has to hash the local resource, but it would be smart to cache that hash too. or maybe the os has too many file handles open and its doing a poor job of reading many files at once, whereas the network offers a 2nd tube to funnel data through...
philsnow 4y ago
> Concatenating / bundling your assets is probably still a good practice, even on H/2 connections. Obviously this comes on balance with cache eviction costs and splitting pre- and post-load bundles. I guess this latter part refers to the trade-off between compiling all your assets into a single file, and then requiring clients to re-download the entire bundle if you change a single CSS color. The other extreme is to not bundle anything (which, I gather from the article, is the standard practice since all major browsers support HTTP/2) but this leads to the described issue. What about aggressively bundling, but also keeping track at compile time of diffs between historical bundles and the new bundle? Re-connecting clients could grab a manifest that names the newest mega-bundle as well as a map from historical versions to the patches needed to bring them up to date. A lot more work on the server side but maybe it could be a good compromise? Of course that's the easy version, but it has a huge flaw which is all first-time clients have to download the entire huge mega bundle before the browser can render anything, so to make it workable it would have to compile things into a few bootstrap stages instead of a single mega-bundle. I am clearly not a frontend dev. If you're going to throw tomatoes please also tell me why ;) * edit: found the repo that made me think of this idea, https://github.com/msolo/msolo/blob/master/vfl/vfl.py https://github.com/msolo/msolo/blob/master/vfl/vfl.py but it's from antiquity and probably predates node and babel/webpack, but the idea is you name individual asset versions with a SHA or tag and let them be cached forever, and to update the web app you just change a reference in the root to make clients download a different resource dependency tree root (and they re-use unchanged ones) *
- staticassertion 4y ago
  There's probably some balance here. Since there's a limit of 9 concurrent requests before throttling occurs you can bucket 9 objects, concatenating into each. So if you have a bunch of static content, concat that into 1 bucket. If you have another object that changes a lot, keep that separate. If you have two other objects that change together, bucket those, etc. Seems like a huge pain to think about tbh. Seems like part of the problem would be solved by compiling everything into a single file that supported streaming execution.
- philsnow 4y ago
  Another tool that bears on this idea is courgette [0], in that it operates on one of the intermediate representations of the final bundle in order to achieve better compression. https://blog.chromium.org/2009/07/smaller-is-faster-and-safer-too.html https://blog.chromium.org/2009/07/smaller-is-faster-and-safe...
neomantra 4y ago
Different anecdote, but similar vibe.... In ~2010, I was benchmarking Solarflare (Xilinx/AMD now) cards and their OpenOnload kernel-bypass network stack. The results showed that two well-tuned systems could communicate faster (lower latency) than two CPU sockets within the same server that had to wait for the kernel to get involved (standard network stack). It was really illuminating and I started re-architecting based on that result. Backing out some of that history... in ~2008, we started using FPGAs to handle specific network loads (US equity market data). It was exotic and a lot of work, but it significantly benefited that use case, both because of DMA to user-land and its filtering capabilities. At that time our network was all 1 Gigabit. Soon thereafter, exchanges started offering 10G handoffs, so we upgraded our entire infrastructure to 10G cut-through switches (Arista) and 10G NICs (Myricom). This performed much better than the 1G FPGA and dramatically improved our entire infrastructure. We then ported our market data feed handler's to Myricom's user-space network stack, because loads were continually increasing and the trading world was continually more competitive... and again we had a narrow solution (this time in software) to a challenging problem. Then about a year later, Solarflare and it's kernel-compatible OpenOnload arrived and we could then apply the power of kernel bypass to our entire infrastructure. After that, the industry returned to FPGAs again with 10G PHY and tons of space to put whole strategies... although I was never involved with that next generation of trading tech. I personally stayed with OpenOnload for all sorts of workloads, growing to use it with containerization and web stacks (Redis, Nginx). Nowadays you can use OpenOnload with XDP; again a narrow technology grows to fit broad applicability.
- oogali 4y ago
  I went down a similar path (Solarflare, Myricom, and Chelsio; Arista, BNT, and 10G Twinax) and found we could get lower latency internally by confining two OpenOnload-enabled processes that needed to communicate with each other to the same NUMA domain. We architected our applications around that as well. The firm continued on to the FPGA path though that was after my time there. I do still pick up SolarFlare cards off of eBay for home lab purposes.
  - neomantra 4y ago
    Nice! We deploy like that too. I document how to do that with Redis here [1]. [1] https://gist.github.com/neomantra/3c9b89887d19be6fa5708bf4017c0ecd https://gist.github.com/neomantra/3c9b89887d19be6fa5708bf401...
btdmaster 4y ago
The cache, for me, is 100:1 winning the races compared to the network. Are there greatly different results for others in about:networking#rcwn?
cyounkins 4y ago
Worth noting that around the end of 2020, Chrome and Firefox enabled cache partitioning by eTLD+1 to prevent scripts from gaining info from a shared cache. This kills the expected high hit rate from CDN-hosted libraries. https://developer.chrome.com/blog/http-cache-partitioning/ https://developer.chrome.com/blog/http-cache-partitioning/ https://developer.mozilla.org/en-US/docs/Web/Privacy/State_Partitioning https://developer.mozilla.org/en-US/docs/Web/Privacy/State_P...
- Thorrez 4y ago
  >partitioning by eTLD+1 They were already partitioned by the requested eTLD+1, because they were already partitioned by the requested URL, which contains the requested eTLD+1. Chrome's change was additionally partitioning it by 2 more eTLD+1s: the top-level eTLD+1 and the frame eTLD+1. It looks like Firefox's change was to add 1 eTLD+1 to the original behavior: the top-level eTLD+1.
- mobilio 4y ago
  First this was implemented in Safari years before 2020.
kreetx 4y ago
Much confusion in the comments. Tl;dr: cache is "slow" because the number of ongoing requests-- including to cache! --are throttled by the browser. I.e the cache isn't slow, but reading it is waited upon and the network request might be ahead in the queue.
dblohm7 4y ago
Firefox's HTTP cache races with the network for precisely this reason.
- staticassertion 4y ago
  That is what the first sentence of the article is about and it is the premise of the entire article?
  - dblohm7 4y ago
    Ah, so it is! I got overexcited because I was one of the people who advocated for a racy cache during the redesign of that code back in the day.
- capableweb 4y ago
  Then that implementation doesn't even save bandwidth (for client nor server), which is one of the purposes of having a cache in the first place (together with other reasons).
  - dblohm7 4y ago
    When the cache wins the network transfer is canceled, so it’s not as bad as you’re making it out to be.
- edflsafoiewq 4y ago
  Can I turn this off?
langsoul-com 4y ago
Never knew caches could take longer to retrieve. Always thought it'd be near instant. No good solutions either, weird throttling isn't feasible when creating Web apps.
acd 4y ago
Sun Microsystems stated: "The network is the computer" https://en.m.wikipedia.org/wiki/The_Network_is_the_Computer https://en.m.wikipedia.org/wiki/The_Network_is_the_Computer Sun was right.
kardos 4y ago
I wonder if browsers could design a heuristic to cache multiple items in one cache entry -- that is, instead of the website doing file concatenation, the browser dynamically decides to concatenate some resources in order to reduce the number of asks to the local cache. For example, the browser would know on the initial load which requests get made together (such as three js files or four images), so they could get batched into a single cache entry
jesprenj 4y ago
https://simonhearne.com/2020/network-faster-than-cache/#by-operating-system https://simonhearne.com/2020/network-faster-than-cache/#by-o... When I first saw this, I was confused why Linux (and Ubuntu and Debian) performed so poorly. But then I asked myself; is the data even representable? Or is Linux worse because users on Linux tend to stick to lower end hardware, since the measurenents were, AFAIR, taken for granted from website visitors instead of being tested on same hardware with different operating systems installed? I don't know, can someone explain the discrepancy? Is the raw data available? What is the number of requests that were included in the measurements for Debian (that doesn't hit cache in the first 5 ms)? Thanks!

331 points by harporoeder 4y ago

deleted 4y ago

[deleted]

hgazx 4y ago

When I had a very old and slow hard disk I ran my browsers without disk cache precisely because of this reason.

dllthomas 4y ago

... oh, that cache.

femiagbabiaka 4y ago
I had the same reaction. It's a good title.

oblak 4y ago

Well, yeah. Disk cache can take hundreds of MS to retrieve, even on modern SSDs. I had a handful of oddly heated discussions with an architect about this exact thing at my previous job. Showing him the network tab did not because he had read articles and was well informed about these things.

TheDudeMan 4y ago
According to this, it can take 10ms (not 100s of ms) and only for a very large read (like 32MB). https://i.gzn.jp/img/2021/06/07/reading-from-external-memory/nvme-single-read.png https://i.gzn.jp/img/2021/06/07/reading-from-external-memory... Full article: https://gigazine.net/gsc_news/en/20210607-reading-from-external-memory/ https://gigazine.net/gsc_news/en/20210607-reading-from-exter...
- TheDudeMan 4y ago
  (Anyone who is not running their OS and temp space on NVME should not expect good performance. Such a configuration has been very cheap for several years now.)
  - zinekeller 4y ago
    > Such a configuration has been very cheap for several years now. This is a very weird comment, considering that a) it's cheaper than yesteryear but SATA SDD (or even modern magnetic HDDs) are still sold and are in active use and b) ignores phones completely, where a large number of sites would have mobile-dominated visitors and can't just switch to an NVMe-like performance even for those with large disposable incomes (because at the end of the day even with UFS phones are still slower than NVMe latency-wise).
- deleted 4y ago
  [deleted]
- Retr0id 4y ago
  This assumes it only takes a single disk read to locate and retreive an object from the cache (which is unlikely to be the case).
  - TheDudeMan 4y ago
    OK, one small read plus one huge read would top-out at about 10.1ms, according to that graph.
    - Retr0id 4y ago
      Also a big assumption (which we could verify by looking at the relevant implementations, but I'm not going to)
  - staticassertion 4y ago
    Why is that unlikely to be the case?
    - Retr0id 4y ago
      There's going to be at least one level of indirection, and at least some of those indirections are going to be on-disk.
staticassertion 4y ago
The issue has nothing to do with disk speed. If you had read the article you'd see a very nice chart that shows the vast majority of cache hits returning in under 2 or 3ms.
divbzero 4y ago
I wish I had a clearer memory or record of this, but I think I’ve also ~100ms for browser cache retrieval on an SSD. Has anyone else observed this and have an explanation? A sibling comment points out that SSD read latency should be ~10ms at most so the limitation must be in the software? OP mentioned specifically that “there have been bugs in Chromium with request prioritisation, where cached resources were delayed while the browser fetched higher priority requests over the network” and that “Chrome actively throttles requests, including those to cached resources, to reduce I/O contention”. I wonder if there are also other limitations with how browsers retrieve from cache.
- staticassertion 4y ago
  > Has anyone else observed this and have an explanation? Yes that is the subject of this post. https://simonhearne.com/2020/network-faster-than-cache/#the-obvious-bottleneck https://simonhearne.com/2020/network-faster-than-cache/#the-...
  - divbzero 4y ago
    The graphs in OP show that cache latency is mostly ~10ms for desktop browsers. ~100ms would still be an outlier.
    - staticassertion 4y ago
      The Y axis of the chart that I linked, entitled 'Cache Retrieval Time by Count of Cached Assets', shows latency well above 100ms.
      - divbzero 4y ago
        Thanks. Switching the metric from Average to Max does show that the cache retrieval time can reach ~100ms even when cached resource count is low.
OJFord 4y ago
Why are you measuring latency in mega Siemens?
kccqzy 4y ago
At a previous job I worked on serving data from SSDs. I wasn't really involved in configuring the hardware but I believe they were good quality enterprise-grade SSDs. My experience was that a random read (which could be a small number of underlying reads) from mmap()'ed files from those SSDs took between 100 and 200 microseconds. That's far from your figure of hundreds of milliseconds. Of course 200 microseconds still isn't fast. That translates to serving 5000 requests per second, leaving the CPU almost completely idle. Another odd fact was that we in fact did have to implement our own semaphores and throttling to limit concurrent reads from SSDs.

TheDudeMan 4y ago

Seems like a badly-designed cache.

charcircuit 4y ago
When you make the request the server the server will have to look up the image from its own "cache" before having to send it back to you. The client's cache would have to be not only slower than it's ping, but slower than it's ping + the server's "cache."
staticassertion 4y ago
The issue has nothing to do with the cache itself. It's about the throttling behavior.
- gowld 4y ago
  The throttling is part of the cache design. It's browser cache, not a multi-client cache in the OS.
  - staticassertion 4y ago
    That doesn't sound right based on my reading of this document. If I'm wrong please do correct me, I don't know a ton about the internals here. https://docs.google.com/document/d/1Aa7OKFRdtmn4IFzgHYfqeqk5lnyTiv4jkwCzMoAlrTU/edit#heading=h.zdlavartcrrw https://docs.google.com/document/d/1Aa7OKFRdtmn4IFzgHYfqeqk5...

cwoolfe 4y ago

So is the takeaway that data in the RAM of some server connected by fast network is sometimes "closer" in retrieval time than that same data on a local SSD?

staticassertion 4y ago
Not really. The issue is with throttling.
philsnow 4y ago
Back in ~2003 I had bought a new motherboard + cpu (a Duron 800MHz IIRC) but as a poor college kid, only had enough money left over for 128MB of RAM.. but the system I was replacing had ~768MB. I made a ~640MB ramdisk on the old system and mounted it on the new system as a network block device, and the result was much, much faster than local swap (this was before consumer SSDs though). [0] "nbd" / https://en.wikipedia.org/wiki/Network_block_device https://en.wikipedia.org/wiki/Network_block_device ; this driver is of course still in the kernel; you could do this today with an anemic raspberry pi if you wanted
- adamius 4y ago
  Now I'm imagining a rack of raspis acting as one giant ram swap drive over nbd. This could work for a given value of "work". cost of pi vs stick of ram. A kv storage as well perhaps. Then again, whats a TB worth on just one xeon server? Probably cheaper... or not?
  - jhartwig 4y ago
    Have you seen the cost of Pis lately :)
    - adamius 4y ago
      Didn't say much about availability either :P

staticassertion 4y ago

Since apparently no one is willing to read this excellent article, which even comes with fun sliders and charts... > It turns out that Chrome actively throttles requests, including those to cached resources, to reduce I/O contention. This generally improves performance, but will mean that pages with a large number of cached resources will see a slower retrieval time for each resource.

cogman10 4y ago
Seems like a shortcut that shouldn't be. I can understand throttling network requests, but disk requests? The only reason to do that would be for power savings (you don't want to push the CPU into a higher state as it loads up the data).
- sieabahlpark 4y ago
- staticassertion 4y ago
  I imagine it's just that the cache is in an awkward place where the throttling logic is unaware of the cache and the cache is not able to preempt the throttling logic.
  - cogman10 4y ago
    I'd guess the same thing is going on. Definitely FEELS like a code structure problem.
- fnordpiglet 4y ago
  Depends on if you value latency for the user. Saying I try both and choose the one coming first hurts no one but the servers not being protected by a client cache. But there’s absolutely no reason to believe a client has a cache that masks requests. AFAIK there’s no standard that says clients use caches for parsimony and not exclusively latency. As a matter of fact I think this is a good idea if it ever takes time to consult the cache, and the trade off is more bandwidth consumption which we are awash in. If you care that much run a caching proxy and use that and you’ll get the same effect of the client side cache masking requests. But I would say it’s superior because it always uses the local cache first and doesn’t waste user time on the edge condition in their cache coherency. It comes from Netscape which famously convinced everyone that it’s one of the hardest problems. That leads to the final benefit, the cache doesn’t have to cohere. If it’s too expensive at that moment to cohere and query then I can use the network source. Again the only downside is the network bandwidth is more consistently user. I would be hard pressed to believe most Firefox users already are grossly bandwidth over provisioned, and the amount of a fraction of a cable line a web browser loading from the cache no one could even notice that.

agumonkey 4y ago

What's the storage capacity of internet cables ?

moralestapia 4y ago
On the order of petabytes.
arccy 4y ago
https://github.com/yarrick/pingfs https://github.com/yarrick/pingfs
LeonenTheDK 4y ago
Explored here: http://tom7.org/harder/ http://tom7.org/harder/ There's a video and a paper linked on this page with the information on this absurdity.
- agumonkey 4y ago
  oh of course, how could I forget this :)

r1ch 4y ago

For me this is very noticeable whenever I open a new Chrome tab. It takes 3+ seconds for the icons of the recently visited sites to appear, whatever cache is used for the favicons is extremely slow. Thankfully the disk cache for other resources runs at a more normal speed.

gowld 4y ago

The article shows a lot of data about cache speed, but I don't see a comparison to cacheless network.

sattoshi 4y ago

> This seemed odd to me, surely using a cached response would always be faster than making the whole request again! Well it turns out that in some cases, the network is faster than the cache. Did I miss a follow up on this, or did it remain unanswered as to what the benefit of racing against the network is? The post basically says that sometimes the cache is slower because of throttling or bugs, but mostly bugs. Why is Firefox sending an extra request instead of figuring out what is slowing down the cache? It seems like an overly expensive mitigation…

8n4vidtmkvmk 4y ago
only reason i can think of is slow mechanical hard drive vs fast network. or if its based on etag and it has to hash the local resource, but it would be smart to cache that hash too. or maybe the os has too many file handles open and its doing a poor job of reading many files at once, whereas the network offers a 2nd tube to funnel data through...

philsnow 4y ago

> Concatenating / bundling your assets is probably still a good practice, even on H/2 connections. Obviously this comes on balance with cache eviction costs and splitting pre- and post-load bundles. I guess this latter part refers to the trade-off between compiling all your assets into a single file, and then requiring clients to re-download the entire bundle if you change a single CSS color. The other extreme is to not bundle anything (which, I gather from the article, is the standard practice since all major browsers support HTTP/2) but this leads to the described issue. What about aggressively bundling, but also keeping track at compile time of diffs between historical bundles and the new bundle? Re-connecting clients could grab a manifest that names the newest mega-bundle as well as a map from historical versions to the patches needed to bring them up to date. A lot more work on the server side but maybe it could be a good compromise? Of course that's the easy version, but it has a huge flaw which is all first-time clients have to download the entire huge mega bundle before the browser can render anything, so to make it workable it would have to compile things into a few bootstrap stages instead of a single mega-bundle. I am clearly not a frontend dev. If you're going to throw tomatoes please also tell me why ;) * edit: found the repo that made me think of this idea, https://github.com/msolo/msolo/blob/master/vfl/vfl.py https://github.com/msolo/msolo/blob/master/vfl/vfl.py but it's from antiquity and probably predates node and babel/webpack, but the idea is you name individual asset versions with a SHA or tag and let them be cached forever, and to update the web app you just change a reference in the root to make clients download a different resource dependency tree root (and they re-use unchanged ones) *

staticassertion 4y ago
There's probably some balance here. Since there's a limit of 9 concurrent requests before throttling occurs you can bucket 9 objects, concatenating into each. So if you have a bunch of static content, concat that into 1 bucket. If you have another object that changes a lot, keep that separate. If you have two other objects that change together, bucket those, etc. Seems like a huge pain to think about tbh. Seems like part of the problem would be solved by compiling everything into a single file that supported streaming execution.
philsnow 4y ago
Another tool that bears on this idea is courgette [0], in that it operates on one of the intermediate representations of the final bundle in order to achieve better compression. https://blog.chromium.org/2009/07/smaller-is-faster-and-safer-too.html https://blog.chromium.org/2009/07/smaller-is-faster-and-safe...

neomantra 4y ago

Different anecdote, but similar vibe.... In ~2010, I was benchmarking Solarflare (Xilinx/AMD now) cards and their OpenOnload kernel-bypass network stack. The results showed that two well-tuned systems could communicate faster (lower latency) than two CPU sockets within the same server that had to wait for the kernel to get involved (standard network stack). It was really illuminating and I started re-architecting based on that result. Backing out some of that history... in ~2008, we started using FPGAs to handle specific network loads (US equity market data). It was exotic and a lot of work, but it significantly benefited that use case, both because of DMA to user-land and its filtering capabilities. At that time our network was all 1 Gigabit. Soon thereafter, exchanges started offering 10G handoffs, so we upgraded our entire infrastructure to 10G cut-through switches (Arista) and 10G NICs (Myricom). This performed much better than the 1G FPGA and dramatically improved our entire infrastructure. We then ported our market data feed handler's to Myricom's user-space network stack, because loads were continually increasing and the trading world was continually more competitive... and again we had a narrow solution (this time in software) to a challenging problem. Then about a year later, Solarflare and it's kernel-compatible OpenOnload arrived and we could then apply the power of kernel bypass to our entire infrastructure. After that, the industry returned to FPGAs again with 10G PHY and tons of space to put whole strategies... although I was never involved with that next generation of trading tech. I personally stayed with OpenOnload for all sorts of workloads, growing to use it with containerization and web stacks (Redis, Nginx). Nowadays you can use OpenOnload with XDP; again a narrow technology grows to fit broad applicability.

oogali 4y ago
I went down a similar path (Solarflare, Myricom, and Chelsio; Arista, BNT, and 10G Twinax) and found we could get lower latency internally by confining two OpenOnload-enabled processes that needed to communicate with each other to the same NUMA domain. We architected our applications around that as well. The firm continued on to the FPGA path though that was after my time there. I do still pick up SolarFlare cards off of eBay for home lab purposes.
- neomantra 4y ago
  Nice! We deploy like that too. I document how to do that with Redis here [1]. [1] https://gist.github.com/neomantra/3c9b89887d19be6fa5708bf4017c0ecd https://gist.github.com/neomantra/3c9b89887d19be6fa5708bf401...

btdmaster 4y ago

The cache, for me, is 100:1 winning the races compared to the network. Are there greatly different results for others in about:networking#rcwn?

cyounkins 4y ago

Worth noting that around the end of 2020, Chrome and Firefox enabled cache partitioning by eTLD+1 to prevent scripts from gaining info from a shared cache. This kills the expected high hit rate from CDN-hosted libraries. https://developer.chrome.com/blog/http-cache-partitioning/ https://developer.chrome.com/blog/http-cache-partitioning/ https://developer.mozilla.org/en-US/docs/Web/Privacy/State_Partitioning https://developer.mozilla.org/en-US/docs/Web/Privacy/State_P...

Thorrez 4y ago
>partitioning by eTLD+1 They were already partitioned by the requested eTLD+1, because they were already partitioned by the requested URL, which contains the requested eTLD+1. Chrome's change was additionally partitioning it by 2 more eTLD+1s: the top-level eTLD+1 and the frame eTLD+1. It looks like Firefox's change was to add 1 eTLD+1 to the original behavior: the top-level eTLD+1.
mobilio 4y ago
First this was implemented in Safari years before 2020.

kreetx 4y ago

Much confusion in the comments. Tl;dr: cache is "slow" because the number of ongoing requests-- including to cache! --are throttled by the browser. I.e the cache isn't slow, but reading it is waited upon and the network request might be ahead in the queue.

dblohm7 4y ago

Firefox's HTTP cache races with the network for precisely this reason.

staticassertion 4y ago
That is what the first sentence of the article is about and it is the premise of the entire article?
- dblohm7 4y ago
  Ah, so it is! I got overexcited because I was one of the people who advocated for a racy cache during the redesign of that code back in the day.
capableweb 4y ago
Then that implementation doesn't even save bandwidth (for client nor server), which is one of the purposes of having a cache in the first place (together with other reasons).
- dblohm7 4y ago
  When the cache wins the network transfer is canceled, so it’s not as bad as you’re making it out to be.
edflsafoiewq 4y ago
Can I turn this off?

langsoul-com 4y ago

Never knew caches could take longer to retrieve. Always thought it'd be near instant. No good solutions either, weird throttling isn't feasible when creating Web apps.

acd 4y ago

Sun Microsystems stated: "The network is the computer" https://en.m.wikipedia.org/wiki/The_Network_is_the_Computer https://en.m.wikipedia.org/wiki/The_Network_is_the_Computer Sun was right.

kardos 4y ago

I wonder if browsers could design a heuristic to cache multiple items in one cache entry -- that is, instead of the website doing file concatenation, the browser dynamically decides to concatenate some resources in order to reduce the number of asks to the local cache. For example, the browser would know on the initial load which requests get made together (such as three js files or four images), so they could get batched into a single cache entry

jesprenj 4y ago

https://simonhearne.com/2020/network-faster-than-cache/#by-operating-system https://simonhearne.com/2020/network-faster-than-cache/#by-o... When I first saw this, I was confused why Linux (and Ubuntu and Debian) performed so poorly. But then I asked myself; is the data even representable? Or is Linux worse because users on Linux tend to stick to lower end hardware, since the measurenents were, AFAIR, taken for granted from website visitors instead of being tested on same hardware with different operating systems installed? I don't know, can someone explain the discrepancy? Is the raw data available? What is the number of requests that were included in the measurements for Debian (that doesn't hit cache in the first 5 ms)? Thanks!