4 ms·

Caching is largely a means to reach a goal of reduced bandwidth but even more so network latency. Almost everything else is fast and cheap, and network latency

by klabb3 3y ago

Caching is largely a means to reach a goal of reduced bandwidth but even more so network latency. Almost everything else is fast and cheap, and network latency can largely be simplified to number of round trips, similar to the N+1 problem for database queries. Reducing number of round trips is a better proxy for performance (although global CDNs can make some round trips faster than others). I don’t know what serving stacks kids use these days but judging from my web developer tools, it appears to be a hodgepodge of dynamically loaded crap and microservices (which adds also TLS handshakes to the mix) that of course fetch things in the least efficient ways possible. Caching (a) won’t even work for many of these cases and (b) wouldn’t help much even when it does work. I’d love to see a post about round trips exclusively and how to reduce it in the bloated stacks of today. I hope it’s not as bad as I think.

hannob 3y ago
Here's how to reduce roundtrips: * Support TLS 1.3 + HTTP/2 or HTTP/3. * Reduce the number of different hosts that you connect to. The optimal number is 1. That's it.
- supriyo-biswas 3y ago
  Additionally, use early hints to push resources to the client before your server responds with the rendered page.
  - ianlevesque 3y ago
    Wasn't that feature already removed?
    - supriyo-biswas 3y ago
      You’re referring to HTTP/2 push, early hints uses two header frames in HTTP/2 or QUIC, the first one is usually pushed by your CDN containing a list of resources used on the page, and the second one is pushed out by your application and proxied by the CDN.
- danielheath 3y ago
  Asset domains to avoid cookies being sent on every js/css/image request?
  - hannob 3y ago
    Is the idea here that saving the data from the cookies will speedup things, because requests become smaller? Or does it have some impact on caching or something else that I am not considering right now? I would expect that unless you have exceptionally large cookies, the saved roundtrips from another TLS handshake matter more than the data transmission for the cookies.
    - webignition 3y ago
      It's a security matter more than a performance matter, although improved performance is a nice side effect. For assets served from a third party (a CDN), you don't want to send cookies that might include secrets (a session cookie that could allow access to a user's account for example). You can trust that a third party won't intentionally log or make use of any sensitive information in cookies but you can't guarantee it. Best not to send it at all.
      - antihero 3y ago
        > assets served from a third party (a CDN) Won't that naturally be on a different domain anyway? How much performance hit do cookies really have in 2023?
      - hannob 3y ago
        I mean - if you separate your html from your assets security wise, that naturally means they need to be on different hosts, as you cannot really reroute requests before TLS decryption based on paths or any other indicator. But the motivation to put stuff on a CDN would be to improve performance. If you put your HTML on your own HW and your assets on a CDN for performance reasons, you might want to check if that really pans out, because those extra roundtrips may kill all performance savings you get from the CDN.
- klabb3 3y ago
  It’s a good start but no guarantee. If you have a sequential chain of requests, for whatever reason, the number of round trips is determined by business logic. If you have anything like say a microservice/API request preceding another, you’ll see more round trips. Given the leftpad-as-a-service trend we’re seeing today, I would expect that to be quite common.
andersrs 3y ago
The madness begun when SPAs (single page applications) became fashionable along with simple REST APIs. This made the client experience smoother but the network waterfall could get a bit long. For a shop you might have a workflow like [click product]->[get /product]->[get /product/reviews]. Then GraphQL came along to solve the N+1 issue as let you query everything in one network call... but only over HTTP POST so it broke caching on CDNs. Then edge workers/lambda came to solve that issue. You can also bring your data closer to the edge with replicated databases. Most newer stacks like Nextjs, Nuxt or SvelteKit behave like a traditional server side rendered page at first and then dynamically load content like an SPA once loaded which is the best of both worlds. They'll also be using HTTP/3 to force-feed a bunch of assets to the client before it even requests them. Ideally you'd have your data distributed around the world with GraphQL servers close to your users and use SSR to render the initial page markup.
- vmfunction 3y ago
  > but only over HTTP POST so it broke caching on CDNs. That's why when making request related to realtime data in the server, a POST is always needed! GET is ONLY get for static content.
  - jrochkind1 3y ago
    That is... not right. While POST is effectively never cached, GET isn't always either. Cache headers should be set properly on GET responses, indicating desired caching behavior (which is more than 'on' or 'off', as the OP gets into), on any content, static or not. The fundamental difference between GET and POST in http is idempotency. An non-cacheable response to an idempotent request is sometimes a (intentional, desirable) thing, which is why you can make GET responses non-cacheable if you want. Static content isn't the only thing you might want to be cached, there are all sorts of use cases for caching (which again can be configured in many ways, it's not just on or off) for non-static content.
    - SkyPuncher 3y ago
      While you’re right in theory, it doesn’t always work in practice. Some systems fail to respect caching headers. Further, in my experience, intermediate caches are mostly useless for non-binary product data. Either you need to make a round trip or you don’t. Sure, you can cache and return “not changed”, but you still get the latency. Just returning the data often isn’t much slower. POST avoids all of those issues by pretty much saying “give us what we neeed every time”

by klabb3 3y ago

hannob 3y ago

Here's how to reduce roundtrips: * Support TLS 1.3 + HTTP/2 or HTTP/3. * Reduce the number of different hosts that you connect to. The optimal number is 1. That's it.

supriyo-biswas 3y ago
Additionally, use early hints to push resources to the client before your server responds with the rendered page.
- ianlevesque 3y ago
  Wasn't that feature already removed?
  - supriyo-biswas 3y ago
    You’re referring to HTTP/2 push, early hints uses two header frames in HTTP/2 or QUIC, the first one is usually pushed by your CDN containing a list of resources used on the page, and the second one is pushed out by your application and proxied by the CDN.
danielheath 3y ago
Asset domains to avoid cookies being sent on every js/css/image request?
- hannob 3y ago
  Is the idea here that saving the data from the cookies will speedup things, because requests become smaller? Or does it have some impact on caching or something else that I am not considering right now? I would expect that unless you have exceptionally large cookies, the saved roundtrips from another TLS handshake matter more than the data transmission for the cookies.
  - webignition 3y ago
    It's a security matter more than a performance matter, although improved performance is a nice side effect. For assets served from a third party (a CDN), you don't want to send cookies that might include secrets (a session cookie that could allow access to a user's account for example). You can trust that a third party won't intentionally log or make use of any sensitive information in cookies but you can't guarantee it. Best not to send it at all.
    - antihero 3y ago
      > assets served from a third party (a CDN) Won't that naturally be on a different domain anyway? How much performance hit do cookies really have in 2023?
    - hannob 3y ago
      I mean - if you separate your html from your assets security wise, that naturally means they need to be on different hosts, as you cannot really reroute requests before TLS decryption based on paths or any other indicator. But the motivation to put stuff on a CDN would be to improve performance. If you put your HTML on your own HW and your assets on a CDN for performance reasons, you might want to check if that really pans out, because those extra roundtrips may kill all performance savings you get from the CDN.
klabb3 3y ago
It’s a good start but no guarantee. If you have a sequential chain of requests, for whatever reason, the number of round trips is determined by business logic. If you have anything like say a microservice/API request preceding another, you’ll see more round trips. Given the leftpad-as-a-service trend we’re seeing today, I would expect that to be quite common.

andersrs 3y ago

The madness begun when SPAs (single page applications) became fashionable along with simple REST APIs. This made the client experience smoother but the network waterfall could get a bit long. For a shop you might have a workflow like [click product]->[get /product]->[get /product/reviews]. Then GraphQL came along to solve the N+1 issue as let you query everything in one network call... but only over HTTP POST so it broke caching on CDNs. Then edge workers/lambda came to solve that issue. You can also bring your data closer to the edge with replicated databases. Most newer stacks like Nextjs, Nuxt or SvelteKit behave like a traditional server side rendered page at first and then dynamically load content like an SPA once loaded which is the best of both worlds. They'll also be using HTTP/3 to force-feed a bunch of assets to the client before it even requests them. Ideally you'd have your data distributed around the world with GraphQL servers close to your users and use SSR to render the initial page markup.

vmfunction 3y ago
> but only over HTTP POST so it broke caching on CDNs. That's why when making request related to realtime data in the server, a POST is always needed! GET is ONLY get for static content.
- jrochkind1 3y ago
  That is... not right. While POST is effectively never cached, GET isn't always either. Cache headers should be set properly on GET responses, indicating desired caching behavior (which is more than 'on' or 'off', as the OP gets into), on any content, static or not. The fundamental difference between GET and POST in http is idempotency. An non-cacheable response to an idempotent request is sometimes a (intentional, desirable) thing, which is why you can make GET responses non-cacheable if you want. Static content isn't the only thing you might want to be cached, there are all sorts of use cases for caching (which again can be configured in many ways, it's not just on or off) for non-static content.
  - SkyPuncher 3y ago
    While you’re right in theory, it doesn’t always work in practice. Some systems fail to respect caching headers. Further, in my experience, intermediate caches are mostly useless for non-binary product data. Either you need to make a round trip or you don’t. Sure, you can cache and return “not changed”, but you still get the latency. Just returning the data often isn’t much slower. POST avoids all of those issues by pretty much saying “give us what we neeed every time”