4 ms·

Due to connection setup roundtrips, TCP slow start mechanics and quality of network connections it is usually better to terminate the client's TCP connection cl

Due to connection setup roundtrips, TCP slow start mechanics and quality of network connections it is usually better to terminate the client's TCP connection close to them. But I agree that moving the frontend rendering near the client doesn't really make sense for almost all cases. So the best general purpose setup would probably look like user → CDN → central PHP+DB. Of course there are always exceptions. Often stale data is ok and you can ship data snapshots to the edge so that they can be served without going back to the central DB. But in many cases some basic cache policies with a CDN can be nearly as effective.

TekMol 2y ago
What I usually do is that I use two hostnames. host1 for http requests which can be cached. That host is behind a CDN. host2 for HTTP requests which can't be cached. That host points directly to my server. Are you saying it would result in a better user experience when I only use host1 which is behind the CDN and add no-cache headers to the request that can't be cached?
- kevincox 2y ago
  It's complicated but typically yes. The simplest reason is that TCP+TLS handshakes require multiple round trips for a fresh connection. The CDN can maintain a persistent connection to the backend that is shared across users. It is also likely that the CDN to backend connection goes over a better connection than the user to backend connection would.
  - TekMol 2y ago
    Very interesting, thanks. That would make my setup even simpler.
  - e1g 2y ago
    > The CDN can maintain a persistent connection to the backend that is shared across users We considered using Cloudflare Workers as a reverse proxy, and I did extensive testing of this (very reasonable) assumption. Turns out that when calling back to the origin from the edge, CF Workers established a new connection almost every time, and so had to pay the penalty of the TCP and TLS handshake on every request. That killed any performance gains, and was a deal breaker for us. It’s rather difficult to predict or monitor network/routing behavior when running on the edge.
    - bluk 2y ago
      Were you using a Cloudflare tunnel for your origin?
      - e1g 2y ago
        Yes, tried tunnels too. There is significant variability among individual requests, but when benchmarking at scale I found no meaningful difference in p50 and p90 between “Worker -> CF Tunnel -> EC2 -> backend app” and “Worker -> AWS Global Accelerator -> EC2 -> backend app”
    - kentonv 2y ago
      This didn't sound right to me so I did some investigation and I think I found a bug. Keep in mind that Cloudflare is a complex stack of proxies. When a worker performs a fetch(), that request has to pass through a few machines on Cloudflare's network before it can actually go to origin. E.g. to implement caching we need to go to the appropriate cache machine, and then to try to reuse connections we need to go to the appropriate egress machine. Point is, the connection to origin isn't literally coming from the machine that called fetch(). So if you call fetch() twice in a row, to the same hostname, does it reuse a connection? If everything were on a single machine, you'd expect so, yes! But in this complex proxy stack, stuff has to happen correctly for those two requests to end up back on the same machine at the other end in order to use the same connection. Well, it looks like heuristics involved here aren't currently handling Workers requests the way they should. They are designed more around regular CDN requests (Workers shares the same egress path that regular non-Workers CDN requests use). In the standard CDN use case where you get a request from a user, possibly rewrite it in a Worker, then forward it to origin, you should be seeing connection reuse. But, it looks like if you have a Worker that performs multiple fetch() requests to origin (e.g. not forwarding the user's requests, but making some API requests or something)... we're not hashing things correctly so that those fetches land on the same egress machine. So... you won't get connection reuse, unless of course you have enough traffic to light up all the egress machines. I'm face-palming a bit here, and wondering why there hasn't been more noise about this. We'll fix it. Talk about low-hanging fruit... (I'm the tech lead for Cloudflare Workers.) (On a side note, enabling Argo Smart Routing will greatly increase the rate of connection reuse in general, even for traffic distributed around the world, as it causes requests to be routed within Cloudflare's network to the location closest to your origin. Also, even if the origin connections aren't reused, the RTT from Cloudflare to origin becomes much shorter, so connection setup becomes much less expensive. However, this is a paid feature.)
  - 1oooqooq 2y ago
    the "it's complicated" part assumes client side requests i guess. none of this comment applies to back end + db in the same server or colo.
- mschuster91 2y ago
  > Are you saying it would result in a better user experience when I only use host1 which is behind the CDN and add no-cache headers to the request that can't be cached? Yes, because that way you can leverage the CDN to defend against DDoS issues, and you can firewall the origin server itself so only the CDN is allowed to communicate with it, but no one else.
  - 1oooqooq 2y ago
    if your market is not usa or Asia, you actually get DDoSed by using cloudflare et all more often (>1) than not (=0)

TekMol 2y ago

What I usually do is that I use two hostnames. host1 for http requests which can be cached. That host is behind a CDN. host2 for HTTP requests which can't be cached. That host points directly to my server. Are you saying it would result in a better user experience when I only use host1 which is behind the CDN and add no-cache headers to the request that can't be cached?

kevincox 2y ago
It's complicated but typically yes. The simplest reason is that TCP+TLS handshakes require multiple round trips for a fresh connection. The CDN can maintain a persistent connection to the backend that is shared across users. It is also likely that the CDN to backend connection goes over a better connection than the user to backend connection would.
- TekMol 2y ago
  Very interesting, thanks. That would make my setup even simpler.
- e1g 2y ago
  > The CDN can maintain a persistent connection to the backend that is shared across users We considered using Cloudflare Workers as a reverse proxy, and I did extensive testing of this (very reasonable) assumption. Turns out that when calling back to the origin from the edge, CF Workers established a new connection almost every time, and so had to pay the penalty of the TCP and TLS handshake on every request. That killed any performance gains, and was a deal breaker for us. It’s rather difficult to predict or monitor network/routing behavior when running on the edge.
  - bluk 2y ago
    Were you using a Cloudflare tunnel for your origin?
    - e1g 2y ago
      Yes, tried tunnels too. There is significant variability among individual requests, but when benchmarking at scale I found no meaningful difference in p50 and p90 between “Worker -> CF Tunnel -> EC2 -> backend app” and “Worker -> AWS Global Accelerator -> EC2 -> backend app”
  - kentonv 2y ago
    This didn't sound right to me so I did some investigation and I think I found a bug. Keep in mind that Cloudflare is a complex stack of proxies. When a worker performs a fetch(), that request has to pass through a few machines on Cloudflare's network before it can actually go to origin. E.g. to implement caching we need to go to the appropriate cache machine, and then to try to reuse connections we need to go to the appropriate egress machine. Point is, the connection to origin isn't literally coming from the machine that called fetch(). So if you call fetch() twice in a row, to the same hostname, does it reuse a connection? If everything were on a single machine, you'd expect so, yes! But in this complex proxy stack, stuff has to happen correctly for those two requests to end up back on the same machine at the other end in order to use the same connection. Well, it looks like heuristics involved here aren't currently handling Workers requests the way they should. They are designed more around regular CDN requests (Workers shares the same egress path that regular non-Workers CDN requests use). In the standard CDN use case where you get a request from a user, possibly rewrite it in a Worker, then forward it to origin, you should be seeing connection reuse. But, it looks like if you have a Worker that performs multiple fetch() requests to origin (e.g. not forwarding the user's requests, but making some API requests or something)... we're not hashing things correctly so that those fetches land on the same egress machine. So... you won't get connection reuse, unless of course you have enough traffic to light up all the egress machines. I'm face-palming a bit here, and wondering why there hasn't been more noise about this. We'll fix it. Talk about low-hanging fruit... (I'm the tech lead for Cloudflare Workers.) (On a side note, enabling Argo Smart Routing will greatly increase the rate of connection reuse in general, even for traffic distributed around the world, as it causes requests to be routed within Cloudflare's network to the location closest to your origin. Also, even if the origin connections aren't reused, the RTT from Cloudflare to origin becomes much shorter, so connection setup becomes much less expensive. However, this is a paid feature.)
- 1oooqooq 2y ago
  the "it's complicated" part assumes client side requests i guess. none of this comment applies to back end + db in the same server or colo.
mschuster91 2y ago
> Are you saying it would result in a better user experience when I only use host1 which is behind the CDN and add no-cache headers to the request that can't be cached? Yes, because that way you can leverage the CDN to defend against DDoS issues, and you can firewall the origin server itself so only the CDN is allowed to communicate with it, but no one else.
- 1oooqooq 2y ago
  if your market is not usa or Asia, you actually get DDoSed by using cloudflare et all more often (>1) than not (=0)