6 ms·

Let me try to understand this "Edge" thing: - The user sends an HTTP request to somesite.com - Their DNS query for somesite.com gets resolved to some d

by TekMol 2y ago

Let me try to understand this "Edge" thing: - The user sends an HTTP request to somesite.com - Their DNS query for somesite.com gets resolved to some datacenter near them - The HTTP request arrives at the datacenter where the PHP in WebAssembly is executed at half the speed of native PHP - The PHP in Webassembly sends database queries to a central DB server over the internet - The PHP in Webassembly templates the data and sends it back to the user How is that faster than resolving somesite.com to the central server, sending the HTTP query to the central server where PHP runs at full speed and talks to the DB on the same machine (or over the LAN)? Even if PHP ran at full speed at the "Edge", won't the two requests over the internet user --1--> edge PHP server --2--> central Db server take longer than the one http request when the user connects to the central server directly? user --1--> central PHP+DB server In reality, the PHP script on the "Edge" server probably makes not one but multiple queries to the DB server. Won't that make it super slow compared to having it all happening on one machine or one LAN?

afavour 2y ago
The post mentions a cache. I think the key here would be not going to the central DB and instead going to a distributed cache. Not an unreasonable concept when I assume the vast majority will be read operations.
- Y-bar 2y ago
  That's OPcache they mentioned, a specific language-level cache native to PHP. https://www.php.net/manual/en/intro.opcache.php https://www.php.net/manual/en/intro.opcache.php
  - afavour 2y ago
    Ah. Point still stands though, distributed storage is what really unlocks this concept. Wordpress is a pretty bad example though, surely you’d just CDN cache the pages!
    - iruoy 2y ago
      opcache doessn't cache user data. it's for caching bytecode in memory as long as the php script/server is running i'd like to see a comparison against a firecracker vm and a docker container
      - fhuici 2y ago
        Hi, on kraft.cloud we use FC, along with a custom controller and very specialized VMs (unikernels) to have extremely efficient deployments (eg, millisecond cold starts). For a PHP web server, for instance, we can cold start things in about 30ms (https://docs.kraft.cloud/guides/php/ https://docs.kraft.cloud/guides/php/). It's also possible to run wasm workloads/blobs (e.g., https://docs.kraft.cloud/guides/wazero/ https://docs.kraft.cloud/guides/wazero/). The builds are based on Dockerfiles, but for deployment we transparently convert that to unikernels.
kevincox 2y ago
Due to connection setup roundtrips, TCP slow start mechanics and quality of network connections it is usually better to terminate the client's TCP connection close to them. But I agree that moving the frontend rendering near the client doesn't really make sense for almost all cases. So the best general purpose setup would probably look like user → CDN → central PHP+DB. Of course there are always exceptions. Often stale data is ok and you can ship data snapshots to the edge so that they can be served without going back to the central DB. But in many cases some basic cache policies with a CDN can be nearly as effective.
- TekMol 2y ago
  What I usually do is that I use two hostnames. host1 for http requests which can be cached. That host is behind a CDN. host2 for HTTP requests which can't be cached. That host points directly to my server. Are you saying it would result in a better user experience when I only use host1 which is behind the CDN and add no-cache headers to the request that can't be cached?
  - kevincox 2y ago
    It's complicated but typically yes. The simplest reason is that TCP+TLS handshakes require multiple round trips for a fresh connection. The CDN can maintain a persistent connection to the backend that is shared across users. It is also likely that the CDN to backend connection goes over a better connection than the user to backend connection would.
    - TekMol 2y ago
      Very interesting, thanks. That would make my setup even simpler.
    - e1g 2y ago
      > The CDN can maintain a persistent connection to the backend that is shared across users We considered using Cloudflare Workers as a reverse proxy, and I did extensive testing of this (very reasonable) assumption. Turns out that when calling back to the origin from the edge, CF Workers established a new connection almost every time, and so had to pay the penalty of the TCP and TLS handshake on every request. That killed any performance gains, and was a deal breaker for us. It’s rather difficult to predict or monitor network/routing behavior when running on the edge.
meekaaku 2y ago
Isnt this true of everything that runs on the edge? Unless you host the database in the same network as your edge provider, appliation <-> database is where the bottleneck is.
the_duke 2y ago
That's a valid concern, and with a classical centralized DB the Edge solution will not be faster in many cases, especially considering how many DB requests some ORMs like Drupal or Wordpress need for each page load... Some other providers have tried to solve the problem with replicated databases that are used for local reads. I can't go into details yet, but Wasmer will offer a solution for this problem quite soon as well, which hopefully will be much more seamless than existing strategies. (note: I work at Wasmer)
- lovasoa 2y ago
  https://electric-sql.com/ https://electric-sql.com/ ?
deleted 2y ago
[deleted]
slaymaker1907 2y ago
You're assuming that network latencies follow the triangle inequality, i.e., that A->C is smaller than A->B + B->C. However, that breaks down because of money. It's possible that the user's ISP is a cheapskate without good peering agreements such that while A->B is fast, they haven't setup the necessary agreements to make A->C fast. I've personally seen this with a game recently and my own ISP. I can ping their West Coast servers in 20-40ms while pinging their Chicago servers takes 200ms+. This is actually one huge flaw with the current way IP is done. Because your ISP is closely tied with the last-mile of routing, you have very little control over optimizing those backend layers of routing. This doesn't explain why you wouldn't just run a thin proxy at the edge and instead want to run a full PHP app. For that, my guess is this makes sense when you don't want to run a full server all the time or if you want it to be rapidly scalable.
- paulddraper 2y ago
  Okay....but that's you providing a backbone for the user. Don't you just use a CDN at that point? (One that allows non-cached requests as well.)
eknkc 2y ago
I guess if you have a use case with higly cacheable data on edge, it might work great. For example, a form builder like typeform could cache form definitions on edge and render them close to users. Submissions would require db communication but the entire experience could be better. Otherwise it is just not worth it. BTW, I don’t even think PHP running slower in wasm would be that important. These things generally depend on IO performance rather than runtime if you are not doing a lot of calculations on the edge. WASM is also pretty fast these days so..
- 1oooqooq 2y ago
  > WASM is also pretty fast these days so.. ironically everything is petty fast and everyone waste that speed adding things that allow for a cheaper labour. wasm included. it went from a "have some expertly optimized code in the browser" to "just compile this hide cpp thing and ship as a react component" to then "it allows our servers to run any crap without knowledgeable staff in each language" we added so many Layers to cheapen labour that now all the labour costs goes into managing the Layers.
  - eknkc 2y ago
    Ok hear this, I was toying with zig last week and compiled a quickjs wrapper that can run React and compiled that to wasm. So I have a JS runtime running wasm which runs quickjs runtime via zig and that runs React. Just trying to learn zig though but not much different than running PHP in WASM haha. Layers and layers of shit pressed together.
    - 1oooqooq 2y ago
      will definitely use it as the wrapper that launches my electron app
zackify 2y ago
That’s why you gotta use liteFS on fly.io or cloudflare d1. And of course, don’t use php
worewood 2y ago
I'm gonna counter the other comments that try to justify this somehow, and say: I agree with you, and it's not faster. With the risk of sounding old (get off my lawn): I assure you the megabytes of unneeded Javascript they're downloading to the client are ten times slower to execute than the milliseconds they may be saving on the wire.
- 1oooqooq 2y ago
  yes you are correct. the top comments disagreeing are either full of wrong basic concepts or are very dense in detail while hand waving the important parts and from CDN-for-hire employers.
shanemhansen 2y ago
I'm working on something in a similar space. It's not as mysterious as people make it sound: You do caching so you don't always have to go back to the DB is the main thing. The other interesting capability is light scripting at the edge. Think "this page is cached but I'm going to add in a custom username". There's some more exotic stuff around sharded database locality. But I'll skip over that because distributed mutable state is hard and its not my specialty. Connection warming and shorter tls negotiation round trip times are also a thing but probably less important compared to the database thing you are mentioning. But mostly this architecture makes sense if you can do some caching or if you're doing more data/log collection stuff. For prior work see ESI (edge side includes). Assembling stuff at the edge does have some benefits but it has been hard to create a good interface for that.
Delmololo 2y ago
Big providers have stable and fast and encrypted connection channels between edge and backend. Client talks to closest system, TLS gets terminated as soon as possible. Certain things can be cached on edge which normally can't be cached just through basic http (like as soon as you are logged in).

by TekMol 2y ago

afavour 2y ago

The post mentions a cache. I think the key here would be not going to the central DB and instead going to a distributed cache. Not an unreasonable concept when I assume the vast majority will be read operations.

Y-bar 2y ago
That's OPcache they mentioned, a specific language-level cache native to PHP. https://www.php.net/manual/en/intro.opcache.php https://www.php.net/manual/en/intro.opcache.php
- afavour 2y ago
  Ah. Point still stands though, distributed storage is what really unlocks this concept. Wordpress is a pretty bad example though, surely you’d just CDN cache the pages!
  - iruoy 2y ago
    opcache doessn't cache user data. it's for caching bytecode in memory as long as the php script/server is running i'd like to see a comparison against a firecracker vm and a docker container
    - fhuici 2y ago
      Hi, on kraft.cloud we use FC, along with a custom controller and very specialized VMs (unikernels) to have extremely efficient deployments (eg, millisecond cold starts). For a PHP web server, for instance, we can cold start things in about 30ms (https://docs.kraft.cloud/guides/php/ https://docs.kraft.cloud/guides/php/). It's also possible to run wasm workloads/blobs (e.g., https://docs.kraft.cloud/guides/wazero/ https://docs.kraft.cloud/guides/wazero/). The builds are based on Dockerfiles, but for deployment we transparently convert that to unikernels.

kevincox 2y ago

Due to connection setup roundtrips, TCP slow start mechanics and quality of network connections it is usually better to terminate the client's TCP connection close to them. But I agree that moving the frontend rendering near the client doesn't really make sense for almost all cases. So the best general purpose setup would probably look like user → CDN → central PHP+DB. Of course there are always exceptions. Often stale data is ok and you can ship data snapshots to the edge so that they can be served without going back to the central DB. But in many cases some basic cache policies with a CDN can be nearly as effective.

TekMol 2y ago
What I usually do is that I use two hostnames. host1 for http requests which can be cached. That host is behind a CDN. host2 for HTTP requests which can't be cached. That host points directly to my server. Are you saying it would result in a better user experience when I only use host1 which is behind the CDN and add no-cache headers to the request that can't be cached?
- kevincox 2y ago
  It's complicated but typically yes. The simplest reason is that TCP+TLS handshakes require multiple round trips for a fresh connection. The CDN can maintain a persistent connection to the backend that is shared across users. It is also likely that the CDN to backend connection goes over a better connection than the user to backend connection would.
  - TekMol 2y ago
    Very interesting, thanks. That would make my setup even simpler.
  - e1g 2y ago
    > The CDN can maintain a persistent connection to the backend that is shared across users We considered using Cloudflare Workers as a reverse proxy, and I did extensive testing of this (very reasonable) assumption. Turns out that when calling back to the origin from the edge, CF Workers established a new connection almost every time, and so had to pay the penalty of the TCP and TLS handshake on every request. That killed any performance gains, and was a deal breaker for us. It’s rather difficult to predict or monitor network/routing behavior when running on the edge.

meekaaku 2y ago

Isnt this true of everything that runs on the edge? Unless you host the database in the same network as your edge provider, appliation <-> database is where the bottleneck is.

the_duke 2y ago

That's a valid concern, and with a classical centralized DB the Edge solution will not be faster in many cases, especially considering how many DB requests some ORMs like Drupal or Wordpress need for each page load... Some other providers have tried to solve the problem with replicated databases that are used for local reads. I can't go into details yet, but Wasmer will offer a solution for this problem quite soon as well, which hopefully will be much more seamless than existing strategies. (note: I work at Wasmer)

lovasoa 2y ago
https://electric-sql.com/ https://electric-sql.com/ ?

deleted 2y ago

[deleted]

slaymaker1907 2y ago

You're assuming that network latencies follow the triangle inequality, i.e., that A->C is smaller than A->B + B->C. However, that breaks down because of money. It's possible that the user's ISP is a cheapskate without good peering agreements such that while A->B is fast, they haven't setup the necessary agreements to make A->C fast. I've personally seen this with a game recently and my own ISP. I can ping their West Coast servers in 20-40ms while pinging their Chicago servers takes 200ms+. This is actually one huge flaw with the current way IP is done. Because your ISP is closely tied with the last-mile of routing, you have very little control over optimizing those backend layers of routing. This doesn't explain why you wouldn't just run a thin proxy at the edge and instead want to run a full PHP app. For that, my guess is this makes sense when you don't want to run a full server all the time or if you want it to be rapidly scalable.

paulddraper 2y ago
Okay....but that's you providing a backbone for the user. Don't you just use a CDN at that point? (One that allows non-cached requests as well.)

eknkc 2y ago

I guess if you have a use case with higly cacheable data on edge, it might work great. For example, a form builder like typeform could cache form definitions on edge and render them close to users. Submissions would require db communication but the entire experience could be better. Otherwise it is just not worth it. BTW, I don’t even think PHP running slower in wasm would be that important. These things generally depend on IO performance rather than runtime if you are not doing a lot of calculations on the edge. WASM is also pretty fast these days so..

1oooqooq 2y ago
> WASM is also pretty fast these days so.. ironically everything is petty fast and everyone waste that speed adding things that allow for a cheaper labour. wasm included. it went from a "have some expertly optimized code in the browser" to "just compile this hide cpp thing and ship as a react component" to then "it allows our servers to run any crap without knowledgeable staff in each language" we added so many Layers to cheapen labour that now all the labour costs goes into managing the Layers.
- eknkc 2y ago
  Ok hear this, I was toying with zig last week and compiled a quickjs wrapper that can run React and compiled that to wasm. So I have a JS runtime running wasm which runs quickjs runtime via zig and that runs React. Just trying to learn zig though but not much different than running PHP in WASM haha. Layers and layers of shit pressed together.
  - 1oooqooq 2y ago
    will definitely use it as the wrapper that launches my electron app

zackify 2y ago

That’s why you gotta use liteFS on fly.io or cloudflare d1. And of course, don’t use php

worewood 2y ago

I'm gonna counter the other comments that try to justify this somehow, and say: I agree with you, and it's not faster. With the risk of sounding old (get off my lawn): I assure you the megabytes of unneeded Javascript they're downloading to the client are ten times slower to execute than the milliseconds they may be saving on the wire.

1oooqooq 2y ago
yes you are correct. the top comments disagreeing are either full of wrong basic concepts or are very dense in detail while hand waving the important parts and from CDN-for-hire employers.

shanemhansen 2y ago

I'm working on something in a similar space. It's not as mysterious as people make it sound: You do caching so you don't always have to go back to the DB is the main thing. The other interesting capability is light scripting at the edge. Think "this page is cached but I'm going to add in a custom username". There's some more exotic stuff around sharded database locality. But I'll skip over that because distributed mutable state is hard and its not my specialty. Connection warming and shorter tls negotiation round trip times are also a thing but probably less important compared to the database thing you are mentioning. But mostly this architecture makes sense if you can do some caching or if you're doing more data/log collection stuff. For prior work see ESI (edge side includes). Assembling stuff at the edge does have some benefits but it has been hard to create a good interface for that.

Delmololo 2y ago

Big providers have stable and fast and encrypted connection channels between edge and backend. Client talks to closest system, TLS gets terminated as soon as possible. Certain things can be cached on edge which normally can't be cached just through basic http (like as soon as you are logged in).