6 ms·

Instead of "edge", a lot of websites should just have 3 locations (us,eu,apac) with a non geo replicated Serverless database in each region. At least that's wha

Instead of "edge", a lot of websites should just have 3 locations (us,eu,apac) with a non geo replicated Serverless database in each region. At least that's what we're building at WunderGraph (https://wundergraph.com/ https://wundergraph.com/). Edge sounds super cool, but if you take state and consistency into consideration, you just can't have servers across the globe that also replicate their state consistently with low latency. TTFB doesn't matter as much as correctness. And if stale content is acceptable, then we can also just push it to a CDN. Most importantly, you'd want to have low latency between server and storage. So if your servers are on the "edge", they are close to the user, but (randomly) further away from the database. Durable objcets might solve this, but they are nowhere near a postgres database. I think the "edge" is good for some stateless use cases, like validating auth and inputs, etc., but it won't make "boring" services, even serverless in "non-edge" Locations obsolete. You can see this on Vercel. Serverless for functions, server side rendering, etc. and cloudflare workers for edge middleware. But they explicitly say that your serverless functions should be close to a database if you're using one.

jstummbillig 4y ago
> But they explicitly say that your serverless functions should be close to a database if you're using one. Or else?
- tmikaeld 4y ago
  You'll get the latencies noted for Heroku in the article.
- jensneuse 4y ago
  Let's say the user is in FRA, the database in SF. If the "server" is on the edge, you'll end up with 100-200ms between server and database, while the user has less than 10ms latency to the "edge". If the server does multiple round-trips to the database, it can take seconds until the first byte. If the server and database are both in SF, TTFB will probably be less than one second, as round trips between database and server are almost zero. One thing to mention is that it would be beneficial of the TLS handshake could be made on the edge, as it's a multi roundtrip transaction. Ideally, we could combine a server close to the DB with a stateless edge service. So, the future of the web might not be on the edge. It's rather: The future if the web will leverage the edge.
lewisl9029 4y ago
The architecture I eventually ended up with for my product (https://reflame.app https://reflame.app) involves: 1. A strongly consistent globally replicated DB for most data that needs fast reads (<100ms) but not necessarily fast writes (>200ms). I've been using Fauna, but there are other options too such as CockroachDB and Spanner, and more in the works. 2. An eventually consistent globally replicated DB for the subset of data that does also need fast writes. I eventually settled on Dynamo for this, but there are even more options here. I think for all but the most latency-sensitive products, 1. will be all they need. IMHO the strongly consistently replicated database is a strictly superior product compared to databases that are single-region by default and only support replication through read-replicas. In a read-replica system, we have to account for stale reads due to replication delays, and redirect writes to the primary, resulting in inconsistent latencies across regions. This is an extremely expensive complexity tax that will significantly increase the cognitive load on every engineer, lead to a ton of bugs around stale reads, and cause edge case handling code to seep into every corner of our codebase. Strongly consistently replicated databases on the other hand, offer the exact same mental model as a database that lives in a single region with a single source of truth, while offering consistent, fast, up-to-date reads everywhere, at the cost of consistently slower writes everywhere. I actually consider the consistently slower writes also a benefit since it doesn't allow us to fool ourselves into thinking our app is fast for everybody, when it's only fast for us because we placed the primary db right next to us, and forces us to actually solve for the higher write latency using other technologies if our use case truly requires it (see 2.). In the super long term, I don't think the future is on what's currently referred to as "the edge", as this "edge" doesn't extend nearly far enough. The true edge is client devices: reading from and writing to client devices is the only way to truly eliminate speed-of-light induced latency. For a long time, most truly client-first apps have been relegated to single-user experiences due to how most popular client-first architectures have not had an answer for collaboration and authorization, but with this new wave of client-first architectures solving for collaboration and authorization with client-side reads and optimistic client-side writes with server-side validation (see Replicache), I've never been more optimistic about the future (an open source alternative to Replicache would do wonders to accelerate us to this future. clientdb looks promising).
- 4y ago
naiv 4y ago
I would even say that for 99% of the existing websites a free oracle cloud instance with 4 cores and 24GB ram + cdn is more than enough.
- jensneuse 4y ago
  I agree. It's probably good enough for a lot of use cases.
- jl6 4y ago
  Agree, except that amount of RAM ain’t free.
  - yusefnapora 4y ago
    The free x86 VMs are capped at 1GB each, but the ARM ones go up to 24GB. > Arm-based Ampere A1 cores and 24 GB of memory usable as 1 VM or up to 4 VMs with 3,000 OCPU hours and 18,000 GB hours per month https://www.oracle.com/cloud/free/#always-free https://www.oracle.com/cloud/free/#always-free
- qudat 4y ago
  We use the cloud free tier for https://prose.sh https://prose.sh We aren’t even close to hitting any bottlenecks at this point
- systemvoltage 4y ago
  Here is my golden setup: Cloudflare Tunnels + All ports closed (except ssh) + bare metal server. You can scale to the moon with like a million active users on a couple of 16 core servers (1 primary, 1 hot failover). You don't need AWS. You don't need Terraform. You don't need kubernetes. Hell, you don't need docker because you know apriori what the deployment environment is. Just run systemd services. 99% of the startsup will never need anything more. They'll fail before that. The ones that succeed have a good problem on hand to actually Scale™. What we're seeing is premature-optimi...errr scaling. Edit more context: For Postgres, setup streaming replication between postgres and hot standby. You need a remote server somewhere to check health of your primary and run promote to your hot standby if it fails. It is not that difficult. Have cron jobs to back up your database with pgdumpall in addition somewhere on Backblaze or S3. Use your hot standby to run Grafana/Prometheus/Loki stack. For extra safety, run both servers on ZFS raid (mirror or raidz2) on nvme drives. You'll get like 100k IOPS which would be 300x of base RDS instance on AWS. Ridiculous savings and performance would be just astonishing. Run your app to call postgres on localhost, it will be the fastest web experience your customers will ever experience, on edge or not.

jstummbillig 4y ago

> But they explicitly say that your serverless functions should be close to a database if you're using one. Or else?

tmikaeld 4y ago
You'll get the latencies noted for Heroku in the article.
jensneuse 4y ago
Let's say the user is in FRA, the database in SF. If the "server" is on the edge, you'll end up with 100-200ms between server and database, while the user has less than 10ms latency to the "edge". If the server does multiple round-trips to the database, it can take seconds until the first byte. If the server and database are both in SF, TTFB will probably be less than one second, as round trips between database and server are almost zero. One thing to mention is that it would be beneficial of the TLS handshake could be made on the edge, as it's a multi roundtrip transaction. Ideally, we could combine a server close to the DB with a stateless edge service. So, the future of the web might not be on the edge. It's rather: The future if the web will leverage the edge.

lewisl9029 4y ago

The architecture I eventually ended up with for my product (https://reflame.app https://reflame.app) involves: 1. A strongly consistent globally replicated DB for most data that needs fast reads (<100ms) but not necessarily fast writes (>200ms). I've been using Fauna, but there are other options too such as CockroachDB and Spanner, and more in the works. 2. An eventually consistent globally replicated DB for the subset of data that does also need fast writes. I eventually settled on Dynamo for this, but there are even more options here. I think for all but the most latency-sensitive products, 1. will be all they need. IMHO the strongly consistently replicated database is a strictly superior product compared to databases that are single-region by default and only support replication through read-replicas. In a read-replica system, we have to account for stale reads due to replication delays, and redirect writes to the primary, resulting in inconsistent latencies across regions. This is an extremely expensive complexity tax that will significantly increase the cognitive load on every engineer, lead to a ton of bugs around stale reads, and cause edge case handling code to seep into every corner of our codebase. Strongly consistently replicated databases on the other hand, offer the exact same mental model as a database that lives in a single region with a single source of truth, while offering consistent, fast, up-to-date reads everywhere, at the cost of consistently slower writes everywhere. I actually consider the consistently slower writes also a benefit since it doesn't allow us to fool ourselves into thinking our app is fast for everybody, when it's only fast for us because we placed the primary db right next to us, and forces us to actually solve for the higher write latency using other technologies if our use case truly requires it (see 2.). In the super long term, I don't think the future is on what's currently referred to as "the edge", as this "edge" doesn't extend nearly far enough. The true edge is client devices: reading from and writing to client devices is the only way to truly eliminate speed-of-light induced latency. For a long time, most truly client-first apps have been relegated to single-user experiences due to how most popular client-first architectures have not had an answer for collaboration and authorization, but with this new wave of client-first architectures solving for collaboration and authorization with client-side reads and optimistic client-side writes with server-side validation (see Replicache), I've never been more optimistic about the future (an open source alternative to Replicache would do wonders to accelerate us to this future. clientdb looks promising).

4y ago

naiv 4y ago

I would even say that for 99% of the existing websites a free oracle cloud instance with 4 cores and 24GB ram + cdn is more than enough.

jensneuse 4y ago
I agree. It's probably good enough for a lot of use cases.
jl6 4y ago
Agree, except that amount of RAM ain’t free.
- yusefnapora 4y ago
  The free x86 VMs are capped at 1GB each, but the ARM ones go up to 24GB. > Arm-based Ampere A1 cores and 24 GB of memory usable as 1 VM or up to 4 VMs with 3,000 OCPU hours and 18,000 GB hours per month https://www.oracle.com/cloud/free/#always-free https://www.oracle.com/cloud/free/#always-free
qudat 4y ago
We use the cloud free tier for https://prose.sh https://prose.sh We aren’t even close to hitting any bottlenecks at this point
systemvoltage 4y ago
Here is my golden setup: Cloudflare Tunnels + All ports closed (except ssh) + bare metal server. You can scale to the moon with like a million active users on a couple of 16 core servers (1 primary, 1 hot failover). You don't need AWS. You don't need Terraform. You don't need kubernetes. Hell, you don't need docker because you know apriori what the deployment environment is. Just run systemd services. 99% of the startsup will never need anything more. They'll fail before that. The ones that succeed have a good problem on hand to actually Scale™. What we're seeing is premature-optimi...errr scaling. Edit more context: For Postgres, setup streaming replication between postgres and hot standby. You need a remote server somewhere to check health of your primary and run promote to your hot standby if it fails. It is not that difficult. Have cron jobs to back up your database with pgdumpall in addition somewhere on Backblaze or S3. Use your hot standby to run Grafana/Prometheus/Loki stack. For extra safety, run both servers on ZFS raid (mirror or raidz2) on nvme drives. You'll get like 100k IOPS which would be 300x of base RDS instance on AWS. Ridiculous savings and performance would be just astonishing. Run your app to call postgres on localhost, it will be the fastest web experience your customers will ever experience, on edge or not.