11 ms·

So, you want to deploy on the edge?

251 points by zknill 3y ago

FooBarWidget 3y ago
Perhaps it's better to launch multiple, region-specific versions of a site, with region-specific data. Something like amazon.com vs amazon.nl vs amazon.de. Account data would still be global but that doesn't change much so you can get away with strong consistency. Added benefit is that it clarifies how to do local compliance. With a global model, the complexity of local laws can really overwhelming. For example a global site based in the US needs to be GDPR-compliant, while a global site based in the EU has to figure out how to file multiple forms of taxes (federal, state, sales, city, district) for the 16000+ US tax jurisdictions. As a European I am more afraid of US taxes than GDPR.
- robertlagrant 3y ago
  > federal, state, sales, city, district Really? I've never really been to the US, other than a couple of brief work trips, and that is astonishing.
  - dsr_ 3y ago
    District is unusual. There's no federal sales tax, but if you're selling internationally, there can be border fees of various kinds. State sales taxes are nearly ubiquitous. Large cities or counties might have a sales-tax addition for local funding. Special-purpose sales taxes exist, infrequently: hotels are probably most common.
    - gumby 3y ago
      Sure, there are also excise taxes levied on different kinds of products (fuel and alcohol being the most common), fees (like disposal deposits on drink containers, tires, and the like) and just in general differential tax rates (certain kinds of foods are not taxed, taxed at a different rate or even, I’ve seen proposed, taxed by the size of container).
    - dclowd9901 3y ago
      Aren't import taxes essentially federal sales taxes? Granted, they're applied long before the product sits in a consumer's cart.
  - badcppdev 3y ago
    Really! For some locations you need a street address to be sure you're collecting the correct Sales Tax for that particular type of product. Also there are Sales Tax holidays where the rate of sales tax is reduced for a set period: https://www.salestaxinstitute.com/resources/sales-tax-holidays https://www.salestaxinstitute.com/resources/sales-tax-holida... Also certain customers are exempt from Sales Tax...
- supriyo-biswas 3y ago
  There are merchants of record which will resell your product, and take care of levying and remitting taxes, such as paddle.com for SaaS software.
beck5 3y ago
At last, an examination of the true nature of 'edge' computing is presented. Despite the appealing promises made by posts from Fly.io and others that depict 'edge' computing as a simple success, the reality can be more complex. I have recently spent a fair bit of time experimenting with this on Fly for my application (https://www.ssrfproxy.com https://www.ssrfproxy.com). It's hard to beat the straightforwardness of deploying in a single region, with the database in close proximity. This approach probably to meets the needs of what 99% of developers require. Aka Heroku.
- __alexs 3y ago
  My suspicion has always been that edge computing was primarily offered by CDNs as away to increase utilisation of hardware rather than something there was significant customer demand for.
  - benbjohnson 3y ago
    LiteFS author here. I think single-node deployments are great and it's a model that fits many (or even most) apps out there. However, the two most common feature requests when I was developing Litestream were HA & replication. While most apps won't need those features, it's good to have it available as an option down the road. I've seen many devs choose Postgres or MySQL over SQLite simply because they didn't have these features. Once those are solved, edge computing is really more of a nice side effect. I think it's also useful to consider that tools like LiteFS aren't just for your main database. We have several internal tools at Fly.io that fan out their data to other internal apps to consume so they can share data without having to build an API or worry about rate limiting.
    - leafmeal 3y ago
      HA?
      - benbjohnson 3y ago
        Sorry, I should have been more clear. HA = High Availability. Essentially, folks wanted the ability to automatically failover to a secondary node if the current primary failed.
arrty88 3y ago
What about local read only replicas of the db in each region, and one primary in your primary region? Or a write through cache in each region.
- zknill 3y ago
  This design is really similar to the approach taken by litefs[1] and turso[2]. Writes are forwarded to the primary (like you would with a write-through cache) and a local copy of the data is held in each region. These local copies are only guaranteed to be up-to-date with the most recent write forwarded by that local copy. [1] https://fly.io/docs/litefs/proxy/ https://fly.io/docs/litefs/proxy/ [2] https://docs.turso.tech/reference/data-consistency#on-the-replicas https://docs.turso.tech/reference/data-consistency#on-the-re...
- moffkalast 3y ago
  Or just having a completely separate database for each region, much like the average MMO does it.
- cj 3y ago
  This is relatively easy to orchestrate with MongoDB replica sets (and I'm sure with other databases, too). In a typical MongoDB deployment you want at least 3 nodes (1 primary, 2 or more secondaries). You can put one of the secondaries on the other side of the world. By default, all reads are sent to the primary. But you can easily specify "nearest" as your "read preference" on a query-by-query basis which would allow the app to read from the nreplica with the lowest latency (whether it be the primary or one of the secondaries). The ability to specify read preference at the query level lets you have 2 categories of queries, 1) queries that must always be served with most recent writes, sent to primary at the expense , and 2) queries that are allowed to occasionally send stale data but benefit from latency improvements by reading from the nearest node.
- donmcronald 3y ago
  > Or a write through cache in each region. This is something I want to play around with using Cloudflare. I wonder if there’s enough of a guarantee for a user to “stick” to a node that you could use Workers KV as an eventually consistent write through cache for user unique data and a Durable Object as a strongly consistent write through cache. I feel like that would give a pretty solid balance, wouldn’t it?
solatic 3y ago
> Most internet apps are read-heavy, there are more reads than writes, so largely it makes sense to deal with the latency on writes. We can do this by forwarding the writes to some leader for that piece of data (e.g. the log for usernames), and waiting until that write is replicated back to us (so we can read our own writes). This is precisely the point. For most CRUD applications, writes are usually so sporadic and user-intentioned that you can show a spinning wheel for it, and users will forgive you for that slowness because it's so sporadic. Edge computing is basically reinventing insecure HTTP caches and mirrors, but allowing you to include authn/authz so that user data stays private to them. If your edge data model is that much more complicated than that, you're probably doing it wrong.
- MuffinFlavored 3y ago
  I had heard authn/authz before, never realized/read into what they were: > How are authn and authz different? To put it simply, authn has to do with identity, or who someone is, while authz has to do with permissions, or what someone is allowed to do.
  - rockostrich 3y ago
    IMO it's a case where shortening the words is pretty detrimental because the point of authentication and authorization are already in the words and makes sense. Plus both words end with "n" so "authn" doesn't make sense at first glance.
  - mcv 3y ago
    Just to be clear, these two are short for authentication and authorization, right? I guess it's awkward that those two related but different concepts share the same first 4 letters. But I think authc would have been clearer for disambiguation than authn.
    - deleted 3y ago
      [deleted]
    - DylanSp 3y ago
supriyo-biswas 3y ago
Most applications tend to benefit from serving an Early Hints[1] from a CDN, rather than actual edge computing. Early hints takes care of the delay in response time by ensuring that the page, when loaded, will tend to have all the resources it needs to load instantly. [1] https://httpwg.org/specs/rfc8297.html https://httpwg.org/specs/rfc8297.html
l5870uoo9y 3y ago
The best use case for edge deployment is the deployment of static resources for the initial web app load. It achieves a latency of around 40ms and almost instantly shows the user the initial render. The later database CRUD actions are much less important when the application has already loaded. Not only because much of data can be cached in the browser or aggressively pre-fetched, but also because many CRUD actions run in the background only show errors if something went wrong.
- zeroCalories 3y ago
  I think this depends on the application. For a complex spa, sure. But for something like a news site or social media the extra load time feels horrible. It's preferable to use server side rendering as much as possible. That also means you gotta speed up your database queries, which is where techniques like in the OP come into play.
  - tracker1 3y ago
    Even for mostly static, edge can add a cache layer, and also do doughnut rendering for sites... By doughnut rendering, I mean you can displace certain static placeholders with user-level content... such as replacing the "login" section of the header with the markup for the User's name and profile image. These are both common use cases for Cloudflare Workers. Not just for complex compute but for near-line cache and simple transforms.
ds0 3y ago
I extend my prayers out to those of you who have pointy-haired bosses looking to seem trendy in their public reports by implementing this.
nik736 3y ago
A CDN makes a lot of sense to me, having static assets that get cached close to users is a brilliant way to speed up requests. As the article points out, this is a very hard problem to solve for anything that touches a database or is not static/cached content. This is also why I don't believe in the idea that Fly.io pitches. Additionally you have the legal things to take care of with GDPR, etc. so a local database is probably the way to go for most projects.
fauigerzigerk 3y ago
>If a user makes a request from Europe, and the apps run in US East, that adds an extra 100-150ms of latency just by round-tripping across the Atlantic. These numbers seem high. This site shows transatlantic RTT of around 75ms: https://wondernetwork.com/pings https://wondernetwork.com/pings If a CDN or edge computing platform reduces that to 20ms then the difference is 55ms. And it matters only for read requests that cannot be cached locally. Whether or not its worth it depends largely on the number of requests. Perhaps reducing that number should be prioritised.
- evilantnie 3y ago
  You've linked to ping/ICMP statistics, TCP is a more common use case and thus tends to be more representative of real-world applications. 100ms is a fairly realistic 90th percentile in my experience, 150ms could be in the 95th.
  - fauigerzigerk 3y ago
    The extra time it takes to process a TCP request compared to ICMP traffic is the same regardless of whether or not you're using an edge computing platform. It seems to me that what actually matters is the difference in latency, not the absolute numbers. That's why I think the article is incorrect when it claims that running an app in US East "adds an extra 100-150ms of latency". Not all of this is "extra". Only about 55ms are.
- roncesvalles 3y ago
  If your application isn't real-time (like online gaming or multi-way voice/video), and you aren't making a bunch of requests serially from the client (very rare), I'm convinced you can serve the whole world with at most 2 datacenters - e.g. one in NYC and another in Singapore - coupled with a CDN to serve static assets. The notion that you need to replicate your infra in 10 regions in order to provide the best user experience sounds like propaganda pushed by cloud companies to make you use more product than you need.
  - tptacek 3y ago
gumby 3y ago
This kind of load distribution is important but please, let’s not redefine the term “edge”. TCP/IP was specifically designed with the devices at the edge to function as peers. Computing on the edge means, say, doing the computing in your phone or laptop or iot frob. This is longstanding use and to pollute a useful technical term just leads to unnecessary confusion.
- fellowmartian 3y ago
  I think that ship has sailed. “Edge computing” is a well established phrase at this point. When I need to refer to the scenario you’re describing I usually say “local computation”.
  - flyingcircus3 3y ago
    If that's were the case, why would the entire first page of Google search results for "edge computing" refer to IOT, and processing data where it's created?
    - fellowmartian 3y ago
      You’re right of course, IoT is edge computing because it’s treated as company’s edge, while we can’t say the same for end users’ phones and computers.
      - flyingcircus3 3y ago
        But that's only because there aren't any compelling IOT applications in the consumer space. Companies are the users with distributed assets, not consumers.
- flyingcircus3 3y ago
  Just wait until we're running transformers on the edge, using DSL and stacks.
  - 3y ago
nop_slide 3y ago
Great intro summary to this topic! I remember these distributed db scenarios being describe when reading "Designing Data Intensive Applications" and this was a really helpful supplementary piece to jog that memory. Cheers
deleted 3y ago
[deleted]
Wissmueller 3y ago
Great post highlighting important considerations before you make the decision to go on edge! If you do though, you should take a look at https://edgenode.com https://edgenode.com, which I helped build (we host server-less containers on edge)
daxfohl 3y ago
This isn't really edge. It's multi-region. It's a great intro to multi-region considerations, but it's not edge. Edge implies putting your stuff in a CDN-sized datacenter that has only a subset of the full regional services, may not be able to scale up significantly on demand, may be more expensive, have less local storage and failover redundancy, etc. The multi-region considerations come in here too, but there's a whole extra set of things to worry about. Basically you rarely want to deploy your whole app to edge; have a central or multi-region thing under the hood, and let the edge handle some very specific functionality. And even then, only if a few ms of latency makes a big difference to your clients (and they can deal with the lack of failover redundancy).
- roncesvalles 3y ago
  The meaning of "edge" keeps changing but currently it just means when your CDN provider offers compute services.
  - michaelt 3y ago
    To CDNs marketing their compute-in-CDN products, sure. But to people selling "Edge AI" products, an edge device is something like a smart speaker or industrial camera.
    - Ekaros 3y ago
      With industrial camera it is probably the device where multiple cameras connect to. With industrial solutions you often want a gateway that is edge device between the internal and external network. At least that is my current IIoT definition. You could have bunch of pretty dump sensors and then somewhat beefy these days gateway. And the gateway here could do some computation.
- steve1977 3y ago
  Edge IMHO implies being (very) close to the customer. As in, for example, IoT or running a service in an Internet Point of Presence maybe. But once your "in the cloud", you're not really on the Edge anymore. But the term seems to get used for everything and anything apparently.
Aeolun 3y ago
No, I do not want to deploy on the edge, and I wish people would stop making noise over it because it’s getting really had to convice buzzword happy business people that there’s nothing wrong with running a monolith on a single big server.
samwillis 3y ago
The real "edge" is the users device, I'm 100% sold on the concept of "local first". There is a great community building around this: https://localfirstweb.dev/ https://localfirstweb.dev/ The point is, by the time you have solved all the problems with consultancy and sync to an edge node, you are most of the way to local first.
- pphysch 3y ago
  AKA heavy client
- DylanSp 3y ago
  I'm glad there's a community working on this; I read the Ink & Switch article a while back, the local-first paradigm is very appealing.
- re-thc 3y ago
  > The real "edge" is the users device Isn't that the "client"?
- dclowd9901 3y ago
  I'm really wanting to investigate what our SaaS app looks like being fully offline, with sync on connection.
bradenb 3y ago
My takeaway from this is that a lot of people disagree on what "edge" is. IMO, "edge" is the furthest resource that you have some computational level of control over. Could be a data center, could be a phone, could be an IoT device, could be a missile. EDIT: I think I'm realizing people will disagree with me because I have a different perspective. For my use cases, my data comes from sensors on the edge, and so for me, I want my edge computing to be as close to those sensors as possible.
- weird-eye-issue 3y ago
  That's a terrible "definition" of edge
  - bradenb 3y ago
    I'd love to hear a better one. Care to comment?
    - weird-eye-issue 3y ago
      To me edge is when you are doing most computation and data retrieval required for the given request in a data center close to each user. How to define close? Probably when you have at least several dozen POPs spread out across the world
- hahn-kev 3y ago
  It's actually really helpful. Looking at a lot of js frameworks I thought I understood what they meant, but now I understand that the term is actually pretty ambiguous.
- paxys 3y ago
  Not quite. Nearly every service out there has computational control over the end client (whether a mobile app, browser JS etc.), but very few are focused on edge compute at that level. It is more helpful to think of it in terms of a traditional client-server architecture, where you want to move the server as close to the client as possible. This covers 95% of what people mean when they say edge compute.
DylanSp 3y ago
Definitely some good points here. Using a single primary database seems easier for a lot of more straightforward use cases, and read replicas are probably sufficient for read-heavy workloads that can tolerate some loss of consistency. I think either the Cloudflare Workers team or the Fly folks have talked about how they envision edge apps as sharding data by region or by tenant, then sticking each shard in the optimal location. That sounds potentially useful for some cases, but then there's the complexity of sharding to deal with.
- tptacek 3y ago
  We host applications that do sharding like this, but when we build our own stuff at Fly.io, or publish tooling like LiteFS, we're generally not thinking about sharding right now. We like things that are easy to keep our heads around. We're a hosting provider, not really a database provider. Systems that do sharding, Raft consistency, and stuff like that are built on top of systems like ours, not so much inside of them. :)
  - DylanSp 3y ago
    Gotcha, that makes sense; I'm probably thinking of Cloudflare Workers, then. It might've come up from kentonv's comments in the recent post about Cap'n Proto 1.0.
    - kentonv 3y ago
      Hmm, not in that thread, but maybe in a thread about D1 or Durable Objects? We are indeed trying to make it as easy as possible for applications to shard databases by user or by "document".
- tracker1 3y ago
  There's various options... my current direction is CockroachLabs for the DB side... but Cloudflare offers D1 (sqlite interface over durable workers/kv) which is kind of what you are talking about in practice. Actual data sharding, without persistence in different regions is more difficult. Different database systems lend themselves differently for different loads and sharding characteristics... I've seen MS-SQL in master-master across the globe, postgresql with sharding that is replicated to base host, and various ScyllaDB/Cassandra setups. It's always a mixed bag.
dclowd9901 3y ago
Nitpicky terminology choices aside, this is a very good, easy-to-follow writeup on multi-region data system design. Understanding that the trick is shuffling around the latency makes it much easier to reason about where you want to focus your latency savings. One thing I wasn't quite sure on, though, is why Database A would care if Database B had a read of a certain kind (vs. a write). Reads don't change state, so if a million reads occur with no writes, the data should still be consistent, no?
- zknill 3y ago
  If you're talking about the case where a local database forwards a read to the 'leader', and that read is put on the leader's log, but that read is only served when the read is replicated in the log back to the original local database then: It's actually less about the the leader caring about the read, and more about creating a point in time that the read will happen. The database is using the read in the log as the point-in-time where that read happened. The writes before that point will be viewed by the read, and the writes after won't be. By sending the read to the leader, when we receive that read back, we know for sure that all the writes that should have been replicated to our local copy have been replicated (because our read has made the full round-trip that all our writes would have, so we're guaranteed to have received all the writes before that read's "point-in-time").
  - dclowd9901 3y ago
    Oh, the author! Great work on the article, and thanks for the answer!
PaulHoule 3y ago
In 2006 or so I was working for an agency that was angling for a contract for the City of Ithaca to develop a system for building inspectors to use cell-phone connected PDAs to do their paperwork on the go. At the time I advocated for an “always connected” model but we were concerned that cellular connections wouldn’t be that reliable so we thought disconnected operation would be necessary. A few year back I was thinking about the future of “low code” and thought an open-source project similar to Lotus Notes but JSON-centric would be a good thing https://en.wikipedia.org/wiki/HCL_Domino https://en.wikipedia.org/wiki/HCL_Domino In particular, Lotus notes has a model for disconnected operation that works quite well. My take is that “progressive web apps” haven’t gotten that far because they don’t have enough batteries included, a really good bidirectional sync model would make a difference. For years you could say ideas from Lotus Notes didn’t make it into the open source world because they were patented but the patents have long since expired and the main reason they are obscure is ignorance.
- tracker1 3y ago
  Might be interested in PouchDB/CouchDB which can be offline and resync relatively easily. Slightly different approach than other SQL based RDBMS though. In the end, it gets complicated to support offline web-apps and really depends on your usage. I had to work on a water dispensing system over a decade ago that needed to be able to work "offline" at a given site... I used firebird embedded locally, and a remotely available server for syncing against. All accounts and balances were synced to each site, so that they could operate in offline for dispensing water to carrier trucks. There were rules on offline term (up to a week) and balance carry to reduce the risk of a negative balance. In the end it worked well, but it wasn't at all simple. Active/Inactive and syncing transactions was at least interesting, and that was for a relatively simple data model. More complex models would be, of course, more complicated... and almost exponentially so.
rando101101101 3y ago
I get the premise, but something must be said for caching cutting down on those DB transactions. Do the math for YOUR app and see what makes sense.
mystic-door29 3y ago
Okay
PeterCorless 3y ago
I've seen people debating what an 'edge' even means, and wanted to chime in. Note that 'edge' is poorly defined in the industry right now. Consider the following • Remote Region Datacenter — It's certainly not this • Local Region Datacenter — It's not this • Local Municipality/County Datacenter/Co-lo/Central Office — Could it be this? • Local mini-datacenters attached to mobile towers or fiber digital loop carriers? — Could it also be this? • On-prem Datacenter — It's not this either A lot of people argue that an 'edge' has to be in the field — like a local mobile tower or a digital loop carrier. I'd argue that it could also be at a nearby co-lo facility or CO. Basically anything ≤300 km should be ≤1 ms latency. Within 75-100 km you're talking 0.25-0.33 ms latencies. And for most applications these days that seems "real-time" enough. YMMV.
LAC-Tech 3y ago
"If a user makes a request from Europe, and the apps run in US East, that adds an extra 100-150ms of latency just by round-tripping across the Atlantic." laughs in Rural New Zealand But seriously really enjoyed this article. I was always a bit confused how the recent influx of edge services handled writes/consistency, and yeah they all seem to have a single leader that routes writes to them - the edge is all about fast reading.
AtNightWeCode 3y ago
Edge is just where something is hosted. It has nothing to do with data consistency. User signup is a unique problem that any large site has. Some sites like HN also actually needs usernames which makes things more complicated. Owasp always recommend to add usernames to any login. It is such a naive approach. User names are garbage in most cases.
abhinav-t 3y ago
Without getting into a discussion on what is the correct definition of "edge", I think the article offers 2 solutions to this problem: you have an internet app and you need to provide your users with low-latency reads while maintaining uniqueness constraint and read-your-writes consistency. The article favours dealing with latency during writes while making some assumptions, - most internet apps are read-heavy (reasonable, but what if it's not the case for you?) - you do not need sharding, all regions have all the data (what if you have a ton of data that is growing quickly? what if you have durations where some data is heavily contended for?) - the primary replica used for writes is highly available (how do you automate handling of the primary replica failure or primary replica getting partitioned?) Another approach to consider would be Amazon Dynamo style databases (Cassandra, Riak etc.) which can help when the above assumptions are not met, - Dynamo was designed to be always-writable and suitable for heavy writes and reads as well (e.g. the paper mentions teams using it for catalog data with R=1 and W=N) - data is partitioned/sharded using a variant of consistent hashing, which helps with large data volumes and limiting the impact of hot spots as well - failure detection and leader election help automate handling the partitions for which a failed node was holding a primary replica
opportune 3y ago
I think the footnote is pretty much the solution.. imagine you implement replication within a DC. Yes you risk data loss and maybe inconsistency (if you fail over to a new region and start doing writes without a way to fix this later) when the DC becomes unavailable but you can now offer strong consistency without expensive cross DC hops. Pretty sure this is how Google Cloud Spanner is implemented: you have a single region for writes and everywhere else is just a read partition? And it’s probably nothing fancy to set the read partition, just pick the first hop at the time of creation Basically: implementing a distributed global db without structuring the topology to minimize hops/making engineering trade offs can’t be fast. It’s all about deciding which part of the CAP theorem you want to relax.
skybrian 3y ago
Leaving aside what "edge" really means, for me personally, this is about connecting a website running on Deno Deploy with a database. As a hobbyist doing recreational programming, I don't want to pay for a website when it gets no traffic. Everything needs to start up on demand and shut down when idle. This means that cold start time is what really matters. Since Deno Deploy has no persistent storage yet, both the frontend and the database need to start up on first request. Having a multi-region database doesn't help if needs to be running all the time and doesn't start up very fast. As for latency when warmed up, I plan to deal with that by avoiding round trips with stored procedures and caching. Here's my "hello world" website. Any advice on technologies to check out would be welcome. https://postgres-counter-demo.deno.dev/ https://postgres-counter-demo.deno.dev/
shrubble 3y ago
One definition of edge I found was '5ms average latency to the customer' , does that seem like a good definition?
kaycey2022 3y ago
Sorry for the unrelated question, but can someone tell me how this blog is built? What tech stack is used here?
byndlimitsfy 3y ago
I had no idea edge was so varied. Isn't Edge primarily involved in IOT?
mhart 3y ago
Another pattern here is to have your edge provider automatically move your compute closer to your backend. Cloudflare released Smart Placement a few months ago to do this exact thing (with surprisingly similar diagrams!) https://blog.cloudflare.com/announcing-workers-smart-placement/ https://blog.cloudflare.com/announcing-workers-smart-placeme... (disclaimer: I was the tech lead on this feature)

251 points by zknill 3y ago

Perhaps it's better to launch multiple, region-specific versions of a site, with region-specific data. Something like amazon.com vs amazon.nl vs amazon.de. Account data would still be global but that doesn't change much so you can get away with strong consistency. Added benefit is that it clarifies how to do local compliance. With a global model, the complexity of local laws can really overwhelming. For example a global site based in the US needs to be GDPR-compliant, while a global site based in the EU has to figure out how to file multiple forms of taxes (federal, state, sales, city, district) for the 16000+ US tax jurisdictions. As a European I am more afraid of US taxes than GDPR.

robertlagrant 3y ago
> federal, state, sales, city, district Really? I've never really been to the US, other than a couple of brief work trips, and that is astonishing.
- dsr_ 3y ago
  District is unusual. There's no federal sales tax, but if you're selling internationally, there can be border fees of various kinds. State sales taxes are nearly ubiquitous. Large cities or counties might have a sales-tax addition for local funding. Special-purpose sales taxes exist, infrequently: hotels are probably most common.
  - gumby 3y ago
    Sure, there are also excise taxes levied on different kinds of products (fuel and alcohol being the most common), fees (like disposal deposits on drink containers, tires, and the like) and just in general differential tax rates (certain kinds of foods are not taxed, taxed at a different rate or even, I’ve seen proposed, taxed by the size of container).
  - dclowd9901 3y ago
    Aren't import taxes essentially federal sales taxes? Granted, they're applied long before the product sits in a consumer's cart.
- badcppdev 3y ago
  Really! For some locations you need a street address to be sure you're collecting the correct Sales Tax for that particular type of product. Also there are Sales Tax holidays where the rate of sales tax is reduced for a set period: https://www.salestaxinstitute.com/resources/sales-tax-holidays https://www.salestaxinstitute.com/resources/sales-tax-holida... Also certain customers are exempt from Sales Tax...
supriyo-biswas 3y ago
There are merchants of record which will resell your product, and take care of levying and remitting taxes, such as paddle.com for SaaS software.

beck5 3y ago

At last, an examination of the true nature of 'edge' computing is presented. Despite the appealing promises made by posts from Fly.io and others that depict 'edge' computing as a simple success, the reality can be more complex. I have recently spent a fair bit of time experimenting with this on Fly for my application (https://www.ssrfproxy.com https://www.ssrfproxy.com). It's hard to beat the straightforwardness of deploying in a single region, with the database in close proximity. This approach probably to meets the needs of what 99% of developers require. Aka Heroku.

__alexs 3y ago
My suspicion has always been that edge computing was primarily offered by CDNs as away to increase utilisation of hardware rather than something there was significant customer demand for.
- benbjohnson 3y ago
  LiteFS author here. I think single-node deployments are great and it's a model that fits many (or even most) apps out there. However, the two most common feature requests when I was developing Litestream were HA & replication. While most apps won't need those features, it's good to have it available as an option down the road. I've seen many devs choose Postgres or MySQL over SQLite simply because they didn't have these features. Once those are solved, edge computing is really more of a nice side effect. I think it's also useful to consider that tools like LiteFS aren't just for your main database. We have several internal tools at Fly.io that fan out their data to other internal apps to consume so they can share data without having to build an API or worry about rate limiting.
  - leafmeal 3y ago
    HA?
    - benbjohnson 3y ago
      Sorry, I should have been more clear. HA = High Availability. Essentially, folks wanted the ability to automatically failover to a secondary node if the current primary failed.

arrty88 3y ago

What about local read only replicas of the db in each region, and one primary in your primary region? Or a write through cache in each region.

zknill 3y ago
This design is really similar to the approach taken by litefs[1] and turso[2]. Writes are forwarded to the primary (like you would with a write-through cache) and a local copy of the data is held in each region. These local copies are only guaranteed to be up-to-date with the most recent write forwarded by that local copy. [1] https://fly.io/docs/litefs/proxy/ https://fly.io/docs/litefs/proxy/ [2] https://docs.turso.tech/reference/data-consistency#on-the-replicas https://docs.turso.tech/reference/data-consistency#on-the-re...
moffkalast 3y ago
Or just having a completely separate database for each region, much like the average MMO does it.
cj 3y ago
This is relatively easy to orchestrate with MongoDB replica sets (and I'm sure with other databases, too). In a typical MongoDB deployment you want at least 3 nodes (1 primary, 2 or more secondaries). You can put one of the secondaries on the other side of the world. By default, all reads are sent to the primary. But you can easily specify "nearest" as your "read preference" on a query-by-query basis which would allow the app to read from the nreplica with the lowest latency (whether it be the primary or one of the secondaries). The ability to specify read preference at the query level lets you have 2 categories of queries, 1) queries that must always be served with most recent writes, sent to primary at the expense , and 2) queries that are allowed to occasionally send stale data but benefit from latency improvements by reading from the nearest node.
donmcronald 3y ago
> Or a write through cache in each region. This is something I want to play around with using Cloudflare. I wonder if there’s enough of a guarantee for a user to “stick” to a node that you could use Workers KV as an eventually consistent write through cache for user unique data and a Durable Object as a strongly consistent write through cache. I feel like that would give a pretty solid balance, wouldn’t it?

solatic 3y ago

> Most internet apps are read-heavy, there are more reads than writes, so largely it makes sense to deal with the latency on writes. We can do this by forwarding the writes to some leader for that piece of data (e.g. the log for usernames), and waiting until that write is replicated back to us (so we can read our own writes). This is precisely the point. For most CRUD applications, writes are usually so sporadic and user-intentioned that you can show a spinning wheel for it, and users will forgive you for that slowness because it's so sporadic. Edge computing is basically reinventing insecure HTTP caches and mirrors, but allowing you to include authn/authz so that user data stays private to them. If your edge data model is that much more complicated than that, you're probably doing it wrong.

MuffinFlavored 3y ago
I had heard authn/authz before, never realized/read into what they were: > How are authn and authz different? To put it simply, authn has to do with identity, or who someone is, while authz has to do with permissions, or what someone is allowed to do.
- rockostrich 3y ago
  IMO it's a case where shortening the words is pretty detrimental because the point of authentication and authorization are already in the words and makes sense. Plus both words end with "n" so "authn" doesn't make sense at first glance.
- mcv 3y ago
  Just to be clear, these two are short for authentication and authorization, right? I guess it's awkward that those two related but different concepts share the same first 4 letters. But I think authc would have been clearer for disambiguation than authn.
  - deleted 3y ago
    [deleted]
  - DylanSp 3y ago

supriyo-biswas 3y ago

Most applications tend to benefit from serving an Early Hints[1] from a CDN, rather than actual edge computing. Early hints takes care of the delay in response time by ensuring that the page, when loaded, will tend to have all the resources it needs to load instantly. [1] https://httpwg.org/specs/rfc8297.html https://httpwg.org/specs/rfc8297.html

l5870uoo9y 3y ago

The best use case for edge deployment is the deployment of static resources for the initial web app load. It achieves a latency of around 40ms and almost instantly shows the user the initial render. The later database CRUD actions are much less important when the application has already loaded. Not only because much of data can be cached in the browser or aggressively pre-fetched, but also because many CRUD actions run in the background only show errors if something went wrong.

zeroCalories 3y ago
I think this depends on the application. For a complex spa, sure. But for something like a news site or social media the extra load time feels horrible. It's preferable to use server side rendering as much as possible. That also means you gotta speed up your database queries, which is where techniques like in the OP come into play.
- tracker1 3y ago
  Even for mostly static, edge can add a cache layer, and also do doughnut rendering for sites... By doughnut rendering, I mean you can displace certain static placeholders with user-level content... such as replacing the "login" section of the header with the markup for the User's name and profile image. These are both common use cases for Cloudflare Workers. Not just for complex compute but for near-line cache and simple transforms.

ds0 3y ago

I extend my prayers out to those of you who have pointy-haired bosses looking to seem trendy in their public reports by implementing this.

nik736 3y ago

A CDN makes a lot of sense to me, having static assets that get cached close to users is a brilliant way to speed up requests. As the article points out, this is a very hard problem to solve for anything that touches a database or is not static/cached content. This is also why I don't believe in the idea that Fly.io pitches. Additionally you have the legal things to take care of with GDPR, etc. so a local database is probably the way to go for most projects.

fauigerzigerk 3y ago

>If a user makes a request from Europe, and the apps run in US East, that adds an extra 100-150ms of latency just by round-tripping across the Atlantic. These numbers seem high. This site shows transatlantic RTT of around 75ms: https://wondernetwork.com/pings https://wondernetwork.com/pings If a CDN or edge computing platform reduces that to 20ms then the difference is 55ms. And it matters only for read requests that cannot be cached locally. Whether or not its worth it depends largely on the number of requests. Perhaps reducing that number should be prioritised.

evilantnie 3y ago
You've linked to ping/ICMP statistics, TCP is a more common use case and thus tends to be more representative of real-world applications. 100ms is a fairly realistic 90th percentile in my experience, 150ms could be in the 95th.
- fauigerzigerk 3y ago
  The extra time it takes to process a TCP request compared to ICMP traffic is the same regardless of whether or not you're using an edge computing platform. It seems to me that what actually matters is the difference in latency, not the absolute numbers. That's why I think the article is incorrect when it claims that running an app in US East "adds an extra 100-150ms of latency". Not all of this is "extra". Only about 55ms are.
roncesvalles 3y ago
If your application isn't real-time (like online gaming or multi-way voice/video), and you aren't making a bunch of requests serially from the client (very rare), I'm convinced you can serve the whole world with at most 2 datacenters - e.g. one in NYC and another in Singapore - coupled with a CDN to serve static assets. The notion that you need to replicate your infra in 10 regions in order to provide the best user experience sounds like propaganda pushed by cloud companies to make you use more product than you need.
- tptacek 3y ago

gumby 3y ago

This kind of load distribution is important but please, let’s not redefine the term “edge”. TCP/IP was specifically designed with the devices at the edge to function as peers. Computing on the edge means, say, doing the computing in your phone or laptop or iot frob. This is longstanding use and to pollute a useful technical term just leads to unnecessary confusion.

fellowmartian 3y ago
I think that ship has sailed. “Edge computing” is a well established phrase at this point. When I need to refer to the scenario you’re describing I usually say “local computation”.
- flyingcircus3 3y ago
  If that's were the case, why would the entire first page of Google search results for "edge computing" refer to IOT, and processing data where it's created?
  - fellowmartian 3y ago
    You’re right of course, IoT is edge computing because it’s treated as company’s edge, while we can’t say the same for end users’ phones and computers.
    - flyingcircus3 3y ago
      But that's only because there aren't any compelling IOT applications in the consumer space. Companies are the users with distributed assets, not consumers.
flyingcircus3 3y ago
Just wait until we're running transformers on the edge, using DSL and stacks.
- 3y ago

nop_slide 3y ago

Great intro summary to this topic! I remember these distributed db scenarios being describe when reading "Designing Data Intensive Applications" and this was a really helpful supplementary piece to jog that memory. Cheers

deleted 3y ago

[deleted]

Wissmueller 3y ago

Great post highlighting important considerations before you make the decision to go on edge! If you do though, you should take a look at https://edgenode.com https://edgenode.com, which I helped build (we host server-less containers on edge)

daxfohl 3y ago

This isn't really edge. It's multi-region. It's a great intro to multi-region considerations, but it's not edge. Edge implies putting your stuff in a CDN-sized datacenter that has only a subset of the full regional services, may not be able to scale up significantly on demand, may be more expensive, have less local storage and failover redundancy, etc. The multi-region considerations come in here too, but there's a whole extra set of things to worry about. Basically you rarely want to deploy your whole app to edge; have a central or multi-region thing under the hood, and let the edge handle some very specific functionality. And even then, only if a few ms of latency makes a big difference to your clients (and they can deal with the lack of failover redundancy).

roncesvalles 3y ago
The meaning of "edge" keeps changing but currently it just means when your CDN provider offers compute services.
- michaelt 3y ago
  To CDNs marketing their compute-in-CDN products, sure. But to people selling "Edge AI" products, an edge device is something like a smart speaker or industrial camera.
  - Ekaros 3y ago
    With industrial camera it is probably the device where multiple cameras connect to. With industrial solutions you often want a gateway that is edge device between the internal and external network. At least that is my current IIoT definition. You could have bunch of pretty dump sensors and then somewhat beefy these days gateway. And the gateway here could do some computation.
steve1977 3y ago
Edge IMHO implies being (very) close to the customer. As in, for example, IoT or running a service in an Internet Point of Presence maybe. But once your "in the cloud", you're not really on the Edge anymore. But the term seems to get used for everything and anything apparently.

Aeolun 3y ago

No, I do not want to deploy on the edge, and I wish people would stop making noise over it because it’s getting really had to convice buzzword happy business people that there’s nothing wrong with running a monolith on a single big server.

samwillis 3y ago

The real "edge" is the users device, I'm 100% sold on the concept of "local first". There is a great community building around this: https://localfirstweb.dev/ https://localfirstweb.dev/ The point is, by the time you have solved all the problems with consultancy and sync to an edge node, you are most of the way to local first.

pphysch 3y ago
AKA heavy client
DylanSp 3y ago
I'm glad there's a community working on this; I read the Ink & Switch article a while back, the local-first paradigm is very appealing.
re-thc 3y ago
> The real "edge" is the users device Isn't that the "client"?
dclowd9901 3y ago
I'm really wanting to investigate what our SaaS app looks like being fully offline, with sync on connection.

bradenb 3y ago

My takeaway from this is that a lot of people disagree on what "edge" is. IMO, "edge" is the furthest resource that you have some computational level of control over. Could be a data center, could be a phone, could be an IoT device, could be a missile. EDIT: I think I'm realizing people will disagree with me because I have a different perspective. For my use cases, my data comes from sensors on the edge, and so for me, I want my edge computing to be as close to those sensors as possible.

weird-eye-issue 3y ago
That's a terrible "definition" of edge
- bradenb 3y ago
  I'd love to hear a better one. Care to comment?
  - weird-eye-issue 3y ago
    To me edge is when you are doing most computation and data retrieval required for the given request in a data center close to each user. How to define close? Probably when you have at least several dozen POPs spread out across the world
hahn-kev 3y ago
It's actually really helpful. Looking at a lot of js frameworks I thought I understood what they meant, but now I understand that the term is actually pretty ambiguous.
paxys 3y ago
Not quite. Nearly every service out there has computational control over the end client (whether a mobile app, browser JS etc.), but very few are focused on edge compute at that level. It is more helpful to think of it in terms of a traditional client-server architecture, where you want to move the server as close to the client as possible. This covers 95% of what people mean when they say edge compute.

DylanSp 3y ago

Definitely some good points here. Using a single primary database seems easier for a lot of more straightforward use cases, and read replicas are probably sufficient for read-heavy workloads that can tolerate some loss of consistency. I think either the Cloudflare Workers team or the Fly folks have talked about how they envision edge apps as sharding data by region or by tenant, then sticking each shard in the optimal location. That sounds potentially useful for some cases, but then there's the complexity of sharding to deal with.

tptacek 3y ago
We host applications that do sharding like this, but when we build our own stuff at Fly.io, or publish tooling like LiteFS, we're generally not thinking about sharding right now. We like things that are easy to keep our heads around. We're a hosting provider, not really a database provider. Systems that do sharding, Raft consistency, and stuff like that are built on top of systems like ours, not so much inside of them. :)
- DylanSp 3y ago
  Gotcha, that makes sense; I'm probably thinking of Cloudflare Workers, then. It might've come up from kentonv's comments in the recent post about Cap'n Proto 1.0.
  - kentonv 3y ago
    Hmm, not in that thread, but maybe in a thread about D1 or Durable Objects? We are indeed trying to make it as easy as possible for applications to shard databases by user or by "document".
tracker1 3y ago
There's various options... my current direction is CockroachLabs for the DB side... but Cloudflare offers D1 (sqlite interface over durable workers/kv) which is kind of what you are talking about in practice. Actual data sharding, without persistence in different regions is more difficult. Different database systems lend themselves differently for different loads and sharding characteristics... I've seen MS-SQL in master-master across the globe, postgresql with sharding that is replicated to base host, and various ScyllaDB/Cassandra setups. It's always a mixed bag.

dclowd9901 3y ago

Nitpicky terminology choices aside, this is a very good, easy-to-follow writeup on multi-region data system design. Understanding that the trick is shuffling around the latency makes it much easier to reason about where you want to focus your latency savings. One thing I wasn't quite sure on, though, is why Database A would care if Database B had a read of a certain kind (vs. a write). Reads don't change state, so if a million reads occur with no writes, the data should still be consistent, no?

zknill 3y ago
If you're talking about the case where a local database forwards a read to the 'leader', and that read is put on the leader's log, but that read is only served when the read is replicated in the log back to the original local database then: It's actually less about the the leader caring about the read, and more about creating a point in time that the read will happen. The database is using the read in the log as the point-in-time where that read happened. The writes before that point will be viewed by the read, and the writes after won't be. By sending the read to the leader, when we receive that read back, we know for sure that all the writes that should have been replicated to our local copy have been replicated (because our read has made the full round-trip that all our writes would have, so we're guaranteed to have received all the writes before that read's "point-in-time").
- dclowd9901 3y ago
  Oh, the author! Great work on the article, and thanks for the answer!

PaulHoule 3y ago

In 2006 or so I was working for an agency that was angling for a contract for the City of Ithaca to develop a system for building inspectors to use cell-phone connected PDAs to do their paperwork on the go. At the time I advocated for an “always connected” model but we were concerned that cellular connections wouldn’t be that reliable so we thought disconnected operation would be necessary. A few year back I was thinking about the future of “low code” and thought an open-source project similar to Lotus Notes but JSON-centric would be a good thing https://en.wikipedia.org/wiki/HCL_Domino https://en.wikipedia.org/wiki/HCL_Domino In particular, Lotus notes has a model for disconnected operation that works quite well. My take is that “progressive web apps” haven’t gotten that far because they don’t have enough batteries included, a really good bidirectional sync model would make a difference. For years you could say ideas from Lotus Notes didn’t make it into the open source world because they were patented but the patents have long since expired and the main reason they are obscure is ignorance.

tracker1 3y ago
Might be interested in PouchDB/CouchDB which can be offline and resync relatively easily. Slightly different approach than other SQL based RDBMS though. In the end, it gets complicated to support offline web-apps and really depends on your usage. I had to work on a water dispensing system over a decade ago that needed to be able to work "offline" at a given site... I used firebird embedded locally, and a remotely available server for syncing against. All accounts and balances were synced to each site, so that they could operate in offline for dispensing water to carrier trucks. There were rules on offline term (up to a week) and balance carry to reduce the risk of a negative balance. In the end it worked well, but it wasn't at all simple. Active/Inactive and syncing transactions was at least interesting, and that was for a relatively simple data model. More complex models would be, of course, more complicated... and almost exponentially so.

rando101101101 3y ago

I get the premise, but something must be said for caching cutting down on those DB transactions. Do the math for YOUR app and see what makes sense.

mystic-door29 3y ago

Okay

PeterCorless 3y ago

I've seen people debating what an 'edge' even means, and wanted to chime in. Note that 'edge' is poorly defined in the industry right now. Consider the following • Remote Region Datacenter — It's certainly not this • Local Region Datacenter — It's not this • Local Municipality/County Datacenter/Co-lo/Central Office — Could it be this? • Local mini-datacenters attached to mobile towers or fiber digital loop carriers? — Could it also be this? • On-prem Datacenter — It's not this either A lot of people argue that an 'edge' has to be in the field — like a local mobile tower or a digital loop carrier. I'd argue that it could also be at a nearby co-lo facility or CO. Basically anything ≤300 km should be ≤1 ms latency. Within 75-100 km you're talking 0.25-0.33 ms latencies. And for most applications these days that seems "real-time" enough. YMMV.

LAC-Tech 3y ago

"If a user makes a request from Europe, and the apps run in US East, that adds an extra 100-150ms of latency just by round-tripping across the Atlantic." laughs in Rural New Zealand But seriously really enjoyed this article. I was always a bit confused how the recent influx of edge services handled writes/consistency, and yeah they all seem to have a single leader that routes writes to them - the edge is all about fast reading.

AtNightWeCode 3y ago

Edge is just where something is hosted. It has nothing to do with data consistency. User signup is a unique problem that any large site has. Some sites like HN also actually needs usernames which makes things more complicated. Owasp always recommend to add usernames to any login. It is such a naive approach. User names are garbage in most cases.

abhinav-t 3y ago

Without getting into a discussion on what is the correct definition of "edge", I think the article offers 2 solutions to this problem: you have an internet app and you need to provide your users with low-latency reads while maintaining uniqueness constraint and read-your-writes consistency. The article favours dealing with latency during writes while making some assumptions, - most internet apps are read-heavy (reasonable, but what if it's not the case for you?) - you do not need sharding, all regions have all the data (what if you have a ton of data that is growing quickly? what if you have durations where some data is heavily contended for?) - the primary replica used for writes is highly available (how do you automate handling of the primary replica failure or primary replica getting partitioned?) Another approach to consider would be Amazon Dynamo style databases (Cassandra, Riak etc.) which can help when the above assumptions are not met, - Dynamo was designed to be always-writable and suitable for heavy writes and reads as well (e.g. the paper mentions teams using it for catalog data with R=1 and W=N) - data is partitioned/sharded using a variant of consistent hashing, which helps with large data volumes and limiting the impact of hot spots as well - failure detection and leader election help automate handling the partitions for which a failed node was holding a primary replica

opportune 3y ago

I think the footnote is pretty much the solution.. imagine you implement replication within a DC. Yes you risk data loss and maybe inconsistency (if you fail over to a new region and start doing writes without a way to fix this later) when the DC becomes unavailable but you can now offer strong consistency without expensive cross DC hops. Pretty sure this is how Google Cloud Spanner is implemented: you have a single region for writes and everywhere else is just a read partition? And it’s probably nothing fancy to set the read partition, just pick the first hop at the time of creation Basically: implementing a distributed global db without structuring the topology to minimize hops/making engineering trade offs can’t be fast. It’s all about deciding which part of the CAP theorem you want to relax.

skybrian 3y ago

Leaving aside what "edge" really means, for me personally, this is about connecting a website running on Deno Deploy with a database. As a hobbyist doing recreational programming, I don't want to pay for a website when it gets no traffic. Everything needs to start up on demand and shut down when idle. This means that cold start time is what really matters. Since Deno Deploy has no persistent storage yet, both the frontend and the database need to start up on first request. Having a multi-region database doesn't help if needs to be running all the time and doesn't start up very fast. As for latency when warmed up, I plan to deal with that by avoiding round trips with stored procedures and caching. Here's my "hello world" website. Any advice on technologies to check out would be welcome. https://postgres-counter-demo.deno.dev/ https://postgres-counter-demo.deno.dev/

shrubble 3y ago

One definition of edge I found was '5ms average latency to the customer' , does that seem like a good definition?

kaycey2022 3y ago

Sorry for the unrelated question, but can someone tell me how this blog is built? What tech stack is used here?

byndlimitsfy 3y ago

I had no idea edge was so varied. Isn't Edge primarily involved in IOT?

mhart 3y ago

Another pattern here is to have your edge provider automatically move your compute closer to your backend. Cloudflare released Smart Placement a few months ago to do this exact thing (with surprisingly similar diagrams!) https://blog.cloudflare.com/announcing-workers-smart-placement/ https://blog.cloudflare.com/announcing-workers-smart-placeme... (disclaimer: I was the tech lead on this feature)