5 ms·

Shameless plug: https://fly.io https://fly.io We built Fly _specifically_ so you can run servers close to your users. We do a lot for you on the network side,

by mrkurt 6y ago

Shameless plug: https://fly.io https://fly.io We built Fly _specifically_ so you can run servers close to your users. We do a lot for you on the network side, too, like terminate TLS in all our regions. One thing to note, though, is that latencies between cities are surprisingly different than their theoretical max. We have apps on Fly with lots of South American users. It's frequently faster for people in Argentina to connect to Miami than it is to hit Sao Paulo or Santiago. The same goes for Africa. And the "Asia Pacific" region is a monster. Most points of presence there are _thousands_ of miles away from each other. So we occasionally see people go from Tokyo -> LA instead of Tokyo -> Hong Kong/Singapore.

toast0 6y ago
> It's frequently faster for people in Argentina to connect to Miami than it is to hit Sao Paulo or Santiago. Yeah, most networks in South America don't interconnect with networks in other countries. They mostly connect in Miami, because international connectivity is difficult to arrange, and if you can only manage to connect to one other country, it needs to be the US. Africa likely goes to Europe rather than the US? But same thing, it's hard to connect, so first you have to connect to where the content is. And there's not (currently) enough network stability and capacity to just connect to your neighbors and rely on them to get you to the US/EU either. I don't know if it's current, but in Japan there used to be two dominant networks, but they weren't both available at the same PoP. You would need a Tokyo-A pop to get to one, and a Tokyo-B pop to get to the other; and it was difficult to interconnect those pops.
subleq 6y ago
The applications I work on do several database calls for single user HTTP request. Shouldn't the application server be as close as possible to the database, rather than to the user? I struggle to think of an example where your model is useful, unless your application has no database.
- danielheath 6y ago
  All the cacheable responses (including assets) benefit from being close to the user.
  - subleq 6y ago
    I'd rather have my app and databases servers together, and let a separate CDN like Cloudfront handle cacheable responses and assets. Much simpler than trying to distribute the application itself.
- mrkurt 6y ago
  App servers should have a fast access to data! Which either means running caches alongside the app servers, or doing clever things with databases. We just released a preview Postgres + regional replicas. This setup keeps the postgres leader in one region and lets people add replicas in other regions they're interested in. It works really well for typical full stack apps. We've also found a surprising number of customers who want to run in only one specific region. They don't spread apps out geographically, they just have a concentrated population of users in, say, Sydney. We're increasingly becoming "Heroku for <region>".
  - subleq 6y ago
    > regional replicas For a read-dominated access pattern that sounds quite interesting.
- kall 6y ago
  If it‘s feasable to shard data by user, you could put it in a close shard, most users don‘t move continents that much. I have never done this but as a latency nut I would like to try it sometime. Seems like Cloudflare‘s Persistent Objects kind of promises to do this automatically. Then there are a lot of requests you can serve from read replicas or caches that can be close.
  - mrkurt 6y ago
    Cockroach has a neat and almost transparent way of doing this. If you key your tables by customer, it'll happily move those chunks of data to the nodes that need them most often. We almost shipped cockroach but it's missing some Postgres features that most full stack frameworks rely on.
- e12e 6y ago
  It can work as a "smart cache" - like for example: https://fly.io/docs/app-guides/graphql-edge-caching-apollo/ https://fly.io/docs/app-guides/graphql-edge-caching-apollo/ I'm not entirely convinced this is a great idea (and apparently the example uses plain http between the graphql prox/cache and openlibrary.org - that might be considered a bug, I suppose: https://github.com/fly-apps/edge-apollo-cache/blob/master/src/books-api.js#L6 https://github.com/fly-apps/edge-apollo-cache/blob/master/sr... At any rate I'd assume whatever source you're proxying (eg your own openapi rest end points) - you might want ssl - or connect the db/api to fly.io via vpn a la: https://fly.io/blog/building-clusters-with-serf/ https://fly.io/blog/building-clusters-with-serf/ See also the linked: https://fly.io/blog/incoming-6pn-private-networks/ https://fly.io/blog/incoming-6pn-private-networks/ ).
  - tptacek 6y ago
    The Apollo thing is just an illustration of a pattern a bunch of our customers want to be able to do --- fine-grained API caching. I hope it's obvious than the HTTP connection to OpenLibrary isn't the point. :) But: while you can very easily use TLS to backhaul to an API, a more "modern" Fly.io way to solve this problem is with WireGuard gateways; it's trivial --- I'd argue, easier than configuring SSH certs --- to get a WireGuard link from your cache app on Fly back to AWS, GCP, or wherever your non-Fly legacy database lives. I really think WireGuard is going to change a lot of the ways we design systems like this.
    - kasey_junk 6y ago
      I for one absolutely agree (disclosure a friend of the fly folk and an absolute fanboy). One of the most interesting things about wireguard to me is cheap ubiquitous application neutral secure comms.
- toast0 6y ago
  Assuming you can't get your database close to your users. It depends on the data dependencies between your database calls. If they're independent, and you do them in parallel, you can do pretty well with a frontend near your user, parallel queries to backends wherever. If your queries have data dependencies, you'd get better results with a front end near your users, and a middle tier api near your database. But, just TCP (or TLS) termination near your users and the real frontend near your database can make a surprising amount of difference. On the other hand. There's a lot you can do to make sure things are fast from limited locations. Keep an eye on data size, make sure you're monitoring for and fixing slow queries, optimize your images, etc. If your html takes seconds to serve, it barely matters where you served it from.
tudorizer 6y ago
I'm intrigued. Do you own datacenters or piggybacking of some existing infra?
- mrkurt 6y ago
  We have dedicated hardware in a bunch of facilities, either leased or colo.

by mrkurt 6y ago

toast0 6y ago

> It's frequently faster for people in Argentina to connect to Miami than it is to hit Sao Paulo or Santiago. Yeah, most networks in South America don't interconnect with networks in other countries. They mostly connect in Miami, because international connectivity is difficult to arrange, and if you can only manage to connect to one other country, it needs to be the US. Africa likely goes to Europe rather than the US? But same thing, it's hard to connect, so first you have to connect to where the content is. And there's not (currently) enough network stability and capacity to just connect to your neighbors and rely on them to get you to the US/EU either. I don't know if it's current, but in Japan there used to be two dominant networks, but they weren't both available at the same PoP. You would need a Tokyo-A pop to get to one, and a Tokyo-B pop to get to the other; and it was difficult to interconnect those pops.

subleq 6y ago

The applications I work on do several database calls for single user HTTP request. Shouldn't the application server be as close as possible to the database, rather than to the user? I struggle to think of an example where your model is useful, unless your application has no database.

danielheath 6y ago
All the cacheable responses (including assets) benefit from being close to the user.
- subleq 6y ago
  I'd rather have my app and databases servers together, and let a separate CDN like Cloudfront handle cacheable responses and assets. Much simpler than trying to distribute the application itself.
mrkurt 6y ago
App servers should have a fast access to data! Which either means running caches alongside the app servers, or doing clever things with databases. We just released a preview Postgres + regional replicas. This setup keeps the postgres leader in one region and lets people add replicas in other regions they're interested in. It works really well for typical full stack apps. We've also found a surprising number of customers who want to run in only one specific region. They don't spread apps out geographically, they just have a concentrated population of users in, say, Sydney. We're increasingly becoming "Heroku for <region>".
- subleq 6y ago
  > regional replicas For a read-dominated access pattern that sounds quite interesting.
kall 6y ago
If it‘s feasable to shard data by user, you could put it in a close shard, most users don‘t move continents that much. I have never done this but as a latency nut I would like to try it sometime. Seems like Cloudflare‘s Persistent Objects kind of promises to do this automatically. Then there are a lot of requests you can serve from read replicas or caches that can be close.
- mrkurt 6y ago
  Cockroach has a neat and almost transparent way of doing this. If you key your tables by customer, it'll happily move those chunks of data to the nodes that need them most often. We almost shipped cockroach but it's missing some Postgres features that most full stack frameworks rely on.
e12e 6y ago
It can work as a "smart cache" - like for example: https://fly.io/docs/app-guides/graphql-edge-caching-apollo/ https://fly.io/docs/app-guides/graphql-edge-caching-apollo/ I'm not entirely convinced this is a great idea (and apparently the example uses plain http between the graphql prox/cache and openlibrary.org - that might be considered a bug, I suppose: https://github.com/fly-apps/edge-apollo-cache/blob/master/src/books-api.js#L6 https://github.com/fly-apps/edge-apollo-cache/blob/master/sr... At any rate I'd assume whatever source you're proxying (eg your own openapi rest end points) - you might want ssl - or connect the db/api to fly.io via vpn a la: https://fly.io/blog/building-clusters-with-serf/ https://fly.io/blog/building-clusters-with-serf/ See also the linked: https://fly.io/blog/incoming-6pn-private-networks/ https://fly.io/blog/incoming-6pn-private-networks/ ).
- tptacek 6y ago
  The Apollo thing is just an illustration of a pattern a bunch of our customers want to be able to do --- fine-grained API caching. I hope it's obvious than the HTTP connection to OpenLibrary isn't the point. :) But: while you can very easily use TLS to backhaul to an API, a more "modern" Fly.io way to solve this problem is with WireGuard gateways; it's trivial --- I'd argue, easier than configuring SSH certs --- to get a WireGuard link from your cache app on Fly back to AWS, GCP, or wherever your non-Fly legacy database lives. I really think WireGuard is going to change a lot of the ways we design systems like this.
  - kasey_junk 6y ago
    I for one absolutely agree (disclosure a friend of the fly folk and an absolute fanboy). One of the most interesting things about wireguard to me is cheap ubiquitous application neutral secure comms.
toast0 6y ago
Assuming you can't get your database close to your users. It depends on the data dependencies between your database calls. If they're independent, and you do them in parallel, you can do pretty well with a frontend near your user, parallel queries to backends wherever. If your queries have data dependencies, you'd get better results with a front end near your users, and a middle tier api near your database. But, just TCP (or TLS) termination near your users and the real frontend near your database can make a surprising amount of difference. On the other hand. There's a lot you can do to make sure things are fast from limited locations. Keep an eye on data size, make sure you're monitoring for and fixing slow queries, optimize your images, etc. If your html takes seconds to serve, it barely matters where you served it from.

tudorizer 6y ago

I'm intrigued. Do you own datacenters or piggybacking of some existing infra?

mrkurt 6y ago
We have dedicated hardware in a bunch of facilities, either leased or colo.