5 ms·

The applications I work on do several database calls for single user HTTP request. Shouldn't the application server be as close as possible to the database, rat

by subleq 6y ago

The applications I work on do several database calls for single user HTTP request. Shouldn't the application server be as close as possible to the database, rather than to the user? I struggle to think of an example where your model is useful, unless your application has no database.

danielheath 6y ago
All the cacheable responses (including assets) benefit from being close to the user.
- subleq 6y ago
  I'd rather have my app and databases servers together, and let a separate CDN like Cloudfront handle cacheable responses and assets. Much simpler than trying to distribute the application itself.
mrkurt 6y ago
App servers should have a fast access to data! Which either means running caches alongside the app servers, or doing clever things with databases. We just released a preview Postgres + regional replicas. This setup keeps the postgres leader in one region and lets people add replicas in other regions they're interested in. It works really well for typical full stack apps. We've also found a surprising number of customers who want to run in only one specific region. They don't spread apps out geographically, they just have a concentrated population of users in, say, Sydney. We're increasingly becoming "Heroku for <region>".
- subleq 6y ago
  > regional replicas For a read-dominated access pattern that sounds quite interesting.
kall 6y ago
If it‘s feasable to shard data by user, you could put it in a close shard, most users don‘t move continents that much. I have never done this but as a latency nut I would like to try it sometime. Seems like Cloudflare‘s Persistent Objects kind of promises to do this automatically. Then there are a lot of requests you can serve from read replicas or caches that can be close.
- mrkurt 6y ago
  Cockroach has a neat and almost transparent way of doing this. If you key your tables by customer, it'll happily move those chunks of data to the nodes that need them most often. We almost shipped cockroach but it's missing some Postgres features that most full stack frameworks rely on.
e12e 6y ago
It can work as a "smart cache" - like for example: https://fly.io/docs/app-guides/graphql-edge-caching-apollo/ https://fly.io/docs/app-guides/graphql-edge-caching-apollo/ I'm not entirely convinced this is a great idea (and apparently the example uses plain http between the graphql prox/cache and openlibrary.org - that might be considered a bug, I suppose: https://github.com/fly-apps/edge-apollo-cache/blob/master/src/books-api.js#L6 https://github.com/fly-apps/edge-apollo-cache/blob/master/sr... At any rate I'd assume whatever source you're proxying (eg your own openapi rest end points) - you might want ssl - or connect the db/api to fly.io via vpn a la: https://fly.io/blog/building-clusters-with-serf/ https://fly.io/blog/building-clusters-with-serf/ See also the linked: https://fly.io/blog/incoming-6pn-private-networks/ https://fly.io/blog/incoming-6pn-private-networks/ ).
- tptacek 6y ago
  The Apollo thing is just an illustration of a pattern a bunch of our customers want to be able to do --- fine-grained API caching. I hope it's obvious than the HTTP connection to OpenLibrary isn't the point. :) But: while you can very easily use TLS to backhaul to an API, a more "modern" Fly.io way to solve this problem is with WireGuard gateways; it's trivial --- I'd argue, easier than configuring SSH certs --- to get a WireGuard link from your cache app on Fly back to AWS, GCP, or wherever your non-Fly legacy database lives. I really think WireGuard is going to change a lot of the ways we design systems like this.
  - kasey_junk 6y ago
    I for one absolutely agree (disclosure a friend of the fly folk and an absolute fanboy). One of the most interesting things about wireguard to me is cheap ubiquitous application neutral secure comms.
toast0 6y ago
Assuming you can't get your database close to your users. It depends on the data dependencies between your database calls. If they're independent, and you do them in parallel, you can do pretty well with a frontend near your user, parallel queries to backends wherever. If your queries have data dependencies, you'd get better results with a front end near your users, and a middle tier api near your database. But, just TCP (or TLS) termination near your users and the real frontend near your database can make a surprising amount of difference. On the other hand. There's a lot you can do to make sure things are fast from limited locations. Keep an eye on data size, make sure you're monitoring for and fixing slow queries, optimize your images, etc. If your html takes seconds to serve, it barely matters where you served it from.

by subleq 6y ago

danielheath 6y ago

All the cacheable responses (including assets) benefit from being close to the user.

subleq 6y ago
I'd rather have my app and databases servers together, and let a separate CDN like Cloudfront handle cacheable responses and assets. Much simpler than trying to distribute the application itself.

mrkurt 6y ago

App servers should have a fast access to data! Which either means running caches alongside the app servers, or doing clever things with databases. We just released a preview Postgres + regional replicas. This setup keeps the postgres leader in one region and lets people add replicas in other regions they're interested in. It works really well for typical full stack apps. We've also found a surprising number of customers who want to run in only one specific region. They don't spread apps out geographically, they just have a concentrated population of users in, say, Sydney. We're increasingly becoming "Heroku for <region>".

subleq 6y ago
> regional replicas For a read-dominated access pattern that sounds quite interesting.

kall 6y ago

If it‘s feasable to shard data by user, you could put it in a close shard, most users don‘t move continents that much. I have never done this but as a latency nut I would like to try it sometime. Seems like Cloudflare‘s Persistent Objects kind of promises to do this automatically. Then there are a lot of requests you can serve from read replicas or caches that can be close.

mrkurt 6y ago
Cockroach has a neat and almost transparent way of doing this. If you key your tables by customer, it'll happily move those chunks of data to the nodes that need them most often. We almost shipped cockroach but it's missing some Postgres features that most full stack frameworks rely on.

e12e 6y ago

It can work as a "smart cache" - like for example: https://fly.io/docs/app-guides/graphql-edge-caching-apollo/ https://fly.io/docs/app-guides/graphql-edge-caching-apollo/ I'm not entirely convinced this is a great idea (and apparently the example uses plain http between the graphql prox/cache and openlibrary.org - that might be considered a bug, I suppose: https://github.com/fly-apps/edge-apollo-cache/blob/master/src/books-api.js#L6 https://github.com/fly-apps/edge-apollo-cache/blob/master/sr... At any rate I'd assume whatever source you're proxying (eg your own openapi rest end points) - you might want ssl - or connect the db/api to fly.io via vpn a la: https://fly.io/blog/building-clusters-with-serf/ https://fly.io/blog/building-clusters-with-serf/ See also the linked: https://fly.io/blog/incoming-6pn-private-networks/ https://fly.io/blog/incoming-6pn-private-networks/ ).

tptacek 6y ago
The Apollo thing is just an illustration of a pattern a bunch of our customers want to be able to do --- fine-grained API caching. I hope it's obvious than the HTTP connection to OpenLibrary isn't the point. :) But: while you can very easily use TLS to backhaul to an API, a more "modern" Fly.io way to solve this problem is with WireGuard gateways; it's trivial --- I'd argue, easier than configuring SSH certs --- to get a WireGuard link from your cache app on Fly back to AWS, GCP, or wherever your non-Fly legacy database lives. I really think WireGuard is going to change a lot of the ways we design systems like this.
- kasey_junk 6y ago
  I for one absolutely agree (disclosure a friend of the fly folk and an absolute fanboy). One of the most interesting things about wireguard to me is cheap ubiquitous application neutral secure comms.

toast0 6y ago

Assuming you can't get your database close to your users. It depends on the data dependencies between your database calls. If they're independent, and you do them in parallel, you can do pretty well with a frontend near your user, parallel queries to backends wherever. If your queries have data dependencies, you'd get better results with a front end near your users, and a middle tier api near your database. But, just TCP (or TLS) termination near your users and the real frontend near your database can make a surprising amount of difference. On the other hand. There's a lot you can do to make sure things are fast from limited locations. Keep an eye on data size, make sure you're monitoring for and fixing slow queries, optimize your images, etc. If your html takes seconds to serve, it barely matters where you served it from.