3 ms·

If one API request makes on average 5-10 round trips to the database, and the database is in Virginia, this only makes the problem (much) worse. How do you solv

by a13n 7y ago

If one API request makes on average 5-10 round trips to the database, and the database is in Virginia, this only makes the problem (much) worse. How do you solve this problem for this use case?

michaeldwan 7y ago
We're not solving db latency yet. A good place to start is aggressively caching at the edge. We offer an in-memory redis cache for this that can replicate commands globally. Beyond that you'd need read replicas which will be possible once we launch persistent storage. That said, latency between data centers on the same continent is often less than I would have thought!
- a13n 7y ago
  Gotcha, honestly that feels pretty niche to me, which might be a good place for a startup to start. I can't think of many back-end applications between purely static content (just use a CDN) and needs a database connection. Probably video game servers, where you don't need the game state to be (immediately) stored/accessed globally.
  - mrkurt 7y ago
    Game servers are a great example. What's interesting is how many different kinds of apps need game-server like infrastructure: https://www.figma.com/blog/how-figmas-multiplayer-technology-works/ https://www.figma.com/blog/how-figmas-multiplayer-technology... We've been talking to a lot of startups doing communications tools, especially for remote work. Lots of full stack apps benefit from app servers + redis cache in different regions. They need a database connection, but if they're already done the work to minimize DB round trips they might just work with no code changes. There are also a bunch of folks doing really dynamic video and image delivery. Where an individual user gets an entirely unique blob of binary data.
  - ShroudedNight 7y ago
    I could see IoT device 'acceleration' to be a significant potential use-case. Something with a tiny bill of materials for the device itself, offloading any non-trivial processing to a virtual device on the closest 'real' infrastructure you can get. Especially for something human-interactive, you would want to be very aggressive about minimizing latency. Also, depending on how tight the limits are for VM lifetime / bandwidth / outbound connections, I could see using these as a kind of virtual NIC / service mesh type thing for consumer-grade internet connections, to restore the inbound routing capabilities precluded by carrier-grade NAT and avoid their traffic discrimination, as well as potentially on-boarding to higher-quality transit as early as possible for use when accessing latency-sensitive services further 'interior' to the cloud.
    - mrkurt 7y ago
      These are great. IoT seems like a thing you could do but that's a really specific use case I hadn't even considered. The second example would be interesting to try. There's no real limit on VM lifetime or outbound connections, bandwidth is more of a budget problem. VMs are ephemeral, so they _can_ go away but we're all happier if they just run forever.
eloff 7y ago
You should solve this by making one request to a db api layer (use GraphQL!) And have that layer do the back and forth with the db, from right next to it. Holding transactions open for long distance round trips is going to get you into trouble in a myriad of ways. It does not scale.
- michaeldwan 7y ago
  This is exactly what we do! We have a Rails app exposing a graphql api for our cli and web app. The web app is a React SPA monster that we're replacing with another Rails app that runs at the edge and serves the customer UI, docs, marketing pages, and some other things. It's really pleasant. We love graphql.
  - jpochtar 7y ago
    Why not just have the rails app exposing GQL also expose the customer UI/etc?
    - mrkurt 7y ago
      That's actually the plan! The Ruby Graphql gem lets you resolve queries directly so the app can have an "edge" or "core" mode and make the right call (edge over HTTP, core direct to DB).
    - michaeldwan 7y ago
      Mostly because the path to here was not a straight one :) Our main Rails app does a bunch of things that can't run globally and it takes a long time to build while the customer facing app is a lightweight Rails app that consolidated several static and single page react apps into one less gross place.
mwcampbell 7y ago
One good solution might be something like Datomic (the classic version, now called Datomic On-Prem), where each application process has the query engine embedded in it, and it pulls data from a relatively dumb storage service such as DynamoDB as needed, then aggressively caches it.
- mrkurt 7y ago
  That would work great. Even something like FaunaDB would work really well.
northstar702 7y ago
You'd want to pair up with a global database like FaunaDB or Cosmos DB for the data backend.

by a13n 7y ago

If one API request makes on average 5-10 round trips to the database, and the database is in Virginia, this only makes the problem (much) worse. How do you solve this problem for this use case?

michaeldwan 7y ago

We're not solving db latency yet. A good place to start is aggressively caching at the edge. We offer an in-memory redis cache for this that can replicate commands globally. Beyond that you'd need read replicas which will be possible once we launch persistent storage. That said, latency between data centers on the same continent is often less than I would have thought!

a13n 7y ago
Gotcha, honestly that feels pretty niche to me, which might be a good place for a startup to start. I can't think of many back-end applications between purely static content (just use a CDN) and needs a database connection. Probably video game servers, where you don't need the game state to be (immediately) stored/accessed globally.
- mrkurt 7y ago
  Game servers are a great example. What's interesting is how many different kinds of apps need game-server like infrastructure: https://www.figma.com/blog/how-figmas-multiplayer-technology-works/ https://www.figma.com/blog/how-figmas-multiplayer-technology... We've been talking to a lot of startups doing communications tools, especially for remote work. Lots of full stack apps benefit from app servers + redis cache in different regions. They need a database connection, but if they're already done the work to minimize DB round trips they might just work with no code changes. There are also a bunch of folks doing really dynamic video and image delivery. Where an individual user gets an entirely unique blob of binary data.
- ShroudedNight 7y ago
  I could see IoT device 'acceleration' to be a significant potential use-case. Something with a tiny bill of materials for the device itself, offloading any non-trivial processing to a virtual device on the closest 'real' infrastructure you can get. Especially for something human-interactive, you would want to be very aggressive about minimizing latency. Also, depending on how tight the limits are for VM lifetime / bandwidth / outbound connections, I could see using these as a kind of virtual NIC / service mesh type thing for consumer-grade internet connections, to restore the inbound routing capabilities precluded by carrier-grade NAT and avoid their traffic discrimination, as well as potentially on-boarding to higher-quality transit as early as possible for use when accessing latency-sensitive services further 'interior' to the cloud.
  - mrkurt 7y ago
    These are great. IoT seems like a thing you could do but that's a really specific use case I hadn't even considered. The second example would be interesting to try. There's no real limit on VM lifetime or outbound connections, bandwidth is more of a budget problem. VMs are ephemeral, so they _can_ go away but we're all happier if they just run forever.

eloff 7y ago

You should solve this by making one request to a db api layer (use GraphQL!) And have that layer do the back and forth with the db, from right next to it. Holding transactions open for long distance round trips is going to get you into trouble in a myriad of ways. It does not scale.

michaeldwan 7y ago
This is exactly what we do! We have a Rails app exposing a graphql api for our cli and web app. The web app is a React SPA monster that we're replacing with another Rails app that runs at the edge and serves the customer UI, docs, marketing pages, and some other things. It's really pleasant. We love graphql.
- jpochtar 7y ago
  Why not just have the rails app exposing GQL also expose the customer UI/etc?
  - mrkurt 7y ago
    That's actually the plan! The Ruby Graphql gem lets you resolve queries directly so the app can have an "edge" or "core" mode and make the right call (edge over HTTP, core direct to DB).
  - michaeldwan 7y ago
    Mostly because the path to here was not a straight one :) Our main Rails app does a bunch of things that can't run globally and it takes a long time to build while the customer facing app is a lightweight Rails app that consolidated several static and single page react apps into one less gross place.

mwcampbell 7y ago

One good solution might be something like Datomic (the classic version, now called Datomic On-Prem), where each application process has the query engine embedded in it, and it pulls data from a relatively dumb storage service such as DynamoDB as needed, then aggressively caches it.

mrkurt 7y ago
That would work great. Even something like FaunaDB would work really well.

northstar702 7y ago

You'd want to pair up with a global database like FaunaDB or Cosmos DB for the data backend.