3 ms·

Author here, happy to answer questions!

abdullin 2y ago
Interesting read, thank you! Do you use any special tools to manage all these separate databases, track performance and debug problems?
- NathanFlurry 2y ago
  The folks over at StarbaseDB (https://starbasedb.com/ https://starbasedb.com/) are working on building tools for shareded SQLite. From the companies I've talked to, most developers using this architecture are building quick scripts to do this in-house. Both Turso and Durable Objects SQLite already a surprising amount of usage that people don't talk about much publicly yet, so I suspect some of this tooling will start to be published in the next year.
robertlagrant 2y ago
I love the idea of Durable Objects, and have thoughts about it in the health space (although having a single provider of them is probably a bad thing) but cross-cutting questions such as this seem to be quite difficult to achieve. From the chat logs example in your article: how do you cope with a requirement such as "I as a user want to see all my latest thread activity in one place, across all my chat rooms?"
- korkybuchek 2y ago
  > From the chat logs example in your article: how do you cope with a requirement such as "I as a user want to see all my latest thread activity in one place, across all my chat rooms?" create a copy, for example
- jmull 2y ago
  That's an easy case: write to two places (one for the chat and one for the user's activity). In this case I think you can let them become inconsistent in the face of, e.g., write errors.
  - NathanFlurry 2y ago
    This. > In this case I think you can let them become inconsistent in the face of, e.g., write errors. For devs using CF Durable Objects, people frequently use CF Queues or CF Workflows to ensure that everything is eventually consistent without significant overhead. It's a similar pattern to what large cos already do at scale with keeping data up to date across multiple partitions with Cassandra/DynamoDB.
neilpa 2y ago
Randomly noticed your post is dated in the future - December 16, 2025
- NathanFlurry 2y ago
  Great Scott!
grounder 2y ago
In your chat channel example, you have a table for messages, and a table for participants. How do you join the participants in this chat channel database with whichever database the participants are actually defined in, so the application would be able to show participant details (name, avatar, etc.)?
- NathanFlurry 2y ago
  The two tables are intended to be part of the same "chat" partition (ie SQLite database). You can join them with a native SQLite query. Seems I should make this more clear. Cheers
  - grounder 2y ago
    I understand the two tables in your chat example, I think. I'm wondering how you get the rest of the user profile data (name, for example). Is that table in a totally different SQLite database? If so, can you join on that, or do you need to query it separately? Thanks!
    - NathanFlurry 2y ago
      Yep. Most designs I've seen put user profiles under a different table which requires a separate query.

Interesting read, thank you! Do you use any special tools to manage all these separate databases, track performance and debug problems?

NathanFlurry 2y ago
The folks over at StarbaseDB (https://starbasedb.com/ https://starbasedb.com/) are working on building tools for shareded SQLite. From the companies I've talked to, most developers using this architecture are building quick scripts to do this in-house. Both Turso and Durable Objects SQLite already a surprising amount of usage that people don't talk about much publicly yet, so I suspect some of this tooling will start to be published in the next year.

robertlagrant 2y ago

I love the idea of Durable Objects, and have thoughts about it in the health space (although having a single provider of them is probably a bad thing) but cross-cutting questions such as this seem to be quite difficult to achieve. From the chat logs example in your article: how do you cope with a requirement such as "I as a user want to see all my latest thread activity in one place, across all my chat rooms?"

korkybuchek 2y ago
> From the chat logs example in your article: how do you cope with a requirement such as "I as a user want to see all my latest thread activity in one place, across all my chat rooms?" create a copy, for example
jmull 2y ago
That's an easy case: write to two places (one for the chat and one for the user's activity). In this case I think you can let them become inconsistent in the face of, e.g., write errors.
- NathanFlurry 2y ago
  This. > In this case I think you can let them become inconsistent in the face of, e.g., write errors. For devs using CF Durable Objects, people frequently use CF Queues or CF Workflows to ensure that everything is eventually consistent without significant overhead. It's a similar pattern to what large cos already do at scale with keeping data up to date across multiple partitions with Cassandra/DynamoDB.

neilpa 2y ago

Randomly noticed your post is dated in the future - December 16, 2025

NathanFlurry 2y ago
Great Scott!

grounder 2y ago

In your chat channel example, you have a table for messages, and a table for participants. How do you join the participants in this chat channel database with whichever database the participants are actually defined in, so the application would be able to show participant details (name, avatar, etc.)?

NathanFlurry 2y ago
The two tables are intended to be part of the same "chat" partition (ie SQLite database). You can join them with a native SQLite query. Seems I should make this more clear. Cheers
- grounder 2y ago
  I understand the two tables in your chat example, I think. I'm wondering how you get the rest of the user profile data (name, for example). Is that table in a totally different SQLite database? If so, can you join on that, or do you need to query it separately? Thanks!
  - NathanFlurry 2y ago
    Yep. Most designs I've seen put user profiles under a different table which requires a separate query.