4 ms·

This seems to be an architecture Cloudflare is aiming to support with their SQLite service. One database per customer, each database located in the customer’s p

by ummonk 3y ago

This seems to be an architecture Cloudflare is aiming to support with their SQLite service. One database per customer, each database located in the customer’s primary region.

jasonwatkinspdx 3y ago
I think there's quite a few people chasing similar ideas, like Azure's Durable Entities. I've been calling it the Lots of Little Databases model vs the Globe Spanning Gorilla. Like the Spanner paper points out, even if your distributed database semantically appears like a single giant instance, in practice performance means developers avoid using distributed joins, etc, because these can lead to shuffling very large amounts of intermediate results across the network. So the illusion of being on a single giant machine ends up leaking through the reality, and people end up writing workarounds for distributed joins like async materialization. If we give up the single machine illusion we get a lot of simplification, at the cost of features devs were unlikely to use anyhow. I see having consistent distributed commit but without cross shard joins as a really interesting alternative. And besides scalability I like the extra security rope of fine grained partitioning from the start. I'll write a blog post along these lines if I get anything worthwhile done.
- themoonisachees 3y ago
  An advantage worth noting is that having actually separated databases means you physically can't make these expensive operations, so a junior dev can't write incredibly inefficient code that would bring down your entire infra.
  - ummonk 3y ago
    Also makes it a lot harder for devs to do some footgun and leak data across domains.
  - aledalgrande 3y ago
    Bring down the infra or foot you a 6 figures bill at the end of the month
- shalabhc 3y ago
  "Lots of Little Databases" reminded me of https://www.actordb.com/ https://www.actordb.com/ which does lots of server-side sqlite instances, but the project now looks defunct.
- ummonk 3y ago
  Interesting. Durable Entities strikes me as closer to Cloudflare's Durable Objects (both in name and in design as actors backed by persistent storage).
fauigerzigerk 3y ago
That would be fantastic. Unfortunately it's not true. D1 doesn't support the one database per customer approach unless you have just a handful of customers that you can set up manually. You have to create each database manually using wrangler or the website. Then you have to create a binding for each database in wrangler.toml so that the database becomes accessible as a variable in your Workers code. Then you have to change your Worker source code to do something with that variable. Then you redeploy. The issue is that Workers cannot create or list databases. There's no API for it.
- ummonk 3y ago
  They have said they're working on some sort of solution (which they intend to have available by the time D1 exists beta) that will allow some sort of dynamic bindings specifically to address this use case. In the meantime, though it's not a great solution, one can create and query databases through the REST API instead https://developers.cloudflare.com/api/operations/cloudflare-d1-create-database https://developers.cloudflare.com/api/operations/cloudflare-...
  - fauigerzigerk 3y ago
    >They have said they're working on some sort of solution That's great to hear, thanks!

by ummonk 3y ago

This seems to be an architecture Cloudflare is aiming to support with their SQLite service. One database per customer, each database located in the customer’s primary region.

jasonwatkinspdx 3y ago

I think there's quite a few people chasing similar ideas, like Azure's Durable Entities. I've been calling it the Lots of Little Databases model vs the Globe Spanning Gorilla. Like the Spanner paper points out, even if your distributed database semantically appears like a single giant instance, in practice performance means developers avoid using distributed joins, etc, because these can lead to shuffling very large amounts of intermediate results across the network. So the illusion of being on a single giant machine ends up leaking through the reality, and people end up writing workarounds for distributed joins like async materialization. If we give up the single machine illusion we get a lot of simplification, at the cost of features devs were unlikely to use anyhow. I see having consistent distributed commit but without cross shard joins as a really interesting alternative. And besides scalability I like the extra security rope of fine grained partitioning from the start. I'll write a blog post along these lines if I get anything worthwhile done.

themoonisachees 3y ago
An advantage worth noting is that having actually separated databases means you physically can't make these expensive operations, so a junior dev can't write incredibly inefficient code that would bring down your entire infra.
- ummonk 3y ago
  Also makes it a lot harder for devs to do some footgun and leak data across domains.
- aledalgrande 3y ago
  Bring down the infra or foot you a 6 figures bill at the end of the month
shalabhc 3y ago
"Lots of Little Databases" reminded me of https://www.actordb.com/ https://www.actordb.com/ which does lots of server-side sqlite instances, but the project now looks defunct.
ummonk 3y ago
Interesting. Durable Entities strikes me as closer to Cloudflare's Durable Objects (both in name and in design as actors backed by persistent storage).

fauigerzigerk 3y ago

That would be fantastic. Unfortunately it's not true. D1 doesn't support the one database per customer approach unless you have just a handful of customers that you can set up manually. You have to create each database manually using wrangler or the website. Then you have to create a binding for each database in wrangler.toml so that the database becomes accessible as a variable in your Workers code. Then you have to change your Worker source code to do something with that variable. Then you redeploy. The issue is that Workers cannot create or list databases. There's no API for it.

ummonk 3y ago
They have said they're working on some sort of solution (which they intend to have available by the time D1 exists beta) that will allow some sort of dynamic bindings specifically to address this use case. In the meantime, though it's not a great solution, one can create and query databases through the REST API instead https://developers.cloudflare.com/api/operations/cloudflare-d1-create-database https://developers.cloudflare.com/api/operations/cloudflare-...
- fauigerzigerk 3y ago
  >They have said they're working on some sort of solution That's great to hear, thanks!