4 ms·

For a Cloudflare article, this one is surprisingly light on technical details. And for the product where it most matters. I'm guessing this is a single master

For a Cloudflare article, this one is surprisingly light on technical details. And for the product where it most matters. I'm guessing this is a single master database with multiple read replicas. That means it's not consistent anymore (the C in ACID). Obviously reads after a write will see stale data until the write propogates. I'm a bit curious how that replication works. Ship the whole db? Binary diffs of the master? Ship the SQL statements that did the write and reapply them? Lots of performance and other tradeoffs here. What's the latency like? This likely doesn't run in every edge location. Does the database ship out on the first request. Get cached with an expiry? Does the request itself move to the database instead of running at the edge - like maybe this runs on a select subset of locations? So many questions, but no details yet.

dragonwriter 4y ago
> I'm guessing this is a single master database with multiple read replicas. That means it's not consistent Single master with read replicas is fully consistent if commits don't return until propagated to and acknowledged by replicas (the expense here being commit latency.)
- otoolep 4y ago
  You've basically described rqlite [1], which uses Raft to coordinate the changes to the Leader, and then across some number of Followers. The write won't be acked until a quorum has persisted the change, and committed to the underlying SQLite database. Disclaimer: I am the creator of rqlite. [1] https://github.com/rqlite/rqlite https://github.com/rqlite/rqlite
  - otoolep 4y ago
    rqlite also supports read-only nodes, so in theory you can have more nodes at the edge, just like D1 -- but these nodes won't participate in the distributed consensus process. Those nodes will keep up-to-date with changes, even catching up in the event of a temporary disconnection.
- eloff 4y ago
  I would say the expense is both latency and availability because if one node doesn't ack within the timeframe then you have to drop it from the cluster. Requests that go there would need to be routed elsewhere to avoid being unavailable. If there's a network partition preventing that, then you have partial downtime. If enough nodes fail then you have full downtime across the whole cluster.
sebk 4y ago
Small nitpick, but that's still consistent as in ACID. I think what you mean is it wouldn't be consistent in the CAP sense (it wouldn't be linearizable). TFA does say that read-replicas will be present at every edge location, which makes sense for a product like Workers. But it doesn't mention writes at all.
- eloff 4y ago
  Yes, that's true.
otoolep 4y ago
I agree -- this blog post is light on details. To me the value Cloudflare believes they are offering is mostly ease-of-use, particularly setup. With minimal work you can have a stateful, relational store available to your code. But in terms of actual database functionality, they are not offering anything particularly novel. Of course, I might be missing something. In fact, I don't see anything D1 is doing that is not already offered by something like rqlite[1], which is also a super-easy-to-use distributed database built on SQLite. Of course Cloudflare will run the database for you, which is a great help -- they take care of the uptime, monitoring, backups, etc. And that's important obviously, because in the real-world databases must be operated. Disclaimer: I am the creator of rqlite. [1] https://github.com/rqlite/rqlite https://github.com/rqlite/rqlite
- rad_gruchalski 4y ago
  I’ve been looking at rqlite for some time and it’s really great to track the product on github. I believe that the power of what Cloudflare offers here isn’t in the actual database. It’s the packaging and how it sits in their serverless world. Even with rqlite, I still need ip addresses to run a resilient system. As someone who sometimes needs a table here snd there, I really, really don’t want a server. I want a table to store a thousand records in and that’s it. This is where I would very much enjoy using something like D1. A combo of D1, R2 and Workers is a serious contender for over-the-top serverless distributed apps. This is great.
  - CRConrad 4y ago
    > It’s the packaging and how it sits in their serverless world. [...] As someone who sometimes needs a table here snd there, I really, really don’t want a server. I want a table to store a thousand records in and that’s it. Sorry, but I don't get it -- WTF does "serverless" even mean here? I mean, sorry for jumping on your comment specifically, I know that wasn't primarily what you were talking about here, but... You seem to know what you're talking about, effortlessly encompassing "their serverless world" etc. The article even mentions that <<SQLite was so ahead of its time, it dubbed itself “serverless” before the term gained connotation with cloud services, and originally meant literally “not involving a server”.>> That makes sense to me; "serverless" means "not having a server". So then you have a local DB; be it SQLite or a DBF or Paradox or MS Access file or whatever. Or even a local DB software "service"; Firebird or MySQL or what have you. But the term, as it's been bandied about online for the last decade(?) or so (including in this article), seems to pretty obviously actually be about... Remote servers (that's what it talks about replicating between, right?). So what's "serverless" about that??? I've been wondering for a good while now. Anyone who has a short explanation, or link to such, please jump in and enlighten me. (Otherwise I'll have to conclude it's like "the Cloud", a.k.a. "Someone else's computer". "Serverless" = Someone else's server? :-) [Edit: Typo.] [Edit: Sigh... Two of them.]
ithrow 4y ago
Yeah, nothing about WAL mode which is what most users will want for web apps. sqlite is accessed via a socket? defeats the whole purpose of using sqlite. Many here are mentioning using one sqlite file per customer but that sounds like a nightmare for migrations and analytics. SQLite is great and all these new services and articles are nice but intentionally shadowing lots of complexity.
detroitcoder 4y ago
Going to be very interesting to see how they glue together R2, edge workers and sqllite. They can manage replication using R2 and make the sqllite process aware of this for eventual consistency. Having edge compute with edge data on a globally consistent data model is the dream.

dragonwriter 4y ago

> I'm guessing this is a single master database with multiple read replicas. That means it's not consistent Single master with read replicas is fully consistent if commits don't return until propagated to and acknowledged by replicas (the expense here being commit latency.)

otoolep 4y ago
You've basically described rqlite [1], which uses Raft to coordinate the changes to the Leader, and then across some number of Followers. The write won't be acked until a quorum has persisted the change, and committed to the underlying SQLite database. Disclaimer: I am the creator of rqlite. [1] https://github.com/rqlite/rqlite https://github.com/rqlite/rqlite
- otoolep 4y ago
  rqlite also supports read-only nodes, so in theory you can have more nodes at the edge, just like D1 -- but these nodes won't participate in the distributed consensus process. Those nodes will keep up-to-date with changes, even catching up in the event of a temporary disconnection.
eloff 4y ago
I would say the expense is both latency and availability because if one node doesn't ack within the timeframe then you have to drop it from the cluster. Requests that go there would need to be routed elsewhere to avoid being unavailable. If there's a network partition preventing that, then you have partial downtime. If enough nodes fail then you have full downtime across the whole cluster.

sebk 4y ago

Small nitpick, but that's still consistent as in ACID. I think what you mean is it wouldn't be consistent in the CAP sense (it wouldn't be linearizable). TFA does say that read-replicas will be present at every edge location, which makes sense for a product like Workers. But it doesn't mention writes at all.

eloff 4y ago
Yes, that's true.

otoolep 4y ago

I agree -- this blog post is light on details. To me the value Cloudflare believes they are offering is mostly ease-of-use, particularly setup. With minimal work you can have a stateful, relational store available to your code. But in terms of actual database functionality, they are not offering anything particularly novel. Of course, I might be missing something. In fact, I don't see anything D1 is doing that is not already offered by something like rqlite[1], which is also a super-easy-to-use distributed database built on SQLite. Of course Cloudflare will run the database for you, which is a great help -- they take care of the uptime, monitoring, backups, etc. And that's important obviously, because in the real-world databases must be operated. Disclaimer: I am the creator of rqlite. [1] https://github.com/rqlite/rqlite https://github.com/rqlite/rqlite

rad_gruchalski 4y ago
I’ve been looking at rqlite for some time and it’s really great to track the product on github. I believe that the power of what Cloudflare offers here isn’t in the actual database. It’s the packaging and how it sits in their serverless world. Even with rqlite, I still need ip addresses to run a resilient system. As someone who sometimes needs a table here snd there, I really, really don’t want a server. I want a table to store a thousand records in and that’s it. This is where I would very much enjoy using something like D1. A combo of D1, R2 and Workers is a serious contender for over-the-top serverless distributed apps. This is great.
- CRConrad 4y ago
  > It’s the packaging and how it sits in their serverless world. [...] As someone who sometimes needs a table here snd there, I really, really don’t want a server. I want a table to store a thousand records in and that’s it. Sorry, but I don't get it -- WTF does "serverless" even mean here? I mean, sorry for jumping on your comment specifically, I know that wasn't primarily what you were talking about here, but... You seem to know what you're talking about, effortlessly encompassing "their serverless world" etc. The article even mentions that <<SQLite was so ahead of its time, it dubbed itself “serverless” before the term gained connotation with cloud services, and originally meant literally “not involving a server”.>> That makes sense to me; "serverless" means "not having a server". So then you have a local DB; be it SQLite or a DBF or Paradox or MS Access file or whatever. Or even a local DB software "service"; Firebird or MySQL or what have you. But the term, as it's been bandied about online for the last decade(?) or so (including in this article), seems to pretty obviously actually be about... Remote servers (that's what it talks about replicating between, right?). So what's "serverless" about that??? I've been wondering for a good while now. Anyone who has a short explanation, or link to such, please jump in and enlighten me. (Otherwise I'll have to conclude it's like "the Cloud", a.k.a. "Someone else's computer". "Serverless" = Someone else's server? :-) [Edit: Typo.] [Edit: Sigh... Two of them.]

ithrow 4y ago

Yeah, nothing about WAL mode which is what most users will want for web apps. sqlite is accessed via a socket? defeats the whole purpose of using sqlite. Many here are mentioning using one sqlite file per customer but that sounds like a nightmare for migrations and analytics. SQLite is great and all these new services and articles are nice but intentionally shadowing lots of complexity.

detroitcoder 4y ago

Going to be very interesting to see how they glue together R2, edge workers and sqllite. They can manage replication using R2 and make the sqllite process aware of this for eventual consistency. Having edge compute with edge data on a globally consistent data model is the dream.