3 ms·

Doesn’t the blog post explain why they chose this step rather than doing the thing you suggest? In particular I don’t see how this comment really addresses any

Doesn’t the blog post explain why they chose this step rather than doing the thing you suggest? In particular I don’t see how this comment really addresses any of their arguments. And if they got on ok with the json file, maybe you should update towards the features of a complicated RDBMS not being worth the cost?

cogman10 5y ago
No, it actually doesn't. The blog post explains "Hey, we've chosen a bunch of crazy ways to store data on the backend. We looked at non-crazy ways, but decided instead it'd be more fun to rsync SQLite data around because SQLite is my favorite DB!" > MySQL (or PostgreSQL) would come next. I’m not particularly familiar with anything MySQL post 1998, but I’m sure it would work. The HA story for open source databases is somewhat surprising, though: you can either have traditional lagging replicas, or commit to no-primary-replica clusters that have very surprising transaction semantics. I wasn’t excited about trying to design a stable API or good network graph calculations on top of those semantics. CockroachDB looked very promising, and indeed still does! But it’s relatively new for a database and I was a little concerned about getting attached to features in a fresh DBMS that would be hard to migrate away from if we needed to. Seriously.. consider this.. they are worried about the transaction semantics of using db replicas and instead CHOSE SQL LITE!
- tptacek 5y ago
  And? (It's sqlite, or SQLite if you're being pedantic; it's not "SQL LITE").
- bccdee 5y ago
  Sqlite replication via litestream has bascially no overhead. It's a great choice if your database only needs to service one client. It sounds like you're essentially arguing that people should always use the most robust data store available regardless of cost or requirements. Their needs were served just fine by a json file for quite some time — if that's all you need, then maintaining a full Posgres cluster is the "crazy" solution actually.

cogman10 5y ago

No, it actually doesn't. The blog post explains "Hey, we've chosen a bunch of crazy ways to store data on the backend. We looked at non-crazy ways, but decided instead it'd be more fun to rsync SQLite data around because SQLite is my favorite DB!" > MySQL (or PostgreSQL) would come next. I’m not particularly familiar with anything MySQL post 1998, but I’m sure it would work. The HA story for open source databases is somewhat surprising, though: you can either have traditional lagging replicas, or commit to no-primary-replica clusters that have very surprising transaction semantics. I wasn’t excited about trying to design a stable API or good network graph calculations on top of those semantics. CockroachDB looked very promising, and indeed still does! But it’s relatively new for a database and I was a little concerned about getting attached to features in a fresh DBMS that would be hard to migrate away from if we needed to. Seriously.. consider this.. they are worried about the transaction semantics of using db replicas and instead CHOSE SQL LITE!

tptacek 5y ago
And? (It's sqlite, or SQLite if you're being pedantic; it's not "SQL LITE").
bccdee 5y ago
Sqlite replication via litestream has bascially no overhead. It's a great choice if your database only needs to service one client. It sounds like you're essentially arguing that people should always use the most robust data store available regardless of cost or requirements. Their needs were served just fine by a json file for quite some time — if that's all you need, then maintaining a full Posgres cluster is the "crazy" solution actually.