3 ms·

I'm curious about how rqlite's performance compares to other distributed databases developed in Go, such as CockroachDB, Vitess, and TiDB.

by west0n 2y ago

jitl 2y ago
It’s going to have much lower write throughput, since SQLite is single-writer and on top of that you need to do Raft consensus. TiDB and CockroachDB can handle concurrent writes easily. Cockroach runs raft per “range” of 128mb of the key space, I’m not as familiar with TiDB. Vitess is an orchestration layer over MySQL, and MySQL handles concurrent writes easily.
- otoolep 2y ago
  rqlite creator here. That's correct, there is a write-performance hit for the reasons you say. All Raft systems will take the same hit, and SQLite is intrinsically single-writer -- nothing about rqlite changes that[2]. That said, there are approaches to increasing write-performance substantially. See [1] for much more information. Write-performance is not the only thing to consider though (assuming one has sufficient performance in that dimension). Ease of deployment and operation are also important, and that's an area in which rqlite excels[3] (at least I think so, but I'm biased). [1] https://rqlite.io/docs/guides/performance/ https://rqlite.io/docs/guides/performance/ [2] https://rqlite.io/docs/faq/#rqlite-is-distributed-does-that-mean-it-can-increase-sqlite-performance https://rqlite.io/docs/faq/#rqlite-is-distributed-does-that-... [3] https://rqlite.io/docs/faq/#why-would-i-use-this-versus-some-other-distributed-database https://rqlite.io/docs/faq/#why-would-i-use-this-versus-some...
  - otoolep 2y ago
    Oh, I also presented some performance numbers in a presentation to a CMU a couple of years back. A little out-of-date, but gives a order-of-magnitude sense. https://youtu.be/JLlIAWjvHxM?t=2690 https://youtu.be/JLlIAWjvHxM?t=2690 The biggest performance improvement since is due to the introduction of Queued Writes. See https://rqlite.io/docs/api/queued-writes/ https://rqlite.io/docs/api/queued-writes/
  - spmurrayzzz 2y ago
    Relevant to the original inquiry — I really admire that you bring up the etcd and consul comparison right up front in the readme. For my own comprehension at least, it makes obvious the type of workloads for which you're optimizing and I appreciate that context as a past user of both of those stacks.
  - joostdecock 2y ago
    > Ease of deployment and operation are also important, and that's an area in which rqlite excels Amen. I've been building something appliance-like where I want to support clustering but I don't want to manage a database cluster inside the project. Rqlite is so easy to run either stand-alone or clustered. It's a godsend. And when people want postgres or whatever, I let them bring their own database. It's not hard to abstract a database storage layet if you plan ahead. But if you want it to 'just work' rqlite is doing that with flying colors.
ClumsyPilot 2y ago
Maybe ETCD is a more appropriate comparison?
- protosam 2y ago
  Depending on what you’re using these tools for. If you want a locking manager and some meta data storage to help your distributed system maintain state, etcd is better for the job than rqlite for that. It’s a better zookeeper. With etcd you can hold a lock and defer unlocking if the connection is disrupted. Rqlite is not a good option for this.
  - otoolep 2y ago
    Agreed, in the sense that while rqlite has a lot in common with etcd (and Consul too -- Consul and rqlite share the same Raft implementation[1]) rqlite's primary use case is not about making it easy to build other distributed systems on top of it. [1] https://github.com/hashicorp/raft https://github.com/hashicorp/raft
    - protosam 2y ago
      Every time I've looked at rqlite, it just falls short features-wise in what I would want to do with it. A single raft group does not scale horizontally, so to me rqlite is a toy rather than a tool worth using (because someone might mistake the toy as production grade software).
      - otoolep 2y ago
        rqlite creator here. That's clearly a mistaken attitude because both Consul and etcd also use a single "Raft group" and they are production-grade software. Ruling out a piece of software simply because it doesn't "scale horizontally" (and only writes don't scale horizontally in practice) is a naive attitude.
        protosam 2y ago

I'm curious about how rqlite's performance compares to other distributed databases developed in Go, such as CockroachDB, Vitess, and TiDB.

by west0n 2y ago

I'm curious about how rqlite's performance compares to other distributed databases developed in Go, such as CockroachDB, Vitess, and TiDB.

jitl 2y ago

It’s going to have much lower write throughput, since SQLite is single-writer and on top of that you need to do Raft consensus. TiDB and CockroachDB can handle concurrent writes easily. Cockroach runs raft per “range” of 128mb of the key space, I’m not as familiar with TiDB. Vitess is an orchestration layer over MySQL, and MySQL handles concurrent writes easily.

otoolep 2y ago
rqlite creator here. That's correct, there is a write-performance hit for the reasons you say. All Raft systems will take the same hit, and SQLite is intrinsically single-writer -- nothing about rqlite changes that[2]. That said, there are approaches to increasing write-performance substantially. See [1] for much more information. Write-performance is not the only thing to consider though (assuming one has sufficient performance in that dimension). Ease of deployment and operation are also important, and that's an area in which rqlite excels[3] (at least I think so, but I'm biased). [1] https://rqlite.io/docs/guides/performance/ https://rqlite.io/docs/guides/performance/ [2] https://rqlite.io/docs/faq/#rqlite-is-distributed-does-that-mean-it-can-increase-sqlite-performance https://rqlite.io/docs/faq/#rqlite-is-distributed-does-that-... [3] https://rqlite.io/docs/faq/#why-would-i-use-this-versus-some-other-distributed-database https://rqlite.io/docs/faq/#why-would-i-use-this-versus-some...
- otoolep 2y ago
  Oh, I also presented some performance numbers in a presentation to a CMU a couple of years back. A little out-of-date, but gives a order-of-magnitude sense. https://youtu.be/JLlIAWjvHxM?t=2690 https://youtu.be/JLlIAWjvHxM?t=2690 The biggest performance improvement since is due to the introduction of Queued Writes. See https://rqlite.io/docs/api/queued-writes/ https://rqlite.io/docs/api/queued-writes/
- spmurrayzzz 2y ago
  Relevant to the original inquiry — I really admire that you bring up the etcd and consul comparison right up front in the readme. For my own comprehension at least, it makes obvious the type of workloads for which you're optimizing and I appreciate that context as a past user of both of those stacks.
- joostdecock 2y ago
  > Ease of deployment and operation are also important, and that's an area in which rqlite excels Amen. I've been building something appliance-like where I want to support clustering but I don't want to manage a database cluster inside the project. Rqlite is so easy to run either stand-alone or clustered. It's a godsend. And when people want postgres or whatever, I let them bring their own database. It's not hard to abstract a database storage layet if you plan ahead. But if you want it to 'just work' rqlite is doing that with flying colors.

ClumsyPilot 2y ago

Maybe ETCD is a more appropriate comparison?

protosam 2y ago
Depending on what you’re using these tools for. If you want a locking manager and some meta data storage to help your distributed system maintain state, etcd is better for the job than rqlite for that. It’s a better zookeeper. With etcd you can hold a lock and defer unlocking if the connection is disrupted. Rqlite is not a good option for this.
- otoolep 2y ago
  Agreed, in the sense that while rqlite has a lot in common with etcd (and Consul too -- Consul and rqlite share the same Raft implementation[1]) rqlite's primary use case is not about making it easy to build other distributed systems on top of it. [1] https://github.com/hashicorp/raft https://github.com/hashicorp/raft
  - protosam 2y ago
    Every time I've looked at rqlite, it just falls short features-wise in what I would want to do with it. A single raft group does not scale horizontally, so to me rqlite is a toy rather than a tool worth using (because someone might mistake the toy as production grade software).
    - otoolep 2y ago
      rqlite creator here. That's clearly a mistaken attitude because both Consul and etcd also use a single "Raft group" and they are production-grade software. Ruling out a piece of software simply because it doesn't "scale horizontally" (and only writes don't scale horizontally in practice) is a naive attitude.
      - protosam 2y ago