4 ms·

rqlite[1] creator here, happy to answer any questions. https://rqlite.io https://rqlite.io

by otoolep 9mo ago

sgarland 9mo ago
Have you done any updated benchmarks on maximum supported writes/sec since the talk you gave comparing single-zone, single-region, etc.? I recently had a potential use case for this, but it required somewhere around 600 writes/sec at a minimum, and it wasn’t clear what the ceiling was for rqlite without sacrificing durability guarantees. Terrific bit of software, BTW!
- otoolep 9mo ago
  rqlite creator here. I have performed a fair amount of performance testing, some of which I outlined in a talk to the CMU Database Group a few years ago. Details: - https://www.philipotoole.com/2021-rqlite-cmu-tech-talk https://www.philipotoole.com/2021-rqlite-cmu-tech-talk - see slide 33. - There is also a recording that goes with the talk: https://www.youtube.com/watch?v=JLlIAWjvHxM https://www.youtube.com/watch?v=JLlIAWjvHxM You can also read about Performance in the docs at: https://rqlite.io/docs/guides/performance/ https://rqlite.io/docs/guides/performance/ An important thing to note: this testing was done 4+ years ago, on moderately-powerful hardware for the time. With higher-end, more modern hardware you may get even better results.
  - sgarland 9mo ago
    Thanks for the reply. Yeah, that’s the talk I was referring to. I suppose I could try my own benchmark out tbf. I’m curious to see what it can do on today’s hardware. I would think it’s mostly network-bound for Raft consensus, though the 10x ping time increase you demonstrated without an appreciable drop in writes suggests it’s more complex than that.
    - otoolep 9mo ago
      Yes, fast networks matter. I did introduce Queued Writes[1] since that talk, allowing you to trade off performance versus immediate durability. It may interest you -- network is much less of a factor then, and you should get a 10-100x increase in throughput. [1] https://rqlite.io/docs/api/queued-writes/ https://rqlite.io/docs/api/queued-writes/
      - sgarland 9mo ago
        Unfortunately for the application I was looking at using rqlite for, the possibility of data loss - however remote - was not an acceptable trade-off.
        otoolep 9mo ago
        OK, well, you could try client-side batching too, if you can. That will also improve performance substantially. Otherwise, if you try with more modern networks and disks, let me know what you see.

rqlite[1] creator here, happy to answer any questions. https://rqlite.io https://rqlite.io

by otoolep 9mo ago

rqlite[1] creator here, happy to answer any questions. https://rqlite.io https://rqlite.io

Have you done any updated benchmarks on maximum supported writes/sec since the talk you gave comparing single-zone, single-region, etc.? I recently had a potential use case for this, but it required somewhere around 600 writes/sec at a minimum, and it wasn’t clear what the ceiling was for rqlite without sacrificing durability guarantees. Terrific bit of software, BTW!

otoolep 9mo ago
rqlite creator here. I have performed a fair amount of performance testing, some of which I outlined in a talk to the CMU Database Group a few years ago. Details: - https://www.philipotoole.com/2021-rqlite-cmu-tech-talk https://www.philipotoole.com/2021-rqlite-cmu-tech-talk - see slide 33. - There is also a recording that goes with the talk: https://www.youtube.com/watch?v=JLlIAWjvHxM https://www.youtube.com/watch?v=JLlIAWjvHxM You can also read about Performance in the docs at: https://rqlite.io/docs/guides/performance/ https://rqlite.io/docs/guides/performance/ An important thing to note: this testing was done 4+ years ago, on moderately-powerful hardware for the time. With higher-end, more modern hardware you may get even better results.
- sgarland 9mo ago
  Thanks for the reply. Yeah, that’s the talk I was referring to. I suppose I could try my own benchmark out tbf. I’m curious to see what it can do on today’s hardware. I would think it’s mostly network-bound for Raft consensus, though the 10x ping time increase you demonstrated without an appreciable drop in writes suggests it’s more complex than that.
  - otoolep 9mo ago
    Yes, fast networks matter. I did introduce Queued Writes[1] since that talk, allowing you to trade off performance versus immediate durability. It may interest you -- network is much less of a factor then, and you should get a 10-100x increase in throughput. [1] https://rqlite.io/docs/api/queued-writes/ https://rqlite.io/docs/api/queued-writes/
    - sgarland 9mo ago
      Unfortunately for the application I was looking at using rqlite for, the possibility of data loss - however remote - was not an acceptable trade-off.
      - otoolep 9mo ago
        OK, well, you could try client-side batching too, if you can. That will also improve performance substantially. Otherwise, if you try with more modern networks and disks, let me know what you see.