5 ms·

While I am a fan of TigerBeetle’s general stance on correctness, coding practices, and their desire to hyper-specialize, I have some critiques on the post. I t

While I am a fan of TigerBeetle’s general stance on correctness, coding practices, and their desire to hyper-specialize, I have some critiques on the post. I think the paragraph about multi-node is a bit misleading. Contrary to what cloud native folk will tell you, a single beefy DB, well-tuned and with a connection pooler, can serve a dizzying amount of QPS just fine. At a former employer, during a maintenance period, I once accidentally had all traffic pointed to our single MySQL 8 RDS instance, instead of sharing it between its read replicas. That was somewhere around 80-90K QPS, and it didn’t care at all. It wasn’t even a giant instance - r6i.12xlarge - we just had a decent schema, mostly sane queries, and good tuning on both ProxySQL and MySQL. At peak, that writer and two .8xlarge read replicas handled 120K QPS without blinking. A DB hosted on a server with node-local NVMe (you know, what used to be normal) will likely hit CPU limits before you saturate its I/O capabilities. For redundancy, all RDBMS designed for networked activity have some form of failover / hot standby capability. My other mild criticism is in the discussion on TigerBeetle’s consensus: yes, it seems quite clever and has no other dependencies, but it’s also not trying to deal with large rows. When you can fit 8,190 transactions into a 1 MiB packet that takes a single trip to be delivered, you can probably manage what would be impossible for a traditional RDBMS. None of this should be taken as belittling their accomplishment; I remain extremely impressed by their product.

justinhj 1y ago
That a traditional modern RDBMS is very fast is not really useful here because the use case of Tigerbeetle includes work loads with high contention. Under that kind of load, in their own tests you can see in a recent conference talk, you can see the throughput drops dramatically in clusters where many transactions affect a single account. edit: from the horses mouth is better https://news.ycombinator.com/item?id=45437046 https://news.ycombinator.com/item?id=45437046
- ffsm8 1y ago
  Huh, that as not been my experience at all. Too be fair, its been something like 10 years iirc. The database in question was MySQL 8, running on plain old enterprise ssds (RAID 10) The workload was processing transactions (financial payments) The database schema was ... Let's call it questionable, with pretty much no normalization because "it's easier when we look at it for debugging", hence extremely long rows with countless updates to the same row throughout the processing, roughly 250-500 writes per row per request/transaction from what I recall. And the application was a unholy combination of a PHP+Java monolith, linked via RPC and transparent class sharing DB IO was _never_ the problem, no matter how high qps got. I can't quote an exact number, but it was definitely a lot higher then what this claims (something like 40-50k on average "load" days like pre Christmas etc) Not sure how they're getting this down to ~250qps, it sounds completely implausible. Heck, I can do single row non-stop updates with >1k qpm on my desktop on a single nvme drive - and that's not even using raid.
  - justinhj 1y ago
    Using your numbers and speaking roughly, if you're doing 50k rps and 500 typically to the same row wasn't your contention around 1%? TigerBeetle clais to be able to handle work loads with very high contention, for example 80 or 90% of transactions filtering out a fee to a single account to make up a use case.
    - ffsm8 1y ago
      Ah, you might be right! Contention is what can be parallelized, right? So with roughly 100-200 requests/s you end up with 1-0.5 contention if I understood that right. That moves me even further towards agarlands points, though - if I plug that into the equation, I end up with >50k qps. The used numbers create an insanely distorted idea wrt real world performance
- mike_hearn 1y ago
  It's worth noting that other databases have features designed specifically to address this use case. Oracle has lock free reservations on numeric columns: https://oracle-base.com/articles/23/lock-free-reservations-23 https://oracle-base.com/articles/23/lock-free-reservations-2...
parthdesai 1y ago
> My other mild criticism is in the discussion on TigerBeetle’s consensus: yes, it seems quite clever and has no other dependencies, but it’s also not trying to deal with large rows. When you can fit 8,190 transactions into a 1 MiB packet that takes a single trip to be delivered, you can probably manage what would be impossible for a traditional RDBMS. Isn't that the point? They're saying to separate out transactions workload from other workloads. They're not saying they'll replace your OLGP db, you remove transactionally important data into another db. It's something similar that we see with another db: https://turbopuffer.com/ https://turbopuffer.com/
zenmac 1y ago
>we just had a decent schema Glad someone mentioned it. In the dev ORM culture now days it is all about how to make the db look like the code. DB is essentially reduced to a object store. Often a proper decent schema can handle most of these load. You can always throw in redis layer later if your traffic is really getting to require that level of optimization.

justinhj 1y ago

That a traditional modern RDBMS is very fast is not really useful here because the use case of Tigerbeetle includes work loads with high contention. Under that kind of load, in their own tests you can see in a recent conference talk, you can see the throughput drops dramatically in clusters where many transactions affect a single account. edit: from the horses mouth is better https://news.ycombinator.com/item?id=45437046 https://news.ycombinator.com/item?id=45437046

ffsm8 1y ago
Huh, that as not been my experience at all. Too be fair, its been something like 10 years iirc. The database in question was MySQL 8, running on plain old enterprise ssds (RAID 10) The workload was processing transactions (financial payments) The database schema was ... Let's call it questionable, with pretty much no normalization because "it's easier when we look at it for debugging", hence extremely long rows with countless updates to the same row throughout the processing, roughly 250-500 writes per row per request/transaction from what I recall. And the application was a unholy combination of a PHP+Java monolith, linked via RPC and transparent class sharing DB IO was _never_ the problem, no matter how high qps got. I can't quote an exact number, but it was definitely a lot higher then what this claims (something like 40-50k on average "load" days like pre Christmas etc) Not sure how they're getting this down to ~250qps, it sounds completely implausible. Heck, I can do single row non-stop updates with >1k qpm on my desktop on a single nvme drive - and that's not even using raid.
- justinhj 1y ago
  Using your numbers and speaking roughly, if you're doing 50k rps and 500 typically to the same row wasn't your contention around 1%? TigerBeetle clais to be able to handle work loads with very high contention, for example 80 or 90% of transactions filtering out a fee to a single account to make up a use case.
  - ffsm8 1y ago
    Ah, you might be right! Contention is what can be parallelized, right? So with roughly 100-200 requests/s you end up with 1-0.5 contention if I understood that right. That moves me even further towards agarlands points, though - if I plug that into the equation, I end up with >50k qps. The used numbers create an insanely distorted idea wrt real world performance
mike_hearn 1y ago
It's worth noting that other databases have features designed specifically to address this use case. Oracle has lock free reservations on numeric columns: https://oracle-base.com/articles/23/lock-free-reservations-23 https://oracle-base.com/articles/23/lock-free-reservations-2...

parthdesai 1y ago

> My other mild criticism is in the discussion on TigerBeetle’s consensus: yes, it seems quite clever and has no other dependencies, but it’s also not trying to deal with large rows. When you can fit 8,190 transactions into a 1 MiB packet that takes a single trip to be delivered, you can probably manage what would be impossible for a traditional RDBMS. Isn't that the point? They're saying to separate out transactions workload from other workloads. They're not saying they'll replace your OLGP db, you remove transactionally important data into another db. It's something similar that we see with another db: https://turbopuffer.com/ https://turbopuffer.com/

zenmac 1y ago

>we just had a decent schema Glad someone mentioned it. In the dev ORM culture now days it is all about how to make the db look like the code. DB is essentially reduced to a object store. Often a proper decent schema can handle most of these load. You can always throw in redis layer later if your traffic is really getting to require that level of optimization.