5 ms·

That a traditional modern RDBMS is very fast is not really useful here because the use case of Tigerbeetle includes work loads with high contention. Under that

That a traditional modern RDBMS is very fast is not really useful here because the use case of Tigerbeetle includes work loads with high contention. Under that kind of load, in their own tests you can see in a recent conference talk, you can see the throughput drops dramatically in clusters where many transactions affect a single account. edit: from the horses mouth is better https://news.ycombinator.com/item?id=45437046 https://news.ycombinator.com/item?id=45437046

ffsm8 1y ago
Huh, that as not been my experience at all. Too be fair, its been something like 10 years iirc. The database in question was MySQL 8, running on plain old enterprise ssds (RAID 10) The workload was processing transactions (financial payments) The database schema was ... Let's call it questionable, with pretty much no normalization because "it's easier when we look at it for debugging", hence extremely long rows with countless updates to the same row throughout the processing, roughly 250-500 writes per row per request/transaction from what I recall. And the application was a unholy combination of a PHP+Java monolith, linked via RPC and transparent class sharing DB IO was _never_ the problem, no matter how high qps got. I can't quote an exact number, but it was definitely a lot higher then what this claims (something like 40-50k on average "load" days like pre Christmas etc) Not sure how they're getting this down to ~250qps, it sounds completely implausible. Heck, I can do single row non-stop updates with >1k qpm on my desktop on a single nvme drive - and that's not even using raid.
- justinhj 1y ago
  Using your numbers and speaking roughly, if you're doing 50k rps and 500 typically to the same row wasn't your contention around 1%? TigerBeetle clais to be able to handle work loads with very high contention, for example 80 or 90% of transactions filtering out a fee to a single account to make up a use case.
  - ffsm8 1y ago
    Ah, you might be right! Contention is what can be parallelized, right? So with roughly 100-200 requests/s you end up with 1-0.5 contention if I understood that right. That moves me even further towards agarlands points, though - if I plug that into the equation, I end up with >50k qps. The used numbers create an insanely distorted idea wrt real world performance
mike_hearn 1y ago
It's worth noting that other databases have features designed specifically to address this use case. Oracle has lock free reservations on numeric columns: https://oracle-base.com/articles/23/lock-free-reservations-23 https://oracle-base.com/articles/23/lock-free-reservations-2...

ffsm8 1y ago

Huh, that as not been my experience at all. Too be fair, its been something like 10 years iirc. The database in question was MySQL 8, running on plain old enterprise ssds (RAID 10) The workload was processing transactions (financial payments) The database schema was ... Let's call it questionable, with pretty much no normalization because "it's easier when we look at it for debugging", hence extremely long rows with countless updates to the same row throughout the processing, roughly 250-500 writes per row per request/transaction from what I recall. And the application was a unholy combination of a PHP+Java monolith, linked via RPC and transparent class sharing DB IO was _never_ the problem, no matter how high qps got. I can't quote an exact number, but it was definitely a lot higher then what this claims (something like 40-50k on average "load" days like pre Christmas etc) Not sure how they're getting this down to ~250qps, it sounds completely implausible. Heck, I can do single row non-stop updates with >1k qpm on my desktop on a single nvme drive - and that's not even using raid.

justinhj 1y ago
Using your numbers and speaking roughly, if you're doing 50k rps and 500 typically to the same row wasn't your contention around 1%? TigerBeetle clais to be able to handle work loads with very high contention, for example 80 or 90% of transactions filtering out a fee to a single account to make up a use case.
- ffsm8 1y ago
  Ah, you might be right! Contention is what can be parallelized, right? So with roughly 100-200 requests/s you end up with 1-0.5 contention if I understood that right. That moves me even further towards agarlands points, though - if I plug that into the equation, I end up with >50k qps. The used numbers create an insanely distorted idea wrt real world performance

mike_hearn 1y ago

It's worth noting that other databases have features designed specifically to address this use case. Oracle has lock free reservations on numeric columns: https://oracle-base.com/articles/23/lock-free-reservations-23 https://oracle-base.com/articles/23/lock-free-reservations-2...