5 ms·

What exactly is a "financial accounting database"? I'm sort of surprised that there's a market for a DB targeting a very specific workflow. I would have thought

What exactly is a "financial accounting database"? I'm sort of surprised that there's a market for a DB targeting a very specific workflow. I would have thought that if you need that volume of specific transactions (major bank, I guess?) you'd want a full RDBMS for things other than simply the transactions, too. Basically, that Oracle would be all over that. I love the pursuit of performance, but who would actually need 1 million journal entries a second (from their home page)? Is that the bottleneck somewhere? Apologies for the stupid questions, I just... have no idea where this thing (which seems pretty technically neat) is actually intended to be used.

jorangreef 5y ago
Thanks for the questions! It's a database with purpose-built primitives for recording financial transactions between accounts, particularly two-phase commit transactions between different payment systems run by different operators. Over the past decade or so, some of our team saw that these primitives kept getting re-implemented over and over again, often with duct tape ad-hoc solutions that are not always safe, or scalable. At the same time, the payments world is rapidly changing. Things are moving to high volume, low value digital transactions, which means that operational cost becomes important. We extracted the design for TigerBeetle out of a real-world payment switch, so that TigerBeetle can be transplanted into these kinds of systems very easily, and in a way that's very hard to get wrong because the interface maps one-to-one to the domain. We also solve a storage fault model that's relatively new in the research literature, something that most traditional databases were never designed for, and we do this because this makes for a safer system of record, not to mention a better operating experience. If anything, we place the pursuit of safety much higher than the pursuit of performance. We work much harder at the safety, the performance came pretty easily with the design right from the beginning. So the safety of TigerBeetle is what we're most excited about. The performance is a nice-to-have to go with it. These are hopefully "the things that don't change" when it comes to databases. Nobody wanted a less convenient, or more dangerous, or slower database.
mxfh 5y ago
"1,000,000 journal entries per second on consumer-grade hardware." Of cause you could just add a few cents more per device hardware to not get a rock bottom clock. Or add a temperature Sensor and start modeling a drift compensation: https://blog.dan.drown.org/rtc-comparison/ https://blog.dan.drown.org/rtc-comparison/ Or why reinvent a homebrew version of what looks like PTP IEEE 1588: https://blog.meinbergglobal.com/2013/09/14/ieee-1588-accurate/ https://blog.meinbergglobal.com/2013/09/14/ieee-1588-accurat...
- jorangreef 5y ago
  The reason is because this is not in fact a substitute for the clock sync service (whether that's PTP, or something that's becoming popular like Chrony). It's complementary, a clock fault failure detector. We're not trying to synchronize clocks, merely to detect when too many clocks in the cluster no longer agree. TigerBeetle needs to detect when the network is partitioned in a way such that the clock sync service (PTP or NTP etc.) is not functioning while the TigerBeetle cluster still has a functioning majority on the other side of the partition. No matter the clock sync service you use, you still need some way to verify that the clock sync is in fact happening, which is what this is. Hope that clarifies it for you!

jorangreef 5y ago

Thanks for the questions! It's a database with purpose-built primitives for recording financial transactions between accounts, particularly two-phase commit transactions between different payment systems run by different operators. Over the past decade or so, some of our team saw that these primitives kept getting re-implemented over and over again, often with duct tape ad-hoc solutions that are not always safe, or scalable. At the same time, the payments world is rapidly changing. Things are moving to high volume, low value digital transactions, which means that operational cost becomes important. We extracted the design for TigerBeetle out of a real-world payment switch, so that TigerBeetle can be transplanted into these kinds of systems very easily, and in a way that's very hard to get wrong because the interface maps one-to-one to the domain. We also solve a storage fault model that's relatively new in the research literature, something that most traditional databases were never designed for, and we do this because this makes for a safer system of record, not to mention a better operating experience. If anything, we place the pursuit of safety much higher than the pursuit of performance. We work much harder at the safety, the performance came pretty easily with the design right from the beginning. So the safety of TigerBeetle is what we're most excited about. The performance is a nice-to-have to go with it. These are hopefully "the things that don't change" when it comes to databases. Nobody wanted a less convenient, or more dangerous, or slower database.

mxfh 5y ago

"1,000,000 journal entries per second on consumer-grade hardware." Of cause you could just add a few cents more per device hardware to not get a rock bottom clock. Or add a temperature Sensor and start modeling a drift compensation: https://blog.dan.drown.org/rtc-comparison/ https://blog.dan.drown.org/rtc-comparison/ Or why reinvent a homebrew version of what looks like PTP IEEE 1588: https://blog.meinbergglobal.com/2013/09/14/ieee-1588-accurate/ https://blog.meinbergglobal.com/2013/09/14/ieee-1588-accurat...

jorangreef 5y ago
The reason is because this is not in fact a substitute for the clock sync service (whether that's PTP, or something that's becoming popular like Chrony). It's complementary, a clock fault failure detector. We're not trying to synchronize clocks, merely to detect when too many clocks in the cluster no longer agree. TigerBeetle needs to detect when the network is partitioned in a way such that the clock sync service (PTP or NTP etc.) is not functioning while the TigerBeetle cluster still has a functioning majority on the other side of the partition. No matter the clock sync service you use, you still need some way to verify that the clock sync is in fact happening, which is what this is. Hope that clarifies it for you!