4 ms·

I disagree. If we are talking about distributed systems in which we have N different databases, then distributed transactions are left as an exercise to the rea

by dakiol 1y ago

I disagree. If we are talking about distributed systems in which we have N different databases, then distributed transactions are left as an exercise to the reader (that’s why we have things like Sagas). Within a single machine, yeah, relational dbs still work like a charm.

Normal_gaussian 1y ago
The other interesting thing to consider is that machines are a lot more powerful in all dimensions than they were 30 years ago - so many tasks that would have required distributed systems no longer do.
- jorangreef 1y ago
  DBMS design has also changed. The bottleneck for a DBMS has moved from spinning disk to CPU and per core memory bandwidth. That's also key to TigerBeetle's design. For example, with static allocation and zero-copy techniques such as zero-deserialization. If you were to design an OLGP DBMS like Postgres today, it would look radically different. Same is true for OLTP.
- jandrewrogers 1y ago
  The caveat is that the hardware constraints have fundamentally changed, so database architectures from 30 years ago are effectively incapable of properly using modern hardware environments. Databases have not been bottlenecked on storage bandwidth in a long time but most databases are designed as if this was still the case. Optimizing for memory bandwidth, the current bottleneck, leads to substantially different architectures than are commonly deployed.
big_hacker 1y ago
If it's gonna fit on one machine I'm picking Postgres, Mongo or Redis before TigerBeetle and let's be honest it's not the difference between the 4 that will make it not fit on one machine.
- jorangreef 1y ago
  And to be fair, those are all great general purpose solutions. For example, you could also use Postgres or Redis as queue, instead of say Redpanda or Kafka. But our customers need separation of concerns in their architecture. While they could put the cash in the general purpose filing cabinet, they actually want the separation of concerns between OLGP system of reference (string database, i.e. PG) in the control plane, and OLTP system of record (integer/counting database, i.e. TB) in the data plane.
pjmlp 1y ago
I was part of a project that successfully deployed them into production using Oracle, sometimes those license costs are actually worth it.
- eitland 1y ago
  Agree. But for anyone tempted by Oracle, do remember that the upfront, agreed licence costs are only a fraction of the true price: You’ll need someone who actually knows Oracle - either already in place or willing to invest a serious amount of time learning it. Without that, you’re almost certainly better off choosing a simpler database that people are comfortable with. There’s always a non-zero risk of an Oracle rep trying to “upsell” you. And by upsell, I mean threatening legal action unless you cough up for additional bits a new salesperson has suddenly decided you’re using. (A company I worked with sold Oracle licences and had a long, happy relationship with them - until one day they went after us over some development databases. Someone higher up at Oracle smoothed it over, but the whole experience was unnerving enough.) Incidental and accidental complexity: I’ve worked with Windows from 3.1 through to Server 2008, with Linux from early Red Hat in 2001 through to the latest production distros, plus a fair share of weird and wonderful applications, everything from 1980s/90s radar acquisition running on 2010 operating systems through a wide range of in house, commercial and open source software and up to modern microservices — and none of it comes close to Oracle’s level of pain. Edit: Installating Delphi 6 with 14 packages came close, I used 3 days when I had to find every package scattered on disks in shelves and drawers and across ancient web paces + posted as abandonware on source forge but I guess I could learn to do that in a day if I had to do it twice a month. Oracle consistently took me 3 days - if I did everything correct on first try and didn't have to start from scratch.
  - MobiusHorizons 1y ago
    Wait, are you saying that oracle lets you use features you don’t have a license for, but then threatens to sue you for using them? I guess I shouldn’t be surprised given oracle’s business model, but I am surprised they let you use the feature in the first place.
sgarland 1y ago
Tbf, this is a natural consequence of companies doing premature optimization / cargo culting. There are many, many reasons to NOT go into microservices. Kelsey Hightower had some thoughts on this [0] back in 2020. You can run a hell of a lot off of a small fleet of beefy servers fronted with a load balancer, all pointing to one DB cluster. [0]: https://changelog.com/posts/monoliths-are-the-future https://changelog.com/posts/monoliths-are-the-future

by dakiol 1y ago

Normal_gaussian 1y ago

The other interesting thing to consider is that machines are a lot more powerful in all dimensions than they were 30 years ago - so many tasks that would have required distributed systems no longer do.

jorangreef 1y ago
DBMS design has also changed. The bottleneck for a DBMS has moved from spinning disk to CPU and per core memory bandwidth. That's also key to TigerBeetle's design. For example, with static allocation and zero-copy techniques such as zero-deserialization. If you were to design an OLGP DBMS like Postgres today, it would look radically different. Same is true for OLTP.
jandrewrogers 1y ago
The caveat is that the hardware constraints have fundamentally changed, so database architectures from 30 years ago are effectively incapable of properly using modern hardware environments. Databases have not been bottlenecked on storage bandwidth in a long time but most databases are designed as if this was still the case. Optimizing for memory bandwidth, the current bottleneck, leads to substantially different architectures than are commonly deployed.

big_hacker 1y ago

If it's gonna fit on one machine I'm picking Postgres, Mongo or Redis before TigerBeetle and let's be honest it's not the difference between the 4 that will make it not fit on one machine.

jorangreef 1y ago
And to be fair, those are all great general purpose solutions. For example, you could also use Postgres or Redis as queue, instead of say Redpanda or Kafka. But our customers need separation of concerns in their architecture. While they could put the cash in the general purpose filing cabinet, they actually want the separation of concerns between OLGP system of reference (string database, i.e. PG) in the control plane, and OLTP system of record (integer/counting database, i.e. TB) in the data plane.

pjmlp 1y ago

I was part of a project that successfully deployed them into production using Oracle, sometimes those license costs are actually worth it.

eitland 1y ago
Agree. But for anyone tempted by Oracle, do remember that the upfront, agreed licence costs are only a fraction of the true price: You’ll need someone who actually knows Oracle - either already in place or willing to invest a serious amount of time learning it. Without that, you’re almost certainly better off choosing a simpler database that people are comfortable with. There’s always a non-zero risk of an Oracle rep trying to “upsell” you. And by upsell, I mean threatening legal action unless you cough up for additional bits a new salesperson has suddenly decided you’re using. (A company I worked with sold Oracle licences and had a long, happy relationship with them - until one day they went after us over some development databases. Someone higher up at Oracle smoothed it over, but the whole experience was unnerving enough.) Incidental and accidental complexity: I’ve worked with Windows from 3.1 through to Server 2008, with Linux from early Red Hat in 2001 through to the latest production distros, plus a fair share of weird and wonderful applications, everything from 1980s/90s radar acquisition running on 2010 operating systems through a wide range of in house, commercial and open source software and up to modern microservices — and none of it comes close to Oracle’s level of pain. Edit: Installating Delphi 6 with 14 packages came close, I used 3 days when I had to find every package scattered on disks in shelves and drawers and across ancient web paces + posted as abandonware on source forge but I guess I could learn to do that in a day if I had to do it twice a month. Oracle consistently took me 3 days - if I did everything correct on first try and didn't have to start from scratch.
- MobiusHorizons 1y ago
  Wait, are you saying that oracle lets you use features you don’t have a license for, but then threatens to sue you for using them? I guess I shouldn’t be surprised given oracle’s business model, but I am surprised they let you use the feature in the first place.

sgarland 1y ago

Tbf, this is a natural consequence of companies doing premature optimization / cargo culting. There are many, many reasons to NOT go into microservices. Kelsey Hightower had some thoughts on this [0] back in 2020. You can run a hell of a lot off of a small fleet of beefy servers fronted with a load balancer, all pointing to one DB cluster. [0]: https://changelog.com/posts/monoliths-are-the-future https://changelog.com/posts/monoliths-are-the-future