6 ms·

Stonebraker on CAP theorem and Databases (2010)

83 points by onurkanbkrc 9mo ago

nine_k 9mo ago
In short: eventual consistency is insufficient in many real-world error scenarios which are outside the CAP theorem. Go for full consistency where possible, which is more practical cases than normally assumed.
- candiddevmike 9mo ago
  But full consistency isn't web scale! There are a lot of times where full consistency with some kind of cache in front of it has the same client quirks as eventually consistency though. As always, the answer is "it depends".
  - sgarland 9mo ago
    Did you unironically use the term “web scale” in reference to a database?
  - DeathArrow 9mo ago
    >But full consistency isn't web scale! But /dev/null is!
redwood 9mo ago
This is why the winning disturbed systems optimize for CP. It's worth preserving consistency at the expense of rare availability losses particularly on cloud infrastructure
- pyrolistical 9mo ago
  Also, giving up on availability doesnt imply we will be down for a longtime. It’s just some requests might get dropped. As long as the client knows the request was rejected they can try later.
sethev 9mo ago
Normally, I'm not a fan of putting the date on a post. However, in this case, the fact that Stonebraker's article was published in 2010 makes it more impressive given the developments over the last 15 years - in which we've relearned the value of consistency (and the fact that it can scale more than people were imagining).
- alecco 9mo ago
  Papers by Stonebraker on that: "What goes around comes around" (2005) https://web.eecs.umich.edu/~mozafari/fall2015/eecs584/papers/whatGoesAroundComesAround.pdf https://web.eecs.umich.edu/~mozafari/fall2015/eecs584/papers... "What Goes Around Comes Around... And Around..." (2024) https://dl.acm.org/doi/pdf/10.1145/3685980.3685984 https://dl.acm.org/doi/pdf/10.1145/3685980.3685984 (ft Andy Pavlo) Or video (2025) https://www.youtube.com/watch?v=8Woy5I511L8 https://www.youtube.com/watch?v=8Woy5I511L8
  - trympet 9mo ago
    As someone not very deep into databases, these papers were fun to read.
wippler 9mo ago
FYI. This was written in 2010 although it feels relevant even now. Didn't catch it until the mention of Amazon SimpleDB.
- redwood 9mo ago
  Indeed I was wondering for a moment if amazon had decided to double down on it
onethumb 9mo ago
Probably needs a (2010) label. Great article, though.
belter 9mo ago
The 2010 is really important here. And Stonebraker is thinking about local databases systems and was a bit upset but the NoSQL movement push at the time. And he is making a mistake in claiming the partitions are "exceedingly rare". Again he is not thinking about a global distributed cloud across continents. The real world works with Eventual Consistency. Embrace it, for most 90% of the Business Scenarios its the best option: https://i.ibb.co/DtxrRH3/eventual-consistency.png https://i.ibb.co/DtxrRH3/eventual-consistency.png
- hobs 9mo ago
  I would say quite the opposite - most business have little need for eventual consistency and at a small scale its not even a requirement for any database you would reasonably used, way more than 90% of companies don't need eventual consistency.
  - belter 9mo ago
    No. The real world is full of eventual consistency, and we simply operate around it. :-) Think about a supermarket: If the store is open 24/7, prices change constantly, and some items still have the old price tag until shelves get refreshed. The system converges over time. Or airlines: They must overbook, because if they wait for perfect certainty, planes fly half empty. They accept inconsistency and correct later with compensation. Even banking works this way. All database books have the usual “you can’t debit twice, so you need transactions”…bullshit. But think of a money transfer across banks and possibly across countries? Not globally atomic... What if you transfer money to an account that was closed an hour ago in another system? The transfer doesn’t instantly fail everywhere. It’s posted as credit/debit, then reconciliation runs later, and you eventually get a reversal. Same with stock markets: Trades happen continuously, but final clearing and settlement occur after the fact. And technically DNS is eventual consistency by design. You update a record, but the world sees it gradually as caches expire. Yet the internet works. Distributed systems aren’t broken when they’re eventually consistent. They’re mirroring how real systems work: commit locally, reconcile globally, compensate when needed.
    - 9mo ago
johnmwilkinson 9mo ago
Sort of related? https://www.usenix.org/system/files/login-logout_1305_mickens.pdf https://www.usenix.org/system/files/login-logout_1305_micken...
- lencastre 9mo ago
  til, thx
dfajgljsldkjag 9mo ago
I think we try too hard to solve problems that we do not even have yet. It is much better to build a simple system that is correct than a messy one that never stops. I see people writing bad code because they are afraid of the network breaking. We should just let the database do its job.
- pyrolistical 9mo ago
  But consistency is a choice we need to make when deciding how to use a database. By letting databases do its job, you let someone else make a trade off for you
oooyay 9mo ago
A lot of these kinds of discussions tend to wipe away all the nuance around why you would or wouldn't care about consistency. Most of the answer has to do with software architecture and some of it has to do with use cases.
deleted 9mo ago
[deleted]
MORPHOICES 9mo ago
[dead]
levischoen 9mo ago
https://web.archive.org/web/20260131005358/https://perspectives.mvdirona.com/2010/04/stonebraker-on-cap-theorem-and-databases/ https://web.archive.org/web/20260131005358/https://perspecti...
thayne 9mo ago
> Hence, in my opinion, one is much better off giving up P rather than sacrificing C. (In a LAN environment, I think one should choose CA rather than AP). That isn't how it works. The only way to completely avoid network partitions is to not have a network, i.e. not have a distributed system. Sure partitions are rare, but unless your network is flawless, and your nodes never go down, they do happen sometimes, and when they do, you have to choose between consistency and availability. That said, in most cases consistency is probably what you want.

83 points by onurkanbkrc 9mo ago

In short: eventual consistency is insufficient in many real-world error scenarios which are outside the CAP theorem. Go for full consistency where possible, which is more practical cases than normally assumed.

candiddevmike 9mo ago
But full consistency isn't web scale! There are a lot of times where full consistency with some kind of cache in front of it has the same client quirks as eventually consistency though. As always, the answer is "it depends".
- sgarland 9mo ago
  Did you unironically use the term “web scale” in reference to a database?
- DeathArrow 9mo ago
  >But full consistency isn't web scale! But /dev/null is!

redwood 9mo ago

This is why the winning disturbed systems optimize for CP. It's worth preserving consistency at the expense of rare availability losses particularly on cloud infrastructure

pyrolistical 9mo ago
Also, giving up on availability doesnt imply we will be down for a longtime. It’s just some requests might get dropped. As long as the client knows the request was rejected they can try later.

sethev 9mo ago

Normally, I'm not a fan of putting the date on a post. However, in this case, the fact that Stonebraker's article was published in 2010 makes it more impressive given the developments over the last 15 years - in which we've relearned the value of consistency (and the fact that it can scale more than people were imagining).

alecco 9mo ago
Papers by Stonebraker on that: "What goes around comes around" (2005) https://web.eecs.umich.edu/~mozafari/fall2015/eecs584/papers/whatGoesAroundComesAround.pdf https://web.eecs.umich.edu/~mozafari/fall2015/eecs584/papers... "What Goes Around Comes Around... And Around..." (2024) https://dl.acm.org/doi/pdf/10.1145/3685980.3685984 https://dl.acm.org/doi/pdf/10.1145/3685980.3685984 (ft Andy Pavlo) Or video (2025) https://www.youtube.com/watch?v=8Woy5I511L8 https://www.youtube.com/watch?v=8Woy5I511L8
- trympet 9mo ago
  As someone not very deep into databases, these papers were fun to read.

wippler 9mo ago

FYI. This was written in 2010 although it feels relevant even now. Didn't catch it until the mention of Amazon SimpleDB.

redwood 9mo ago
Indeed I was wondering for a moment if amazon had decided to double down on it

onethumb 9mo ago

Probably needs a (2010) label. Great article, though.

belter 9mo ago

The 2010 is really important here. And Stonebraker is thinking about local databases systems and was a bit upset but the NoSQL movement push at the time. And he is making a mistake in claiming the partitions are "exceedingly rare". Again he is not thinking about a global distributed cloud across continents. The real world works with Eventual Consistency. Embrace it, for most 90% of the Business Scenarios its the best option: https://i.ibb.co/DtxrRH3/eventual-consistency.png https://i.ibb.co/DtxrRH3/eventual-consistency.png

hobs 9mo ago
I would say quite the opposite - most business have little need for eventual consistency and at a small scale its not even a requirement for any database you would reasonably used, way more than 90% of companies don't need eventual consistency.
- belter 9mo ago
  No. The real world is full of eventual consistency, and we simply operate around it. :-) Think about a supermarket: If the store is open 24/7, prices change constantly, and some items still have the old price tag until shelves get refreshed. The system converges over time. Or airlines: They must overbook, because if they wait for perfect certainty, planes fly half empty. They accept inconsistency and correct later with compensation. Even banking works this way. All database books have the usual “you can’t debit twice, so you need transactions”…bullshit. But think of a money transfer across banks and possibly across countries? Not globally atomic... What if you transfer money to an account that was closed an hour ago in another system? The transfer doesn’t instantly fail everywhere. It’s posted as credit/debit, then reconciliation runs later, and you eventually get a reversal. Same with stock markets: Trades happen continuously, but final clearing and settlement occur after the fact. And technically DNS is eventual consistency by design. You update a record, but the world sees it gradually as caches expire. Yet the internet works. Distributed systems aren’t broken when they’re eventually consistent. They’re mirroring how real systems work: commit locally, reconcile globally, compensate when needed.
  - 9mo ago

johnmwilkinson 9mo ago

Sort of related? https://www.usenix.org/system/files/login-logout_1305_mickens.pdf https://www.usenix.org/system/files/login-logout_1305_micken...

lencastre 9mo ago
til, thx

dfajgljsldkjag 9mo ago

I think we try too hard to solve problems that we do not even have yet. It is much better to build a simple system that is correct than a messy one that never stops. I see people writing bad code because they are afraid of the network breaking. We should just let the database do its job.

pyrolistical 9mo ago
But consistency is a choice we need to make when deciding how to use a database. By letting databases do its job, you let someone else make a trade off for you

oooyay 9mo ago

A lot of these kinds of discussions tend to wipe away all the nuance around why you would or wouldn't care about consistency. Most of the answer has to do with software architecture and some of it has to do with use cases.

deleted 9mo ago

[deleted]

MORPHOICES 9mo ago

[dead]

levischoen 9mo ago

https://web.archive.org/web/20260131005358/https://perspectives.mvdirona.com/2010/04/stonebraker-on-cap-theorem-and-databases/ https://web.archive.org/web/20260131005358/https://perspecti...

thayne 9mo ago

> Hence, in my opinion, one is much better off giving up P rather than sacrificing C. (In a LAN environment, I think one should choose CA rather than AP). That isn't how it works. The only way to completely avoid network partitions is to not have a network, i.e. not have a distributed system. Sure partitions are rare, but unless your network is flawless, and your nodes never go down, they do happen sometimes, and when they do, you have to choose between consistency and availability. That said, in most cases consistency is probably what you want.