4 ms·

First of all https://en.wikipedia.org/wiki/Consistency_(database_systems) https://en.wikipedia.org/wiki/Consistency_(database_systems) There is no strawman, pos

First of all https://en.wikipedia.org/wiki/Consistency_(database_systems) https://en.wikipedia.org/wiki/Consistency_(database_systems) There is no strawman, postgres is literally acid and mongo literally isn't, full stop. Second of all, Mongo marketing and ecosystem is foremost about making dashboard crud apps easy with not a lot of engineering. dashboards are not about point reads. Whether or not mongo is engineered for it, this is what the customers are using it for. Third, I just wrote a proposal today for a client on Mongo, not at scale, who experiences silent database failures due to lack of acid. they only noticed because Mongo sometimes tells them there are -2 people left in the room. Nobody at scale uses mongo because somewhere along the way you spend your giant bags of scale-money to hire a real engineering team to solve the silent errors and failwhales you've been suffering by migrating to an architecture that actually lets you reason about failure a little better than "it looks like something bad happened but we have no idea when how or why"

mayank 9y ago
Please read OP's comment. Postgres is not ACID with read replicas. The replication lag yields eventual consistency.
- smilliken 9y ago
  "ACID" isn't a term that used to describe distributed systems— that's physically impossible— only systems on a single machine. To say PostgreSQL isn't ACID is incorrect and disingenuous.
  - elvinyung 9y ago
    This is completely incorrect. ACID was defined upon Jim Gray's formalization of the terms "transaction", "atomicity", "consistency", and "durability" [1]. It applies regardless of however many nodes a system consists of. It cares about a system holistically, not what it's physically made up of. Gray defined them acknowledging the preexisting work on fault tolerance and high availability with replication and hot standbys, literally situations where it's worth talking about C-as-in-CAP consistency. A transaction is literally defined upon an agreement protocol like two-phase commit. Similarly, Eric Brewer literally defined the C in CAP following the C in ACID [2]. The two concepts are clearly related, if not identical. It's not like I magically can't ever have ACID transactions once a network is involved, as long as the network heals. To come at this from another angle: ACID transactions are an abstraction. They serve only to make it easier to reason about concurrency and synchronization. If you're telling me that this abstraction falls apart inherently because of the network, then frankly it's not a very good abstraction at all. PostgreSQL is ACID when it's a system that encompasses a single node. PostgreSQL is not ACID when it's a system that encompasses more than one node. Simple as that. I think it is even more disingenuous to lull people into a false sense of security just because Postgres is technically "consistent" on a single node. It's 2017 -- your data getting fsync'ed to a single disk, in a single machine on a single rack in a single datacenter in a single region -- is simply not enough anymore. [1] http://jimgray.azurewebsites.net/papers/thetransactionconcept.pdf http://jimgray.azurewebsites.net/papers/thetransactionconcep... [2] https://pdfs.semanticscholar.org/5015/8bc1a8a67295ab7bce0550886a9859000dc2.pdf https://pdfs.semanticscholar.org/5015/8bc1a8a67295ab7bce0550...
    - empthought 9y ago
      > PostgreSQL is not ACID when it's a system that encompasses more than one node. Simple as that. Pretty sure Postgres has supported two-phase commit since at least version 8.
      - elvinyung 9y ago
        And there's also quorum commits too! Point is, these features aren't on by default, so you really have to know what you're doing, and not just assume you're safe because of the magic word "ACID".
        empthought 9y ago
        Vs. MongoDB, which doesn't support any of that; I think it's fair to draw a distinction even if people think they're using it when they're not.
        elvinyung 9y ago
        MongoDB is actually strongly consistent in a single document, though. It uses Raft by default to replicate changes to all replicas. In practice, it has the A, C, and D of ACID for point writes.
        empthought 9y ago
        Those are really some low expectations. The whole point of the ACID terminology is to apply it to transactions of arbitrary complexity with respect to the underlying data store. It's not saying much to say that a database management system is ACID as long as you constrain your updates to a single record in a single table.
        elvinyung 9y ago
        Sure. I actually agree, and like I said elsewhere in this thread, Postgres is my favorite database, and using it is something that I'd like to do for as long as possible. MongoDB's main selling point, after all, is expediency. What I'm tired of is the rhetoric built upon the dichotomy that Postgres is ACID and MongoDB/NoSQL is not. As it stands, neither is, in any scalable fashion. And in no way is Postgres a panacea for every possible kind of data problem. And like you said. It's not saying much to say that a database management system is ACID as long as you constrain your updates to a single machine. There are better things on the horizon, databases that can actually do it all.
  - mayank 9y ago
    I'm afraid we're in violent agreement. I never said "Postgres isn't ACID", that's never been in dispute. I said that you can't treat Postgres with a read replica as an ACID database, because as you and I both pointed out, it's then a distributed system (no self respecting, single-node databases is eventually consistent, so my use of that term should have implied a distributed system).
elvinyung 9y ago
First of all https://en.wikipedia.org/wiki/CAP_theorem https://en.wikipedia.org/wiki/CAP_theorem, and that's something that you of all people should know. (Or even https://en.wikipedia.org/wiki/Consistency_(database_systems)#As_a_CAP_trade-off https://en.wikipedia.org/wiki/Consistency_(database_systems)... in the article that you linked.) Please read the rest of my previous comment. Repeat after me: PostgreSQL isn't ACID in a distributed system. PostgreSQL isn't actually ACID when your database is multiple machines. PostgreSQL isn't actually ACID when you set up your first read replica on RDS. PostgreSQL isn't actually ACID when you use streaming or logical replication. PostgreSQL isn't actually ACID when you start sharding your users table. PostgreSQL isn't actually ACID in a world in which people need to scale out from one big box. It's eventually consistent, which is even less than I can say about MongoDB on a single-document level. Personally, PostgreSQL is my favorite DBMS. It's well-engineered, extremely mature, and has lots of great features. Working seamlessly and providing strong guarantees outside a single box, unfortunately, isn't one of them. Not yet, at least. To address the rest of your points: those people are using MongoDB wrong. PostgreSQL basically stops working well too, once you start mixing transaction processing and business reporting and BI in the same box at scale. If you're going to claim that MongoDB is bad because it's not ACID or somesuch, you better hold PostgreSQL to the same standard, and understand precisely why it is that the two systems were built so differently.
- dustingetz 9y ago
  ACID's C is about the write side: "ACID is a set of properties of database transactions" (wikipedia) which is different than CAP's C: https://en.wikipedia.org/wiki/Consistency_(database_systems)#As_a_CAP_trade-off https://en.wikipedia.org/wiki/Consistency_(database_systems)... Postgresql with read replicas is ACID-Consistent transactions because single writer, and CAP-AP eventually consistent because distributed reads. It is easy to agree with you here: "PostgreSQL basically stops working well too, once you start mixing transaction processing and business reporting and BI in the same box at scale." It's worse than that - the app has to be coded before it can be scaled so you get fucked by complexity-scale via ORM a lot sooner than bigdata-scale. But as you sort of say, the point of a data warehouse is to be cap-AP (eventually consistent) through read-replicas without breaking ACID-consistent transactions for the system as a whole. as to the argument "everything is bad at scale" i think that has merit too, I just gave a talk at ClojureNYC about building real apps with Postgres (tables) vs Neo4J (graph) vs Mongo (documents) vs Datomic (eavt): https://github.com/hyperfiddle/hypercrud.browser/issues/4 https://github.com/hyperfiddle/hypercrud.browser/issues/4 (Did I say real "apps"? I meant "balls of mud")

mayank 9y ago

Please read OP's comment. Postgres is not ACID with read replicas. The replication lag yields eventual consistency.

smilliken 9y ago
"ACID" isn't a term that used to describe distributed systems— that's physically impossible— only systems on a single machine. To say PostgreSQL isn't ACID is incorrect and disingenuous.
- elvinyung 9y ago
  This is completely incorrect. ACID was defined upon Jim Gray's formalization of the terms "transaction", "atomicity", "consistency", and "durability" [1]. It applies regardless of however many nodes a system consists of. It cares about a system holistically, not what it's physically made up of. Gray defined them acknowledging the preexisting work on fault tolerance and high availability with replication and hot standbys, literally situations where it's worth talking about C-as-in-CAP consistency. A transaction is literally defined upon an agreement protocol like two-phase commit. Similarly, Eric Brewer literally defined the C in CAP following the C in ACID [2]. The two concepts are clearly related, if not identical. It's not like I magically can't ever have ACID transactions once a network is involved, as long as the network heals. To come at this from another angle: ACID transactions are an abstraction. They serve only to make it easier to reason about concurrency and synchronization. If you're telling me that this abstraction falls apart inherently because of the network, then frankly it's not a very good abstraction at all. PostgreSQL is ACID when it's a system that encompasses a single node. PostgreSQL is not ACID when it's a system that encompasses more than one node. Simple as that. I think it is even more disingenuous to lull people into a false sense of security just because Postgres is technically "consistent" on a single node. It's 2017 -- your data getting fsync'ed to a single disk, in a single machine on a single rack in a single datacenter in a single region -- is simply not enough anymore. [1] http://jimgray.azurewebsites.net/papers/thetransactionconcept.pdf http://jimgray.azurewebsites.net/papers/thetransactionconcep... [2] https://pdfs.semanticscholar.org/5015/8bc1a8a67295ab7bce0550886a9859000dc2.pdf https://pdfs.semanticscholar.org/5015/8bc1a8a67295ab7bce0550...
  - empthought 9y ago
    > PostgreSQL is not ACID when it's a system that encompasses more than one node. Simple as that. Pretty sure Postgres has supported two-phase commit since at least version 8.
    - elvinyung 9y ago
      And there's also quorum commits too! Point is, these features aren't on by default, so you really have to know what you're doing, and not just assume you're safe because of the magic word "ACID".
      - empthought 9y ago
        Vs. MongoDB, which doesn't support any of that; I think it's fair to draw a distinction even if people think they're using it when they're not.
        elvinyung 9y ago
        MongoDB is actually strongly consistent in a single document, though. It uses Raft by default to replicate changes to all replicas. In practice, it has the A, C, and D of ACID for point writes.
        empthought 9y ago
        Those are really some low expectations. The whole point of the ACID terminology is to apply it to transactions of arbitrary complexity with respect to the underlying data store. It's not saying much to say that a database management system is ACID as long as you constrain your updates to a single record in a single table.
        elvinyung 9y ago
        Sure. I actually agree, and like I said elsewhere in this thread, Postgres is my favorite database, and using it is something that I'd like to do for as long as possible. MongoDB's main selling point, after all, is expediency. What I'm tired of is the rhetoric built upon the dichotomy that Postgres is ACID and MongoDB/NoSQL is not. As it stands, neither is, in any scalable fashion. And in no way is Postgres a panacea for every possible kind of data problem. And like you said. It's not saying much to say that a database management system is ACID as long as you constrain your updates to a single machine. There are better things on the horizon, databases that can actually do it all.
- mayank 9y ago
  I'm afraid we're in violent agreement. I never said "Postgres isn't ACID", that's never been in dispute. I said that you can't treat Postgres with a read replica as an ACID database, because as you and I both pointed out, it's then a distributed system (no self respecting, single-node databases is eventually consistent, so my use of that term should have implied a distributed system).

elvinyung 9y ago

First of all https://en.wikipedia.org/wiki/CAP_theorem https://en.wikipedia.org/wiki/CAP_theorem, and that's something that you of all people should know. (Or even https://en.wikipedia.org/wiki/Consistency_(database_systems)#As_a_CAP_trade-off https://en.wikipedia.org/wiki/Consistency_(database_systems)... in the article that you linked.) Please read the rest of my previous comment. Repeat after me: PostgreSQL isn't ACID in a distributed system. PostgreSQL isn't actually ACID when your database is multiple machines. PostgreSQL isn't actually ACID when you set up your first read replica on RDS. PostgreSQL isn't actually ACID when you use streaming or logical replication. PostgreSQL isn't actually ACID when you start sharding your users table. PostgreSQL isn't actually ACID in a world in which people need to scale out from one big box. It's eventually consistent, which is even less than I can say about MongoDB on a single-document level. Personally, PostgreSQL is my favorite DBMS. It's well-engineered, extremely mature, and has lots of great features. Working seamlessly and providing strong guarantees outside a single box, unfortunately, isn't one of them. Not yet, at least. To address the rest of your points: those people are using MongoDB wrong. PostgreSQL basically stops working well too, once you start mixing transaction processing and business reporting and BI in the same box at scale. If you're going to claim that MongoDB is bad because it's not ACID or somesuch, you better hold PostgreSQL to the same standard, and understand precisely why it is that the two systems were built so differently.

dustingetz 9y ago
ACID's C is about the write side: "ACID is a set of properties of database transactions" (wikipedia) which is different than CAP's C: https://en.wikipedia.org/wiki/Consistency_(database_systems)#As_a_CAP_trade-off https://en.wikipedia.org/wiki/Consistency_(database_systems)... Postgresql with read replicas is ACID-Consistent transactions because single writer, and CAP-AP eventually consistent because distributed reads. It is easy to agree with you here: "PostgreSQL basically stops working well too, once you start mixing transaction processing and business reporting and BI in the same box at scale." It's worse than that - the app has to be coded before it can be scaled so you get fucked by complexity-scale via ORM a lot sooner than bigdata-scale. But as you sort of say, the point of a data warehouse is to be cap-AP (eventually consistent) through read-replicas without breaking ACID-consistent transactions for the system as a whole. as to the argument "everything is bad at scale" i think that has merit too, I just gave a talk at ClojureNYC about building real apps with Postgres (tables) vs Neo4J (graph) vs Mongo (documents) vs Datomic (eavt): https://github.com/hyperfiddle/hypercrud.browser/issues/4 https://github.com/hyperfiddle/hypercrud.browser/issues/4 (Did I say real "apps"? I meant "balls of mud")