4 ms·

I love me some document based databases. But I find it ridiculous how much FUD there is out there about them. After being severely burned by some "It's not full

I love me some document based databases. But I find it ridiculous how much FUD there is out there about them. After being severely burned by some "It's not fully ACID" types in my organization I am seriously reluctant to even use them on projects where they are ideally suited. Sorry for the rant. I think they are wonderful tools, but I also think people should know about the political implications of using them in organizations where 'old fashioned' or conservative thinking might come in and ruin their day.

ppj606 12y ago
I get your point of view. My hypothesis for this behaviour, or part of the reason for the behaviour, is that I think the education system has traditionally been geared to teaching about SQL systems over their NoSQL counter-parts. This creates developers with strong opinions of one, without necessarily having a sound understanding of the other(s). The people who cause pain in your organisation are almost definitely committing a logical fallacy - see [https://yourlogicalfallacyis.com/ https://yourlogicalfallacyis.com/]
matthewmacleod 12y ago
I'm not sure the level of FUD is unjustified. I've seen far, far more cases of document databases being used in situations that are entirely inappropriate, than in situations where they are ideally suited. I'd argue that the situations in which it's a good choice to use a document database are pretty rare, and that you're almost always better off using a bog-standard relational database.
- mberning 12y ago
  I think many problems could be solved equally well using either technology IF you understand the strengths and weaknesses of each technology and implement them appropriately. It just so happens that knowledge of how to use a RDBMS appropriately is MUCH more common than document databases.
- ppj606 12y ago
  With the rise of realtime applications I'd argue that the situations for using NoSQL for the common project are growing. I think the author of the post has made a good point, that many seem to forget - it is always a trade-off whichever tool you use.
  - moe 12y ago
    With the rise of realtime applications What does "realtime" have to do with the datastore?
    - ppj606 12y ago
      The usage and access of data in your data store when building a realtime application. If you were building a chat-app these days you are given a number of very easy to adopt (minimal complexity/edge cases) NoSQL solutions - Firebase, Parse, RethinkDB. Whereas with SQL you have the option of using cache layers (which you could say is NoSQL anyway), message queues, naive timely polling and log monitoring. With NoSQL the solutions are just simpler. For example Firebase has a tutorial for creating a real-time chat [1] - it takes 5 minutes. [1] https://www.firebase.com/tutorial/#example/chat https://www.firebase.com/tutorial/#example/chat
theseoafs 12y ago
If people in your organization are concerned about document-based databases because "they're not fully ACID", they're right. If ACID guarantees are important for your use case, then that automatically nullifies all the other hypothetical advantages of NoSQL databases.
takeda 12y ago
The ACID vs non ACID all comes down to: can you afford to have invalid/missing data in your program? The answer to this question really depends on the project you are working on. If you think you don't need ACID even though your data is very important to you then I recommend you to read some great Jepsen articles series by Kyle Kingsbury[1][2] showing how popular distributed systems lose data on during network partition. If you don't believe it you can download the tests and run them yourself. Overall it's a very good read especially since he also tries to explain why they fail. [1] https://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-and-the-perils-of-network-partitions https://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-a... [2] https://aphyr.com/tags/jepsen https://aphyr.com/tags/jepsen
- pbowyer 12y ago
  I remember a comment on HN sometime in the last month [1] talking about how even an ACID-compliant RDBMS can be rendered non-ACID, through configuration or real-world usage (replication?). Do you agree? [1] Unfortunately, searching hasn't turned up the comment... yet.
  - takeda 12y ago
    Yep. In Postgres for example: * A - you have the ability to not use transactions (technically it still implicitly uses transactions per statement, but that's kind of expected behavior of the NoSQL databases). * C - simply don't use rules such as constraints, triggers etc. * I - reduce isolation level of the transaction. Postgres due to its design doesn't allow you to have dirty reads (ability to read from concurrent uncommitted transaction) * D - there are several settings (e.g. disable sync, wal etc, create an unlogged table) which increase performance at the cost of potentially losing your data. You will also reduce it when you use asynchronous replication. The ACID is a feature that is generally desired when your data is important to you and you also want to be certain that the data is always consistent at any point in time. It's quite hard to implement it right, so it should not be treated like a bug. Generally you should think of RDBMS as general purpose databases that have strong consistency guarantees, and the NoSQL as a specialized databases which no longer provide ACID guarantees in order to tackle specific domain. There are also NoSQL databases which try to be general purpose, such as MongoDB. Those are joke. MongoDB for example currently not only is much slower than Postgres[1] it also doesn't horizontally scale[2] on top of that you don't even have ACID guarantees so you're getting the worst out of both worlds. [1] http://www.enterprisedb.com/postgres-plus-edb-blog/marc-linster/postgres-outperforms-mongodb-and-ushers-new-developer-reality http://www.enterprisedb.com/postgres-plus-edb-blog/marc-lins... [2] https://www.datastax.com/wp-content/uploads/2013/02/WP-Benchmarking-Top-NoSQL-Databases.pdf https://www.datastax.com/wp-content/uploads/2013/02/WP-Bench...

ppj606 12y ago

I get your point of view. My hypothesis for this behaviour, or part of the reason for the behaviour, is that I think the education system has traditionally been geared to teaching about SQL systems over their NoSQL counter-parts. This creates developers with strong opinions of one, without necessarily having a sound understanding of the other(s). The people who cause pain in your organisation are almost definitely committing a logical fallacy - see [https://yourlogicalfallacyis.com/ https://yourlogicalfallacyis.com/]

matthewmacleod 12y ago

I'm not sure the level of FUD is unjustified. I've seen far, far more cases of document databases being used in situations that are entirely inappropriate, than in situations where they are ideally suited. I'd argue that the situations in which it's a good choice to use a document database are pretty rare, and that you're almost always better off using a bog-standard relational database.

mberning 12y ago
I think many problems could be solved equally well using either technology IF you understand the strengths and weaknesses of each technology and implement them appropriately. It just so happens that knowledge of how to use a RDBMS appropriately is MUCH more common than document databases.
ppj606 12y ago
With the rise of realtime applications I'd argue that the situations for using NoSQL for the common project are growing. I think the author of the post has made a good point, that many seem to forget - it is always a trade-off whichever tool you use.
- moe 12y ago
  With the rise of realtime applications What does "realtime" have to do with the datastore?
  - ppj606 12y ago
    The usage and access of data in your data store when building a realtime application. If you were building a chat-app these days you are given a number of very easy to adopt (minimal complexity/edge cases) NoSQL solutions - Firebase, Parse, RethinkDB. Whereas with SQL you have the option of using cache layers (which you could say is NoSQL anyway), message queues, naive timely polling and log monitoring. With NoSQL the solutions are just simpler. For example Firebase has a tutorial for creating a real-time chat [1] - it takes 5 minutes. [1] https://www.firebase.com/tutorial/#example/chat https://www.firebase.com/tutorial/#example/chat

theseoafs 12y ago

If people in your organization are concerned about document-based databases because "they're not fully ACID", they're right. If ACID guarantees are important for your use case, then that automatically nullifies all the other hypothetical advantages of NoSQL databases.

takeda 12y ago

The ACID vs non ACID all comes down to: can you afford to have invalid/missing data in your program? The answer to this question really depends on the project you are working on. If you think you don't need ACID even though your data is very important to you then I recommend you to read some great Jepsen articles series by Kyle Kingsbury[1][2] showing how popular distributed systems lose data on during network partition. If you don't believe it you can download the tests and run them yourself. Overall it's a very good read especially since he also tries to explain why they fail. [1] https://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-and-the-perils-of-network-partitions https://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-a... [2] https://aphyr.com/tags/jepsen https://aphyr.com/tags/jepsen

pbowyer 12y ago
I remember a comment on HN sometime in the last month [1] talking about how even an ACID-compliant RDBMS can be rendered non-ACID, through configuration or real-world usage (replication?). Do you agree? [1] Unfortunately, searching hasn't turned up the comment... yet.
- takeda 12y ago
  Yep. In Postgres for example: * A - you have the ability to not use transactions (technically it still implicitly uses transactions per statement, but that's kind of expected behavior of the NoSQL databases). * C - simply don't use rules such as constraints, triggers etc. * I - reduce isolation level of the transaction. Postgres due to its design doesn't allow you to have dirty reads (ability to read from concurrent uncommitted transaction) * D - there are several settings (e.g. disable sync, wal etc, create an unlogged table) which increase performance at the cost of potentially losing your data. You will also reduce it when you use asynchronous replication. The ACID is a feature that is generally desired when your data is important to you and you also want to be certain that the data is always consistent at any point in time. It's quite hard to implement it right, so it should not be treated like a bug. Generally you should think of RDBMS as general purpose databases that have strong consistency guarantees, and the NoSQL as a specialized databases which no longer provide ACID guarantees in order to tackle specific domain. There are also NoSQL databases which try to be general purpose, such as MongoDB. Those are joke. MongoDB for example currently not only is much slower than Postgres[1] it also doesn't horizontally scale[2] on top of that you don't even have ACID guarantees so you're getting the worst out of both worlds. [1] http://www.enterprisedb.com/postgres-plus-edb-blog/marc-linster/postgres-outperforms-mongodb-and-ushers-new-developer-reality http://www.enterprisedb.com/postgres-plus-edb-blog/marc-lins... [2] https://www.datastax.com/wp-content/uploads/2013/02/WP-Benchmarking-Top-NoSQL-Databases.pdf https://www.datastax.com/wp-content/uploads/2013/02/WP-Bench...