5 ms·

I never really "got" the new wave of NoSQL databases. Mongo seemed to be the one I could most easily wrap my head around, but still. I was never sure, though,

by krylon 12y ago

I never really "got" the new wave of NoSQL databases. Mongo seemed to be the one I could most easily wrap my head around, but still. I was never sure, though, if that meant I had never faced a problem suitable for one of these DBMSs or if my mind is just so warped by years of using relational engines (mostly Postgres, or SQLite for simple projects) that I could not think of modeling my data any other way. Recently though, I had to get familiar with the database schema of the ERP system we use at work, plus some modifications that have been done to it over the years, and it kind of feels to me like somebody was trying force a square peg through a round hole (i.e. trying to model data in relational terms, either not fully "getting" the relational model or using data that simply refuses to be modeled in that way). I sometimes think the people who wrote the ERP system might have enjoyed a NoSQL DBMS. Then again, with a multi-user ERP system, you <i>really</i> want transactions (personally, I feel that ACID-compliant transactions are single most useful benefit of RDBMS engines), and most NoSQL-engines seem to kind of not have them.

sergiosgc 12y ago
I see where you are coming from, and my opinion is mostly the same. However, I can see a use case for relaxing constraints in a distributed scenario, as described by the CAP theorem. In a distributed scenario, when a partition event occurs, relational databases opt for consistency, whereas nosql opts for availability. This is formally correct behaviour by relational dbs, but comes with a cost. The cost is a serial performance component, that can't be parallelized. Nosql DBs (most of them), in this scenario, go for availability, and may thus eschew some consistency guarantee tasks, with a cost in data consistency and an advantage in parallel performance. The trick, as ever, is to use each tool in its function. Nosql and relational dbs are wholly different tools, for wholly different problem classes. Using nosql where consistency is paramount irks me to no end, and that is the case 80% of the time I see people using nosql. On the other hand, in specific cases, nosql DBs are a new useful tool in my arsenal.
romanovcode 12y ago
It is for storing huge number of data with dynamic keys. For example when site admin says "I want new archive that I can fill with items, items will have Id (automatically), Name (string), IsMale (bool)". He also want to do complex queries on this data as well. That's where NoSQL comes to help. And to answer why exactly MongoDb is so popular - it's because it has awesome driver support for every popular language. I don't understand what's so hard to understand here. It's a simple solution to EAV/nulltable nightmare.
- yangyang 12y ago
  The nice thing about using an RDBMS with JSON support, rather than a NoSQL solution, is that you can store all the fixed-schema stuff in column as usual, and benefit from the performance, consistency, ease of joins and so on with that, but you can also store your JSON documents alongside that data in the same table, efficiently indexed.
  - mikegioia 12y ago
    Yes, but what happens if your JSON data size grows so large that it can't fit on a single machine? Multi-master replication or sharding is a terrible pain in any RDBMS (at least according to my research and trials).
    - vertex-four 12y ago
      At the end of 2013, Stack Overflow worked on one SQL server (plus a redis server for caching). The rest of Stack Exchange runs on another SQL server.[0] For the most part, for most projects, worrying about multi-master replication is going to be pointless. You can always put some data in a distributed K/V (or document) store and point to that from your SQL if you need to. [0] http://nickcraver.com/blog/2013/11/22/what-it-takes-to-run-stack-overflow/ http://nickcraver.com/blog/2013/11/22/what-it-takes-to-run-s...
      - mikegioia 12y ago
        What I meant was a data-size that was too large for a machine. Adding arbitrarily large JSON to your table could expand the data-size to be too big for one machine, or even too big for block storage. Plus you might not want it all in block storage. My point is that an RDBMS can be better served storing the relational data alone with a separate DB Engine for the potentially massive JSON data.
        yangyang 12y ago
        Why does the format the data is stored in matter more than the data itself? JSON or columns in rows - it's not fundamentally different.
Nitramp 12y ago
I think people mix up three things: (1) Transactional model. Many NoSQL databases are non-ACID, but others are (Google's stores all have some transactional guarantees). Some databases try to gain efficiency by relaxing their transactional guarantees, some probably just didn't get around to implementing a proper transactional system yet. (2) Data model. The relational models can be overly restrictive as you cannot easily represent contained, repeated elements in an object without ending up in a crazy join smorgasbord. Note that that doesn't mean you have to be dynamically typed. (3) Distribution/sharding/clustering. RDBMSes are traditionally single machine, and getting them to cluster is usually a huge source of pain. NoSQL databases are often built from the ground for sharding. I think people go for MongoDB mostly for (2) and ease of use. Very few people have an actual big data problem where you really need (3), and for reliability there are simpler solutions (hot standby). (1) makes it so much easier to build reliable systems that it'd be a real deal breaker for me. I personally don't understand why so many people go for NoSQL, seems to me like that creates a substantial cost, both for performance, but more importantly missing the transaction guarantees, with no real benefit, at least none that's obvious to me. MongoDB with its unapplied writes, no real transactions, but no real distribution story seems like an odd choice in particular.
- jl6 12y ago
  > ... contained, repeated elements in an object Pardon me if I'm just sniping on the word "object" here, but if you think of your data as objects then you will find the relational model restrictive. In my experience, objects are an application concept, closely coupled to an implementation. If you can conceive of your data in implementation-independent terms, i.e. as entities and relationships, then you can put a RDBMS to effective use.
- admiun 12y ago
  You don't need big data for (3), we use mongo sharding primarily for spreading database write load. Not saying that it's the best way to do that but it's what we use it for and I doubt we're the only ones.
Shorel 12y ago
> I never really "got" the new wave of NoSQL databases. It's easy: Overzealous DBAs who insist in normalization at all costs. A new technology allows developers to try new approaches to the challenges, by sidestepping those DBAs.
- marcosdumay 12y ago
  So, a technological solution to a political problem. Now I can understand where that people are coming from.
  - spacemanmatt 12y ago
    If I hadn't seen so many dev teams eschew constraints and triggers in favor of broken client code being allowed to screw up the data, I would agree to the political element easily. But in practice, the ignorance is so great, I can't even be sure there is a political choice being made.
MoOmer 12y ago
Allow me to present Cassandra: http://planetcassandra.org/what-is-apache-cassandra/ http://planetcassandra.org/what-is-apache-cassandra/ Note that Cassandra scales linearly. http://wiki.apache.org/cassandra/ArticlesAndPresentations http://wiki.apache.org/cassandra/ArticlesAndPresentations

by krylon 12y ago

sergiosgc 12y ago

I see where you are coming from, and my opinion is mostly the same. However, I can see a use case for relaxing constraints in a distributed scenario, as described by the CAP theorem. In a distributed scenario, when a partition event occurs, relational databases opt for consistency, whereas nosql opts for availability. This is formally correct behaviour by relational dbs, but comes with a cost. The cost is a serial performance component, that can't be parallelized. Nosql DBs (most of them), in this scenario, go for availability, and may thus eschew some consistency guarantee tasks, with a cost in data consistency and an advantage in parallel performance. The trick, as ever, is to use each tool in its function. Nosql and relational dbs are wholly different tools, for wholly different problem classes. Using nosql where consistency is paramount irks me to no end, and that is the case 80% of the time I see people using nosql. On the other hand, in specific cases, nosql DBs are a new useful tool in my arsenal.

romanovcode 12y ago

It is for storing huge number of data with dynamic keys. For example when site admin says "I want new archive that I can fill with items, items will have Id (automatically), Name (string), IsMale (bool)". He also want to do complex queries on this data as well. That's where NoSQL comes to help. And to answer why exactly MongoDb is so popular - it's because it has awesome driver support for every popular language. I don't understand what's so hard to understand here. It's a simple solution to EAV/nulltable nightmare.

yangyang 12y ago
The nice thing about using an RDBMS with JSON support, rather than a NoSQL solution, is that you can store all the fixed-schema stuff in column as usual, and benefit from the performance, consistency, ease of joins and so on with that, but you can also store your JSON documents alongside that data in the same table, efficiently indexed.
- mikegioia 12y ago
  Yes, but what happens if your JSON data size grows so large that it can't fit on a single machine? Multi-master replication or sharding is a terrible pain in any RDBMS (at least according to my research and trials).
  - vertex-four 12y ago
    At the end of 2013, Stack Overflow worked on one SQL server (plus a redis server for caching). The rest of Stack Exchange runs on another SQL server.[0] For the most part, for most projects, worrying about multi-master replication is going to be pointless. You can always put some data in a distributed K/V (or document) store and point to that from your SQL if you need to. [0] http://nickcraver.com/blog/2013/11/22/what-it-takes-to-run-stack-overflow/ http://nickcraver.com/blog/2013/11/22/what-it-takes-to-run-s...
    - mikegioia 12y ago
      What I meant was a data-size that was too large for a machine. Adding arbitrarily large JSON to your table could expand the data-size to be too big for one machine, or even too big for block storage. Plus you might not want it all in block storage. My point is that an RDBMS can be better served storing the relational data alone with a separate DB Engine for the potentially massive JSON data.
      - yangyang 12y ago
        Why does the format the data is stored in matter more than the data itself? JSON or columns in rows - it's not fundamentally different.

Nitramp 12y ago

I think people mix up three things: (1) Transactional model. Many NoSQL databases are non-ACID, but others are (Google's stores all have some transactional guarantees). Some databases try to gain efficiency by relaxing their transactional guarantees, some probably just didn't get around to implementing a proper transactional system yet. (2) Data model. The relational models can be overly restrictive as you cannot easily represent contained, repeated elements in an object without ending up in a crazy join smorgasbord. Note that that doesn't mean you have to be dynamically typed. (3) Distribution/sharding/clustering. RDBMSes are traditionally single machine, and getting them to cluster is usually a huge source of pain. NoSQL databases are often built from the ground for sharding. I think people go for MongoDB mostly for (2) and ease of use. Very few people have an actual big data problem where you really need (3), and for reliability there are simpler solutions (hot standby). (1) makes it so much easier to build reliable systems that it'd be a real deal breaker for me. I personally don't understand why so many people go for NoSQL, seems to me like that creates a substantial cost, both for performance, but more importantly missing the transaction guarantees, with no real benefit, at least none that's obvious to me. MongoDB with its unapplied writes, no real transactions, but no real distribution story seems like an odd choice in particular.

jl6 12y ago
> ... contained, repeated elements in an object Pardon me if I'm just sniping on the word "object" here, but if you think of your data as objects then you will find the relational model restrictive. In my experience, objects are an application concept, closely coupled to an implementation. If you can conceive of your data in implementation-independent terms, i.e. as entities and relationships, then you can put a RDBMS to effective use.
admiun 12y ago
You don't need big data for (3), we use mongo sharding primarily for spreading database write load. Not saying that it's the best way to do that but it's what we use it for and I doubt we're the only ones.

Shorel 12y ago

> I never really "got" the new wave of NoSQL databases. It's easy: Overzealous DBAs who insist in normalization at all costs. A new technology allows developers to try new approaches to the challenges, by sidestepping those DBAs.

marcosdumay 12y ago
So, a technological solution to a political problem. Now I can understand where that people are coming from.
- spacemanmatt 12y ago
  If I hadn't seen so many dev teams eschew constraints and triggers in favor of broken client code being allowed to screw up the data, I would agree to the political element easily. But in practice, the ignorance is so great, I can't even be sure there is a political choice being made.

MoOmer 12y ago

Allow me to present Cassandra: http://planetcassandra.org/what-is-apache-cassandra/ http://planetcassandra.org/what-is-apache-cassandra/ Note that Cassandra scales linearly. http://wiki.apache.org/cassandra/ArticlesAndPresentations http://wiki.apache.org/cassandra/ArticlesAndPresentations