7 ms·

This is the problem with most of the guys who go with MongoDB. Obviously, this person is very technical, so I am not flaming him nor accusing him, but this is m

by neya 13y ago

This is the problem with most of the guys who go with MongoDB. Obviously, this person is very technical, so I am not flaming him nor accusing him, but this is my view of the rest of them who pick MongoDB without exactly having a clue as to why (hipsters) or when they should use a NoSQL db and when they shouldn't. I do not hesitate to admit that I was a hipster sometime back too. I chose MongoDB for many of my projects and it went well, till it reached some kind of moderate scale where I realized it was a terrible choice going with a NoSQL db (Sometimes, I'd have to duplicate data because there were no Joins, etc). And that's when you start to realize, that NoSQL is not a pure-white solution. It is designed to satisfy very specific use-cases. Relational databases are really good enough for 99% of the use cases out there. Unless otherwise you are COMPLETELY unable to design your schema in a relational database, you SHOULD NOT simply opt for a NoSQL database. The claimed NoSQL performance benefits will easily be outrun by a terribly designed schema, if you use the wrong db for the wrong scenario. Trust me, MySQL has had so much negativity because of these hipsters, but even something as basically relational as MySQL scales really reaaallllly well. Infact, many top guys still use MySQL till date, for a reason, in production.[1] Next time you launch your start-up, spend some time carefully evaluating your db design decisions, as the wrong db for the wrong use-case could easily become the most expensive mistake of your startup. [1]http://www.quora.com/Quora-Infrastructure/Why-does-Quora-use-MySQL-as-the-data-store-instead-of-NoSQLs-such-as-Cassandra-MongoDB-or-CouchDB http://www.quora.com/Quora-Infrastructure/Why-does-Quora-use...

mattzito 13y ago
I used to spend a lot of time in the RDBMS space, and I'd hear the DBAs bitch about how NoSQL was just a bullshit excuse for not having any discipline around your data model, and I'd defend it, saying, "Well, you have to understand, certain use cases, etc. etc. etc." And then I see an presentation like this and think, "Yes, of course, you should have put that stuff in a relational database, what were you thinking". Of course, most of the open source databases do sharding and replication and the like equally terribly, so it's not a perfect solution....
- Xorlev 13y ago
  I tend to take the view of RDBMS works until it doesn't. Most cases that things like click data or event data that ends up taking billions of rows, in which case you aren't going to want it in a RDBMS anyways (you'd like to, but hah). I'd like LevelDB on MySQL personally.
tracker1 13y ago
I have to completely agree... in many cases a classic SQL RDBMS is probably best. In some (such as single records having very dynamically shaped data, where most instances can be brought forward based on a key alone, and searches are on a few keys), NoSQL is better. In some, having data replicated from one to the other works well, one for primary records, one for search/display. I've even contemplated scenarios where ElasticSearch could be a source record. It really just depends on what you want/need from your data and searching.
jchrisa 13y ago
Please don't conflate MongoDB and NoSQL. NoSQL is about the right tool for the job. For instance many of our Couchbase customers are very technical and risk averse. They choose Couchbase because other solutions (often MySQL or Oracle) have become the wrong tool for the job, as requirements change. Mongo is a little different bc they actively position themselves as a MySQL replacement for your average Rails app.
- jeffdavis 13y ago
  "NoSQL is about the right tool for the job." I never liked that rationale: * The job almost certainly will change. Data lives a long time, usually much longer than the original application. * It's likely to conflate marketing claims with actual fitness for a purpose. A special-purpose system with special-purpose marketing may sound great if the special case lines up with what you're doing. But that doesn't mean that it's actually a better fit for that special purpose. And, thanks to confirmation bias, people almost always think their situation does sound like the special purpose for which the system was built. * A database system is a system, not a tool. A tool is used in a particular instance for a short period of time, the results are obvious, and it's easy to switch tools if one is not working out. Of course, there are always valid reasons to choose the special-purpose system, and I'm not saying you shouldn't have used Couchbase. I just don't like the "right tool for the job" argument.
  - frenchy 13y ago
    I've heard DBA complain that many programmers see data as not all that important: that the data simply exists to run the program. This obviously not the case in any serious real-life environments, but I think it's the sort of attitude that many NoSQL supporters have.
    - jchrisa 13y ago
      One thing that become much easier with the JSON document model is sync, which is a valuable abstraction over the network layer. http://blog.couchbase.com/why-mobile-sync http://blog.couchbase.com/why-mobile-sync Another thing that you win with the document model is lots of primary key lookups, making it easy to model your data access around your hot path. http://blog.couchbase.com/performance-oriented-architecture http://blog.couchbase.com/performance-oriented-architecture
eranation 13y ago
All I can say is this: if the saying "Always plan to throw away your MVP" is true, then I can't see any other storage solution other than MongoDB (or a similar schema-less document storage DB) for MVPs. The speed of development and flexibility are simply worth it. Yes, it is hard to refactor a live product and move it from MongoDB to MySQL / Postgre but was done before and you only do that if you get traction, so its a good problem to have. I would start with MongoDB, get a grip on what on earth the product is doing, finalize the schema on the fly based on A/B tests, customer feedback and analytics, only once the schema is finalized move it to a SQL database if needed. If you are not using a good ORM + DB migration system, then MongoDB will make perfect sense when you are quickly iterating through ideas and trying to find a product / market match. You really have no clue how your data schema is going to look like in the end, why confine it at the start? the vast majority of startups sadly won't get even near the phase where they really reach scale related performance issues, so choosing MongoDB for prototyping your business absolutely makes sense to me.
- jdoliner 13y ago
  This is really the killer feature of MongoDB its API embeds very nicely in host languages and is very intuitive to use. I work for a competitor called RethinkDB and we've spent a lot of effort trying to make something that can compete in terms of ease of use. The leap we're really trying to make is for people to not have to throw their database out with their MVP.
- cobbzilla 13y ago
  You make a VERY big "if" in your first sentence, one that (admittedly anecdotally) I've very rarely seen hold true in tech companies. Much more often, the MVP becomes the product, and all those shortcuts and poor design decisions come back to kill your productivity when it becomes necessary to refactor foundational tech/designs that have metastasized throughout the codebase. I am curious if this is other folks' experience as well, or do you actually throw out the MVP and start all over at some point? If so, at what point do you make the break?
  - daurnimator 13y ago
jdoliner 13y ago
Disclaimer, I work for RethinkDB a competitor of MongoDB. I completely agree with you that NoSQL is not a pure-white solution but I disagree that it's only useful for specific use-cases. NoSQL has frequently been touted as this magic concept that makes databases scale well. It doesn't as many of the poorly scaling NoSQL databases on the market today show you. The most you can say is that it maybe lets you ignore a few particularly hard to scale operations. However there's also no reason that a NoSQL database can't scale well it's just that you have to actually design it in such a way that it does and Mongo failed at this. Joins are a very useful feature of structured databases but there's no reason you can't have them in a NoSQL database. I speak from experience here, RethinkDB has joins (distributed joins no less). They're just as useful in a NoSQL world as they are in a SQL world. I think the author's analogy "NoSQL : SQL :: Dynamic Typing : Static Typing" (to use SAT syntax) is a good one. I know some damn good programmers who prefer dynamically typed languages like LISP and I know some damn good ones who prefer statically typed languages like Haskell and I think programmers can be similarly productive with either a NoSQL or a SQL databases backing them up. Currently NoSQL has the very real downside that none of the NoSQL databases on the market have really nailed it (ours included but we're working on it) whereas there are some very well established and well tested SQL databases on the market so it's a perfectly rational choice to say you prefer the better tested SQL route. However I contend there's nothing inherently flawed about the NoSQL concept.
- zimbatm 13y ago
  Isn't Google BigQuery a SQL on top of a schemaless database ?
  - Xorlev 13y ago
    Well, SQL and NoSQL get thrown around a lot but generally people mean 'standard relational' vs. 'newer, potentially schemaless' DB. By a 'SQL/NoSQL' view, Cassandra has CQL which isn't ANSI standard but close enough to be a SQL.
  - deno 13y ago
    It’s a distributed near–real‐time querying system for data at rest that supports subset of SQL. Google has scalable SQL database called Spanner[1]. They also have a proper NoSQL database called internally Megastore[2] that provides distributed transactions based on entity‐groups on top of BigTable. Also known as High Replication Datastore and available via AppEngine[3]. [1] https://en.wikipedia.org/wiki/Spanner_%28database%29 https://en.wikipedia.org/wiki/Spanner_%28database%29 [2] https://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf‎ https://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf‎ [3] And hopefully via Google Compute pending announcement in few days at I/O. That is if Google actually wants their Cloud to succeed.
skyebook 13y ago
I would argue that its actually a multi-tiered question where the choice isn't really "NoSQL or RDBMS" but "NoSQL or ORM talking to RDBMS or going straight to RDBMS" Here, the naive developer uses MongoDB as an all-in-one that rolls in the part of ORM that too often gets sole consideration, 'How do I persist my objects?', and effectively ignoring everything else. Try as I might to eloquently phrase my frustration with this all, it usually ends up in a rant against those relying on automagic who don't feel the need to understand every decision being made for you.
shin_lao 13y ago
I think the big problem is that most "NoSQL" engines are naive in their approach, thinking that the people at Oracle, Sybase, IBM, etc. are stupid. Very few people need scalability beyond what a regular RDBMS can do. So yes, RDBMS have limits, but you'll most likely never reach them.
- camus 13y ago
  No , there is something specific to MongoDB. Some NoSQL dbs are fine for some purposes ( multi-master sync for couchdb ,graphs for Neo4j ... ) MongoDB problem is that it promises too much and fail to deliver on a lot of levels. You cant expect to do all the operations one can do on a RDBMS on a NoSQL database and still scale horizontally,while maintaining data integrity. A good NoSQL db should have little features , but features that work very well. I like CouchDB for instance i use it in my apps to store customer infos that need to be synced accross multiple device, but i would NOT build an ecommerce app with it only... NoSQL dbs solves specific problems. It does not solve the problems relational databases are for. So selling MongoDB as a replacement for PostGreSQL or Mysql is a blatant lie.It will not work.
  - shin_lao 13y ago
    You are pretty violent against MongoDB. Any reason why?
    - AlisdairO 13y ago
      I suspect the reason that Mongo attracts so much hate is that it's more heavily marketed/evangelised than other NoSQL DBs - so more in the public eye and people expect more from it. Since the reality is that Mongo is not yet a technically strong product (regardless of its merits in terms of ease of use and getting simple things done quickly), it's bound to attract more negative attention.
      - davidw 13y ago
        Bingo. My brief experience with Mongo a few years ago was in a situation where it got selected in order to use something "new and cool". My job was to produce some reports from the data. Guess what reports require? Joins. The whole mess was slow and sucked, and the client belatedly realized that MongoDB was a shitty choice for what they needed, which could have been handled by Postgres without breaking a sweat.
ionforce 13y ago
What does "pure-white solution" mean?
- rwj 13y ago
  In this case, I believe it means all benefit and no disadvantage.
- deleted 13y ago
  [deleted]
jshen 13y ago
"till it reached some kind of moderate scale where I realized it was a terrible choice going with a NoSQL db (Sometimes, I'd have to duplicate data because there were no Joins, etc)" That statement doesn't make sense to me. One of the first things you do with a sql database when you reach moderate scale is denormalize data so you don't have to do joins, or shard which prevents a lot of joins. Joins are super convenient, but they don't scale.
- marcosdumay 13y ago
  You denormalize SOME joins. On nearly all of the cases, they are very usefull, but once in a while one of them gets too slow, so you make a bit more of work enforcing the consistence by hand, accepts a bit more of risk, and denormalize it. Throwing all of them away because a few may create problems later is a bit of an over reaction.
  - jshen 13y ago
    How many joins do you think this page uses? A denormalized version that doesn't scale probably uses 25-50. I'll bet the scalable implementation uses close to zero.
treenyc 13y ago
If you use java stack like JSP or RingoJS, I would just use H2. http://www.h2database.com/html/main.html http://www.h2database.com/html/main.html It bets MySQL and Postgresql in performance.

by neya 13y ago

mattzito 13y ago

I used to spend a lot of time in the RDBMS space, and I'd hear the DBAs bitch about how NoSQL was just a bullshit excuse for not having any discipline around your data model, and I'd defend it, saying, "Well, you have to understand, certain use cases, etc. etc. etc." And then I see an presentation like this and think, "Yes, of course, you should have put that stuff in a relational database, what were you thinking". Of course, most of the open source databases do sharding and replication and the like equally terribly, so it's not a perfect solution....

Xorlev 13y ago
I tend to take the view of RDBMS works until it doesn't. Most cases that things like click data or event data that ends up taking billions of rows, in which case you aren't going to want it in a RDBMS anyways (you'd like to, but hah). I'd like LevelDB on MySQL personally.

tracker1 13y ago

I have to completely agree... in many cases a classic SQL RDBMS is probably best. In some (such as single records having very dynamically shaped data, where most instances can be brought forward based on a key alone, and searches are on a few keys), NoSQL is better. In some, having data replicated from one to the other works well, one for primary records, one for search/display. I've even contemplated scenarios where ElasticSearch could be a source record. It really just depends on what you want/need from your data and searching.

jchrisa 13y ago

Please don't conflate MongoDB and NoSQL. NoSQL is about the right tool for the job. For instance many of our Couchbase customers are very technical and risk averse. They choose Couchbase because other solutions (often MySQL or Oracle) have become the wrong tool for the job, as requirements change. Mongo is a little different bc they actively position themselves as a MySQL replacement for your average Rails app.

jeffdavis 13y ago
"NoSQL is about the right tool for the job." I never liked that rationale: * The job almost certainly will change. Data lives a long time, usually much longer than the original application. * It's likely to conflate marketing claims with actual fitness for a purpose. A special-purpose system with special-purpose marketing may sound great if the special case lines up with what you're doing. But that doesn't mean that it's actually a better fit for that special purpose. And, thanks to confirmation bias, people almost always think their situation does sound like the special purpose for which the system was built. * A database system is a system, not a tool. A tool is used in a particular instance for a short period of time, the results are obvious, and it's easy to switch tools if one is not working out. Of course, there are always valid reasons to choose the special-purpose system, and I'm not saying you shouldn't have used Couchbase. I just don't like the "right tool for the job" argument.
- frenchy 13y ago
  I've heard DBA complain that many programmers see data as not all that important: that the data simply exists to run the program. This obviously not the case in any serious real-life environments, but I think it's the sort of attitude that many NoSQL supporters have.
  - jchrisa 13y ago
    One thing that become much easier with the JSON document model is sync, which is a valuable abstraction over the network layer. http://blog.couchbase.com/why-mobile-sync http://blog.couchbase.com/why-mobile-sync Another thing that you win with the document model is lots of primary key lookups, making it easy to model your data access around your hot path. http://blog.couchbase.com/performance-oriented-architecture http://blog.couchbase.com/performance-oriented-architecture

eranation 13y ago

All I can say is this: if the saying "Always plan to throw away your MVP" is true, then I can't see any other storage solution other than MongoDB (or a similar schema-less document storage DB) for MVPs. The speed of development and flexibility are simply worth it. Yes, it is hard to refactor a live product and move it from MongoDB to MySQL / Postgre but was done before and you only do that if you get traction, so its a good problem to have. I would start with MongoDB, get a grip on what on earth the product is doing, finalize the schema on the fly based on A/B tests, customer feedback and analytics, only once the schema is finalized move it to a SQL database if needed. If you are not using a good ORM + DB migration system, then MongoDB will make perfect sense when you are quickly iterating through ideas and trying to find a product / market match. You really have no clue how your data schema is going to look like in the end, why confine it at the start? the vast majority of startups sadly won't get even near the phase where they really reach scale related performance issues, so choosing MongoDB for prototyping your business absolutely makes sense to me.

jdoliner 13y ago
This is really the killer feature of MongoDB its API embeds very nicely in host languages and is very intuitive to use. I work for a competitor called RethinkDB and we've spent a lot of effort trying to make something that can compete in terms of ease of use. The leap we're really trying to make is for people to not have to throw their database out with their MVP.
cobbzilla 13y ago
You make a VERY big "if" in your first sentence, one that (admittedly anecdotally) I've very rarely seen hold true in tech companies. Much more often, the MVP becomes the product, and all those shortcuts and poor design decisions come back to kill your productivity when it becomes necessary to refactor foundational tech/designs that have metastasized throughout the codebase. I am curious if this is other folks' experience as well, or do you actually throw out the MVP and start all over at some point? If so, at what point do you make the break?
- daurnimator 13y ago

jdoliner 13y ago

Disclaimer, I work for RethinkDB a competitor of MongoDB. I completely agree with you that NoSQL is not a pure-white solution but I disagree that it's only useful for specific use-cases. NoSQL has frequently been touted as this magic concept that makes databases scale well. It doesn't as many of the poorly scaling NoSQL databases on the market today show you. The most you can say is that it maybe lets you ignore a few particularly hard to scale operations. However there's also no reason that a NoSQL database can't scale well it's just that you have to actually design it in such a way that it does and Mongo failed at this. Joins are a very useful feature of structured databases but there's no reason you can't have them in a NoSQL database. I speak from experience here, RethinkDB has joins (distributed joins no less). They're just as useful in a NoSQL world as they are in a SQL world. I think the author's analogy "NoSQL : SQL :: Dynamic Typing : Static Typing" (to use SAT syntax) is a good one. I know some damn good programmers who prefer dynamically typed languages like LISP and I know some damn good ones who prefer statically typed languages like Haskell and I think programmers can be similarly productive with either a NoSQL or a SQL databases backing them up. Currently NoSQL has the very real downside that none of the NoSQL databases on the market have really nailed it (ours included but we're working on it) whereas there are some very well established and well tested SQL databases on the market so it's a perfectly rational choice to say you prefer the better tested SQL route. However I contend there's nothing inherently flawed about the NoSQL concept.

zimbatm 13y ago
Isn't Google BigQuery a SQL on top of a schemaless database ?
- Xorlev 13y ago
  Well, SQL and NoSQL get thrown around a lot but generally people mean 'standard relational' vs. 'newer, potentially schemaless' DB. By a 'SQL/NoSQL' view, Cassandra has CQL which isn't ANSI standard but close enough to be a SQL.
- deno 13y ago
  It’s a distributed near–real‐time querying system for data at rest that supports subset of SQL. Google has scalable SQL database called Spanner[1]. They also have a proper NoSQL database called internally Megastore[2] that provides distributed transactions based on entity‐groups on top of BigTable. Also known as High Replication Datastore and available via AppEngine[3]. [1] https://en.wikipedia.org/wiki/Spanner_%28database%29 https://en.wikipedia.org/wiki/Spanner_%28database%29 [2] https://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf‎ https://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf‎ [3] And hopefully via Google Compute pending announcement in few days at I/O. That is if Google actually wants their Cloud to succeed.

skyebook 13y ago

I would argue that its actually a multi-tiered question where the choice isn't really "NoSQL or RDBMS" but "NoSQL or ORM talking to RDBMS or going straight to RDBMS" Here, the naive developer uses MongoDB as an all-in-one that rolls in the part of ORM that too often gets sole consideration, 'How do I persist my objects?', and effectively ignoring everything else. Try as I might to eloquently phrase my frustration with this all, it usually ends up in a rant against those relying on automagic who don't feel the need to understand every decision being made for you.

shin_lao 13y ago

I think the big problem is that most "NoSQL" engines are naive in their approach, thinking that the people at Oracle, Sybase, IBM, etc. are stupid. Very few people need scalability beyond what a regular RDBMS can do. So yes, RDBMS have limits, but you'll most likely never reach them.

camus 13y ago
No , there is something specific to MongoDB. Some NoSQL dbs are fine for some purposes ( multi-master sync for couchdb ,graphs for Neo4j ... ) MongoDB problem is that it promises too much and fail to deliver on a lot of levels. You cant expect to do all the operations one can do on a RDBMS on a NoSQL database and still scale horizontally,while maintaining data integrity. A good NoSQL db should have little features , but features that work very well. I like CouchDB for instance i use it in my apps to store customer infos that need to be synced accross multiple device, but i would NOT build an ecommerce app with it only... NoSQL dbs solves specific problems. It does not solve the problems relational databases are for. So selling MongoDB as a replacement for PostGreSQL or Mysql is a blatant lie.It will not work.
- shin_lao 13y ago
  You are pretty violent against MongoDB. Any reason why?
  - AlisdairO 13y ago
    I suspect the reason that Mongo attracts so much hate is that it's more heavily marketed/evangelised than other NoSQL DBs - so more in the public eye and people expect more from it. Since the reality is that Mongo is not yet a technically strong product (regardless of its merits in terms of ease of use and getting simple things done quickly), it's bound to attract more negative attention.
    - davidw 13y ago
      Bingo. My brief experience with Mongo a few years ago was in a situation where it got selected in order to use something "new and cool". My job was to produce some reports from the data. Guess what reports require? Joins. The whole mess was slow and sucked, and the client belatedly realized that MongoDB was a shitty choice for what they needed, which could have been handled by Postgres without breaking a sweat.

ionforce 13y ago

What does "pure-white solution" mean?

rwj 13y ago
In this case, I believe it means all benefit and no disadvantage.
deleted 13y ago
[deleted]

jshen 13y ago

"till it reached some kind of moderate scale where I realized it was a terrible choice going with a NoSQL db (Sometimes, I'd have to duplicate data because there were no Joins, etc)" That statement doesn't make sense to me. One of the first things you do with a sql database when you reach moderate scale is denormalize data so you don't have to do joins, or shard which prevents a lot of joins. Joins are super convenient, but they don't scale.

marcosdumay 13y ago
You denormalize SOME joins. On nearly all of the cases, they are very usefull, but once in a while one of them gets too slow, so you make a bit more of work enforcing the consistence by hand, accepts a bit more of risk, and denormalize it. Throwing all of them away because a few may create problems later is a bit of an over reaction.
- jshen 13y ago
  How many joins do you think this page uses? A denormalized version that doesn't scale probably uses 25-50. I'll bet the scalable implementation uses close to zero.

treenyc 13y ago

If you use java stack like JSP or RingoJS, I would just use H2. http://www.h2database.com/html/main.html http://www.h2database.com/html/main.html It bets MySQL and Postgresql in performance.