3 ms·

I don't intend this comment to be an insightful deconstruction of the NoSQL space and/or Mongo... but does anyone else notice that the level of energy around a

by rkalla 15y ago

I don't intend this comment to be an insightful deconstruction of the NoSQL space and/or Mongo... but does anyone else notice that the level of energy around a project (positive or negative) usually indicates its progress along the hype-dissolution cycle[1]? I try not to take the actual comments or articles as law, but rather use them as a temperature reading to figure out where in the slope we are currently for a given tech or trend. Given that Mongo-talk has absolutely dominated HN over the weekend as well as reddit/r/programming, I am interpreting this as being in the bottom of the dissolution curve right now. It is this point where the community push-back and temporary "hating on" forces the team to go into overdrive, addressing whatever pain points the community has griped on the loudest, in this case: - Write locks - Durability / Replication consistency (already addressed) This is like tempering steel by pounding on it... but instead it is the community pounding on the people over at 10gen. That sucks, but this right of passage for them will see sweeter days on the other side. I imagine Mongo 3.0 will represent the final climb out of the dissolution curve where all open complaints have been addressed and we actually get back to solving problems with the technology. I don't know that Cassandra or Redis or CouchDB have completed their hype-cycles yet, because they haven't had the hyper-aggressive response from the community during the dissolution step. They are all popular and well-liked, but it seems like their popularity is still climbing. It is all interesting none the less. Mongo is a huge success, regardless of how many of these articles are written. I've not seen a team as dedicated and involved like 10gen is for a long time; Eliot still answered 100s of messages on the group every week (Along with every new member of the team) -- which I find the biggest indicator of Mongo's future success. If the CTO is carving out that much time during the day to stay involved, while still bug fixing, replying to posts like these and testing bug reports... that's a lot of love right there. [1] http://en.wikipedia.org/wiki/Hype_cycle http://en.wikipedia.org/wiki/Hype_cycle

socratic 15y ago
This seems like an odd analysis if you mean that MongoDB is hitting the trough of disillusionment. MongoDB, Cassandra, HBase, and Redis all came out at roughly the same time (2008, 2009) according to Wikipedia and their project pages. Is there a reason they would be on totally different hype cycles? As far as I can tell, no one has hated on Redis or HBase (except for the brief period when antirez tried to add VM to Redis) because they both (a) work and (b) solve real use cases. Has there been any suggestion that Redis or HBase lose data? However, maybe you are right in a more general sense. Do you think that the idea of NOSQL itself is reaching the trough of disillusionment? Are we seeing a shake out of which of these data stores are actually designed by people who know what they are doing, both in (database) theory and in (systems coding) practice?
- rkalla 15y ago
  Oh I do mean that -- I don't think their time in existence effects the rate at which you move along the hype cycle, I think popularity and deployment does. I would say Mongo is the most popular NoSQL data store at the moment; whether it is mindshare or deployments and that is what caused the move along the cycle so much faster. I don't mean to detract from any of the other NoSQL projects; they don't have the marketing or manpower budget that 10gen has so I wouldn't expect them to be at the same place in the cycle. MongoDB came on the scene with the only NoSQL solution that promised SQL-esque queries, insane magnitudes jump in performance AND a big commercial company behind to. To anyone trying to understand "NoSQL", it was the clearest and safest place to look. Since then we've seen the cracks in that original argument (fast and unstable means terror in production), and 10gen has changed focus as needed and addressed those. During that time it wasn't just coding like a lot of these other projects, they were putting on conference after conference, garnering mindshare and getting developers on board. An open source Apache project just won't move along a hype path as quickly as a force like that. (I am making no statement towards quality, performance or worthiness... just positions on the hype-cycle). > Do you think that the idea of NOSQL itself is reaching > the trough of disillusionment? Are we seeing a shake out > of which of these data stores are actually designed by > people who know what they are doing, both in (database) > theory and in (systems coding) practice? I couldn't have phrased it better; yes I think this is exactly what is happening. The early days it was so exciting to see different ways to store/retrieve data. We had been with SQL for decade(s) and it was very exciting to see something new/fresh and fast popup. Then everyone started storing data every which way they could think of. Then a few of us starting solving problems with those new ideas... so far so good. Then some of those projects and new projects built on those new techniques blew up in popularity, and suddenly the "real world" came knocking and we started to actually test the metal of these things in production... with disk failures, network failures, power failures and administration failures. Like shaking out a rug, the weakest approaches got shaken out and the strongest teams/products weathered the storm to grow stronger and more stable. 2011 was the year NoSQL "Grew up", I imagine 2012 and 2013 will be the year that NoSQL comes all the way out of the dissolution curve completely and, in a metaphysical sense, "goes into production". I mean that in the most hand-wavy way, not literally... literally LOTS of people have it in production. I mean it in the sense that you stop seeing articles like these that sparked all the Mongo hype recently or articles about horrible shortcomings or failures about XYZ datastore. Early on the teams making the NoSQL solutions AND the users didn't really understand where this boat was going or how the puzzle pieces fit together... they just kept working and refining. This entire year we've seen more and more specialization in the NoSQL community: - Antirez gave up on data-larger-than-ram approaches and wants to focus Redis on what it is amazing at: being fast, in memory. - CouchDB, building on its uniquely awesome m-m replication, moves into the mobile space with data sync solutions that are awesome. - MongoDB keeps replacing MySQL in production at many large-scale startups in the valley; showing more and more the exact migration path to take. - Cassandra becomes markedly easier to use with CQL and combined with its CouchDB-esque replication behavior, suddenly makes all sorts of sense in densely populated deployments. Back in 2010 I couldn't have told you which NoSQL solution was best for which job... closing in on the end of 2011 it is glaringly obvious to me when you would use Redis and when you would use CouchDB (for example). This seems silly in hindsight, but I don't think we or the teams really honestly knew where this trip was taking the technology a year or more ago. 2012 will be a year of polish, stability and deployments. 2013 will be production deployments and replacing MySQL in more and more places. 2015, it all starts all over again as SSD-optimized data structures and data stores revamp our understanding of databases :) -- I am half-kidding. That's my 2 cents anyway.

by rkalla 15y ago

socratic 15y ago

This seems like an odd analysis if you mean that MongoDB is hitting the trough of disillusionment. MongoDB, Cassandra, HBase, and Redis all came out at roughly the same time (2008, 2009) according to Wikipedia and their project pages. Is there a reason they would be on totally different hype cycles? As far as I can tell, no one has hated on Redis or HBase (except for the brief period when antirez tried to add VM to Redis) because they both (a) work and (b) solve real use cases. Has there been any suggestion that Redis or HBase lose data? However, maybe you are right in a more general sense. Do you think that the idea of NOSQL itself is reaching the trough of disillusionment? Are we seeing a shake out of which of these data stores are actually designed by people who know what they are doing, both in (database) theory and in (systems coding) practice?

rkalla 15y ago
Oh I do mean that -- I don't think their time in existence effects the rate at which you move along the hype cycle, I think popularity and deployment does. I would say Mongo is the most popular NoSQL data store at the moment; whether it is mindshare or deployments and that is what caused the move along the cycle so much faster. I don't mean to detract from any of the other NoSQL projects; they don't have the marketing or manpower budget that 10gen has so I wouldn't expect them to be at the same place in the cycle. MongoDB came on the scene with the only NoSQL solution that promised SQL-esque queries, insane magnitudes jump in performance AND a big commercial company behind to. To anyone trying to understand "NoSQL", it was the clearest and safest place to look. Since then we've seen the cracks in that original argument (fast and unstable means terror in production), and 10gen has changed focus as needed and addressed those. During that time it wasn't just coding like a lot of these other projects, they were putting on conference after conference, garnering mindshare and getting developers on board. An open source Apache project just won't move along a hype path as quickly as a force like that. (I am making no statement towards quality, performance or worthiness... just positions on the hype-cycle). > Do you think that the idea of NOSQL itself is reaching > the trough of disillusionment? Are we seeing a shake out > of which of these data stores are actually designed by > people who know what they are doing, both in (database) > theory and in (systems coding) practice? I couldn't have phrased it better; yes I think this is exactly what is happening. The early days it was so exciting to see different ways to store/retrieve data. We had been with SQL for decade(s) and it was very exciting to see something new/fresh and fast popup. Then everyone started storing data every which way they could think of. Then a few of us starting solving problems with those new ideas... so far so good. Then some of those projects and new projects built on those new techniques blew up in popularity, and suddenly the "real world" came knocking and we started to actually test the metal of these things in production... with disk failures, network failures, power failures and administration failures. Like shaking out a rug, the weakest approaches got shaken out and the strongest teams/products weathered the storm to grow stronger and more stable. 2011 was the year NoSQL "Grew up", I imagine 2012 and 2013 will be the year that NoSQL comes all the way out of the dissolution curve completely and, in a metaphysical sense, "goes into production". I mean that in the most hand-wavy way, not literally... literally LOTS of people have it in production. I mean it in the sense that you stop seeing articles like these that sparked all the Mongo hype recently or articles about horrible shortcomings or failures about XYZ datastore. Early on the teams making the NoSQL solutions AND the users didn't really understand where this boat was going or how the puzzle pieces fit together... they just kept working and refining. This entire year we've seen more and more specialization in the NoSQL community: - Antirez gave up on data-larger-than-ram approaches and wants to focus Redis on what it is amazing at: being fast, in memory. - CouchDB, building on its uniquely awesome m-m replication, moves into the mobile space with data sync solutions that are awesome. - MongoDB keeps replacing MySQL in production at many large-scale startups in the valley; showing more and more the exact migration path to take. - Cassandra becomes markedly easier to use with CQL and combined with its CouchDB-esque replication behavior, suddenly makes all sorts of sense in densely populated deployments. Back in 2010 I couldn't have told you which NoSQL solution was best for which job... closing in on the end of 2011 it is glaringly obvious to me when you would use Redis and when you would use CouchDB (for example). This seems silly in hindsight, but I don't think we or the teams really honestly knew where this trip was taking the technology a year or more ago. 2012 will be a year of polish, stability and deployments. 2013 will be production deployments and replacing MySQL in more and more places. 2015, it all starts all over again as SSD-optimized data structures and data stores revamp our understanding of databases :) -- I am half-kidding. That's my 2 cents anyway.