13 ms·

MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. The RethinkDB

by adwhit 9y ago

MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. The RethinkDB retrospective[0] contains a lot of insight into how MongoDB has succeeded despite being vastly inferior on a technical level back when it first launched. I have to admit them a certain respect for executing their strategy so successfully. Choice quote: Every time MongoDB shipped a new release and people congratulated them on making improvements, I felt pangs of resentment. They’d announce they fixed the BKL, but really they’d get the granularity level down from a database to a collection. They’d add more operations, but instead of a composable interface that fits with the rest of the system, they’d simply bolt on one-off commands. They’d make sharding improvements, but it was obvious they were unwilling or unable to make even rudimentary data consistency guarantees. But over time I learned to appreciate the wisdom of the crowds. MongoDB turned regular developers into heroes when people needed it, not years after the fact. It made data storage fast, and let people ship products quickly. And over time, MongoDB grew up. One by one, they fixed the issues with the architecture, and now it is an excellent product. It may not be as beautiful as we would have wanted, but it does the job, and it does it well. [0] http://www.defmacro.org/2017/01/18/why-rethinkdb-failed.html http://www.defmacro.org/2017/01/18/why-rethinkdb-failed.html

andy_ppp 9y ago
I agree here, but I’d go further; build things people want, not the idealistic future some day version where we eventually get to a priority feature for a lot of people like releasing fast scalable software quickly (I’m not saying Rethink didn’t do this but they prioritised correctness and sharding, features fewer people need). For most apps built with Mongo this transaction support isn’t a problem (until it is).
- nathan_long 9y ago
  Both approaches have downsides. The TCP/IP stack was built and used while the OSI model was being designed, and it won all the mindshare. Perhaps it would have been better to have separate presentation and session layers, but we don't; the application layer handles that stuff. It works well enough. OTOH, this quote is wise: > It is easier to optimize correct code than to correct optimized code (Bill Harlan) I think this is doubly true for databases; at least with obfuscated code, you can recover the underlying meaning with work and exploration. Losing or corrupting data is the worst thing a database can do. Given "this will be correct and hopefully we can scale it" vs "this will be fast and hopefully we can keep it correct", I'd choose the former for any "source of truth" data every time. There are tricks for speeding up queries - indexes, cacheing (including materialized views), sharding, read replicas, etc. There are no tricks for recovering data you lost.
  - andy_ppp 9y ago
    > I think this is doubly true for databases; at least with obfuscated code, you can recover the underlying meaning with work and exploration. True for databases, but not true for businesses. > Losing or corrupting data is the worst thing a database can do. Clearly people building simple crud websites with slick JS features didn’t agree otherwise Mongo would be gone and Rethink would be worth hundreds of millions of dollars.
    - crdoconnor 9y ago
      >Clearly people building simple crud websites with slick JS features didn’t agree I doubt it's that they didn't agree, it's more likely that the thought simply never occurred to them. Mongo's marketing is directed with laser like focus on the beginner developer seeking out tutorials to build a website, etc. Questions about data consistency simply never arise in that context. Later on that developer who was gently guided towards using mongo by all of the slick marketing will likely try to defend their decision when somebody attacks it ("their data consistency problems aren't that bad" or "data consistency isn't that important"), but that's something else.
    - gameswithgo 9y ago
      >Clearly people building simple crud websites with slick JS features didn’t agree otherwise Mongo would be gone Popularity is not always a good measure of what ideas are good ones.
- makmanalp 9y ago
  > build things people want, not the idealistic future some day version where we eventually get to a priority feature FWIW, I don't think this was what happened. RethinkDB started out as an SSD optimized database, and quickly repositioned itself (due to "is this what people want") to something more generally useful, and was one of the most feature-rich databases at the time, I thought. MongoDB however got first mover advantage and a bunch of cash that comes with it. They could afford to invest heavily in developer evangelism. Then they bought WiredTiger. If I sound bitter, I am a bit - not that Mongo did well in the end, but that RethinkDB went the way it did.
im_down_w_otp 9y ago
Clearly the evidence bears out the success of that strategy, but it's hard not to summarize it as, "Apparently a lot of developers, applications, and users don't need a database that works." But I don't know what that's really an indictment of exactly.
SCdF 9y ago
> MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. I have no idea how capable MongoDB is these days, as I haven't used Mongo in years (and even then it was not for long). However, I do not know any developers who, after living through the "hype first, features later" strategy, have been left with a positive enough opinion of MongoDB to ever want to use it again.
- drcongo 9y ago
  There's a whole 'nother generation of devs coming through who have never been burned by MongoDB though. Obviously they will be eventually, but by then another generation will come along to repeat the cycle.
  - e12e 9y ago
    I love deriding mongodb as much as the next dev that hasn't used it much; but I'll just note that while I'd still be hard pressed to prefer mysql over postgres - there was a long period where mysql was put to tasks it was ill suited for, especially prior to around version 4.x. So while "hype first" might reap a deservedly abundant and bitter harvest of developer hatred - it doesn't preclude evolving into a genuinely useful product...
    - da_chicken 9y ago
      Both true, although I can completely understand why devs went with MySQL over PostgreSQL at that time. I remember that during the same time period that MySQL was drawing seemingly endless criticism for generally poor RDBMS behavior (3.x and 4.x), PostgreSQL was notorious for having poor performance due to insanely undersized default settings. Like out of the box it was sized to run with at most 10 MB of RAM or similar that was just unrealistic. I also remember it also had a lot of quirks and missing features prior to v8. I assume it was leftover cruft from Ingres, but I remember PostgreSQL v6 and v7 being unreasonably complicated to get configured just because the defaults were so off reality. One thing you can say about PostgreSQL, though, is that it's developers don't rest on their heels. Every major release packs in a ton of new features. They've gone from being fairly low or middling on the feature set to being pretty near the top. Even point releases have me saying, "Wow, that's really nice to have."
Gogogogirl 9y ago
Yes and Mongo helped us a lot to get thinks off the ground, but we're now moving everything to postgres with JSON for some time.
Analemma_ 9y ago
I was badly burned by Mongo hype back in the day, and as a result I won’t touch it with a 10-foot pole for the rest of my life, no matter how many times people say “No, really, it’s good now”. Falling for that was how I got into trouble in the first place. I know a lot of other devs like this. If they can be successful despite us, more power to em I suppose. I’m a little annoyed that their path to success was built on the flaming wreckage of so many products that fell apart because of Mongo, by using us as their beta testers instead of building a non-shitty product, and I’m at least going to get this comment in so we aren’t completely forgotten among the congratulation.
FPGAhacker 9y ago
There is a trend I noticed where I work. Most people that “get it right” the first time around do not get any recognition whatsoever. It is the people that screw up, release with big flaws that the customer then pressures the company about, that are heralded as heroes and bacon savers when the fix those flaws. After 3 years and as many releases.
- tejasmanohar 9y ago
  >Most people that “get it right” the first time around do not get any recognition whatsoever. I've heard similar complaints before. And, I get it, too-- at a glance, that person is playing the "superhero" by saving the project. But, good management will insist on root causing failures where this will unravel. If it's a recurring problem, you should bring it up with management.
  - luckydata 9y ago
    My biggest gripe as a "lateral manager" (I don't manage engineers, I manage products) is that I see those things happen all the time, and I spend time coaching developers to interact effectively with their managers as much as I can. It's frustrating when I see people that should know better (because I know they heard me) not taking notes about serious issues they want to discuss with their superiors, not knowing how to escalate issues that threaten the well being of the product or the team but that their direct superior doesn't believe are urgent etc... Developers complain about management but tend to forget that managers are people just like everyone else, and we need to apply some skill to our interactions if we are to get the results we desire.
    - mycelium 9y ago
      This completely squares with my experiences as well — a lot of instances of complaints about management are hollow because developers aren't managing upward correctly. Their followup on their issues is missing, or non-actionable. Do you have any resources you've found helpful improving your skill at this?
  - 9y ago
harryh 9y ago
It's linked in the RethinkDB essay, but it's always worth explicitly calling out the "Worse Is Better" essay: http://dreamsongs.com/RiseOfWorseIsBetter.html http://dreamsongs.com/RiseOfWorseIsBetter.html Ignore its lessons at your peril. Your job isn't to build an engineering masterpiece. Your job, as pg says, is to build something people want.
- btilly 9y ago
  I agree with the lessons in Worse Is Better, but I don't think that the author properly understood what he was observing. The result was a confused and confusing essay. The way that I understand it is that what is "Good" depends on how you measure it. When we measure in terms of technical quality, we get one answer. When we measure in terms of suited to be widely adopted, we get a different answer. We tend to idealize for technical quality, but popularity is what matters more. And once something is widely enough adopted, the technical inferiority tends to be fixable.
- pweissbrod 9y ago
  Shipping early and working on stability later may work for something like a video game but not for a database my system depends on thanks
  - aneutron 9y ago
    If you're in nation-wide healthcare, that strategy quickly becomes unacceptable.
  - overcast 9y ago
    I think many high profile games have over the last few years, have proven this does not work. Gamers are finally fighting back with their wallets.
zappo2938 9y ago
With all its problems, I built a MEAN (MongoDB, Express, Angular, Node) app from zero knowledge to production 2 years ago far faster than this React, Apollo, GraphQL, and Postgres app I'm building from zero knowledge.
- gremlinsinc 9y ago
  Speed isn't always a great thing... If it takes you 2x faster to build but 10x extra support/maintenance after the fact and eventually you need to migrate to postgres anyway because of acid features and stability.. then the time/money loss > benefits. Build something the right way first, even if it does take longer though I use rbdms(mysql or postgres) all the time with an ORM and the ORM does most of the heavy lifting. (Laravel/Eloquent in my case).. so I still develop pretty rapidly. I'm sure if you use pg+react on multiple projects eventually the speed to launch will increase...
  - zappo2938 9y ago
    It is honestly a nightmare. I had to make a decision which framework to invest learning in with very limited funds. At the time, the big choices where Angular which had been established and backed by Google and React which was still very new with a much smaller community. I went with Angular and by the time I learned everything I needed, everyone wanted to hire React developers. Running out of money, I ended selling all my belongings, moving to a new city, and doing Backbone.js development. I've been working on learning React for the last several months not earning money and it is more difficult to learn than either Angular or Backbone because it isn't as opinionated driving me to have to learn each tool to decide which is best. My mind craves structure. Whereas most developers have 2 years React experience on me. I figure if I waited 6 months to learn JavaScript frameworks, React would have been the better choice and I would have been far better off than what happened. In a way the MEAN stack screwed me.
mooreds 9y ago
The flip side of 'hype first, features later' is that if you are a user who is burnt by mongodb (or another solution) you'll recommend against using it for a long time. So there's a knife edge to balance on--hype enough, but not so much that too many people get burned.
nemild 9y ago
I also quoted Rethink's post in "The Marketing Behind MongoDB" in part 3 of my series on MongoDB: > I sympathize with RethinkDB's team — they did what thoughtful engineers are trained to do. Engineering purity and humility is a tiny part of building a sustainable, venture-backed company. https://www.nemil.com/mongo/ https://www.nemil.com/mongo/
- williamstein 9y ago
  Despite their claims to the contrary, RethinkDB also released and claimed 'ready for production use' a version of the product that was pretty broken. I used it heavily and hit numerous serious bugs. The RethinkDB devs did do a very good job of tracking down and fixing them. Software is hard.
misterbowfinger 9y ago
From that post mortem: It was unfathomable to us why people would choose a system that barely does the thing it’s supposed to do (store data), has a big kernel lock, throws away errors at random, implements single node features that stop working when you shard, has a barely working sharding system despite it being one of the core features of the product, provides essentially no correctness guarantees, and exposes a hodge-podge of interfaces that have no discernible consistency or unity of vision. I mean... that's unfathomable to me too. He explains it later, that " MongoDB turned regular developers into heroes when people needed it" I have a hard time understanding why devs choose / chose MongoDB. Postgres with JSON columns gets you so far, why would you go with MongoDB, given the issues it's had?
- s_kilk 9y ago
  > I have a hard time understanding why devs choose / chose MongoDB. Postgres with JSON columns gets you so far, why would you go with MongoDB, given the issues it's had? Jsonb is a pretty recent addition to postgres, when compared with the MongoDB timeline. And even today postgres still doesn't have the replication/failover story that made MongoDB pretty compelling. I know, it's coming, whatever, but the point is that there was a time where if you wanted a json store that could stay alive through network issues, MongoDB was one of the only choices available, and postgres simply didn't have what was needed.
  - tensor 9y ago
    The problem with that thinking is that the replication mattered. I'd argue it didn't, it was essentially a scam that people fell for. Who cares about failover when you're losing data due to a bad implementation? Who cares about replication when you can gain the same performance by using a performant database on a single node? Did mongodb truly allow anyone to really horizontally scale? Most places that need massive horizontal scaling using something like mysql as far as I know.
    - imtringued 9y ago
      The thing I don't understand about mongodb is that it makes a tradeoff for scalability. The secret ingredient in the horizontal scaling sauce is giving up inter node ACID transactions. Nothing prevents you from making the same tradeoff with mysql or postgresql.
vog 9y ago
On the other hand, PostgreSQL is a very good example of a successful implementation of the opposite strategy, that is, "correctness first". And since PostgreSQL fills that niche very well (correctness + real ACID + extensibility + decent performance), maybe it was really PostgreSQL who killed RethinkDB?
- luckydata 9y ago
  If you're playing the long game and not looking to make a profit that's fine, but PostgreSQL as a company would have been doomed a long time ago. You have to keep in mind the timelines of the business and what they need to do to keep the lights on. MongoDB has identified a real pain point: many developers don't like to use SQL to interface with a transactional database. I'm not going into the merits of SQL vs. NoSQL, I'm just stating that it's clear there's a need or they wouldn't have gotten any traction. Now they are maturing the product to the point it might be a safe bet for some use cases, it remains to be seen if their approach to product development will pay dividends or the reputation they have created for themselves has created a time bomb that will eventually kill them.
  - williamstein 9y ago
    PostgreSQL as a company: https://www.citusdata.com https://www.citusdata.com
    - chipotle_coyote 9y ago
      There are several companies that are leveraging PostgreSQL for their own businesses, but that doesn't seem to me to be a rebuttal of the OP's assertion that PostgreSQL couldn't survive as a company itself. Citus Data is not "PostgreSQL as a company," it is "a company that exists because PostgreSQL already existed."
      - qaq 9y ago
        Well majority of key Postgres contributors work for 2ndQuadrant, EnterpriseDB, Crunchy Data, Citus etc. It basically means PostgreSQL is a distributed company and it would survive fine but being distributed it looks like it is able to innovate faster and is more resilient.
Fiahil 9y ago
While MongoDB might be a decent document store, I found that Elasticsearch is better at this job (as a secondary datastore). Its aggregation capabilities are juste far better than MongoDB's with the added bonus of being really good for all kind of searches.
btilly 9y ago
MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. I was going to say that I won't believe that it is on its way to being a decent database until after an article appears on https://aphyr.com/tags/jepsen https://aphyr.com/tags/jepsen saying that MongoDB actually delivers on what it claims. So I looked for the most recent analysis of MongoDB and found https://jepsen.io/analyses/mongodb-3-4-0-rc3 https://jepsen.io/analyses/mongodb-3-4-0-rc3. I still want to see verification of the latest release, and hear battle stories from it in production. But I'm provisionally optimistic that a lot of the glaring "it is a pile of shit that doesn't work when the chips are down" issues are now addressed. That said, I bet that it will be many years before most people who got burned by MongoDB ever rethink their attitudes about it. Once burned, twice shy. And it really was an overhyped steaming pile of shit for a very long time.
- threeseed 9y ago
  I have used MongoDB in production for a number of Fortune 100 sized companies. It has always been a unique database that was ideal for scenarios when your data model was document orientated. > was an overhyped steaming pile of shit for a very long time No it wasn't. This is something you heard from people who never really used it. It had its faults but it was never a pile of shit nor was it substantially worse than other databases.
  - gaius 9y ago
    This is something you heard from people who never really used it I used it at a previous job. Project to move a multi-tera dataset from an Oracle box (24 CPUs, 24G RAM, SAN) to a MongoDB cluster (10 boxes, each with 48 cores, 96G RAM and internal SSD). MongoDB couldn't perform for shit, and it couldn't stay up in a usable state for more than a few hours at a time. This is with 20x the processors and 40x the memory of the system it was replacing. It's a complete joke of a product, sold on the basis of outright lies as far as what they told us and what it could actually do. Having been that badly burned I consider it an act of selfless public service to warn people off it. If you're just using it for a personal blog that gets 10 views a day, sure it might be barely adequate for that. But I'd still use Postgres.
overcast 9y ago
I loved working with RethinkDB, and the changefeed stuff was awesome. It gave me relational documents, which is all I wanted for most projects. Bummed that project has been basically slowed to nothing.
juancampa 9y ago
I'm still mourning RethinkDB. It's supposed to still be alive but the release cycle, or lack thereof, says otherwise.
Perseids 9y ago
> I have to admit them a certain respect for executing their strategy so successfully. But be certain not to conflate your respect as a business strategist with your judgment as a mindful developer. To speak clearly: By systematically playing a weak spot of ours [1] they have used countless of small teams as a stepping stone to sell their business contracts to large players, while hurting a lot of these small teams with an (at the time) inappropriate product for their needs. And through these huge costs they still made a product that is inferior to one that was designed properly. As a community (both as a startup, as well as a developer community) we should resent these tactics and try to find ways to protect us against players that abuse the common good of mindshare. And lest you say, that is the price you have to pay to at all get a product like MongoDB in harsh business environments: We could also lobby for open source funds that are organized like research funds, producing fundamental technology that benefit everyone. Not every technology fits the model of for-profit startup innovation. [1] Our community has very little defenses against marketing that comes from our midst, aiming to produce the (false) impression that a disproportionate amount of our fellows have evaluated the product and found it to be excellent. See https://www.nemil.com/mongo/3.html https://www.nemil.com/mongo/3.html for a discussion about MongoDb specifically (HN thread: https://news.ycombinator.com/item?id=15124306 https://news.ycombinator.com/item?id=15124306 )
Lunatic666 9y ago
A database which utterly fails the Jepsen test should not be considered for production. It might be good enough for a cache, but trusting it with real data is reckless.
- eloff 9y ago
  They have since passed the Jepsen test to be fair. But before then I fully agree, why people trusted MongoDB with their critical data is beyond me.
boubiyeah 9y ago
Agreed. It was a complete piece of crap when it was around version 1.4-1.6 but it's pretty good now!

by adwhit 9y ago

andy_ppp 9y ago

I agree here, but I’d go further; build things people want, not the idealistic future some day version where we eventually get to a priority feature for a lot of people like releasing fast scalable software quickly (I’m not saying Rethink didn’t do this but they prioritised correctness and sharding, features fewer people need). For most apps built with Mongo this transaction support isn’t a problem (until it is).

nathan_long 9y ago
Both approaches have downsides. The TCP/IP stack was built and used while the OSI model was being designed, and it won all the mindshare. Perhaps it would have been better to have separate presentation and session layers, but we don't; the application layer handles that stuff. It works well enough. OTOH, this quote is wise: > It is easier to optimize correct code than to correct optimized code (Bill Harlan) I think this is doubly true for databases; at least with obfuscated code, you can recover the underlying meaning with work and exploration. Losing or corrupting data is the worst thing a database can do. Given "this will be correct and hopefully we can scale it" vs "this will be fast and hopefully we can keep it correct", I'd choose the former for any "source of truth" data every time. There are tricks for speeding up queries - indexes, cacheing (including materialized views), sharding, read replicas, etc. There are no tricks for recovering data you lost.
- andy_ppp 9y ago
  > I think this is doubly true for databases; at least with obfuscated code, you can recover the underlying meaning with work and exploration. True for databases, but not true for businesses. > Losing or corrupting data is the worst thing a database can do. Clearly people building simple crud websites with slick JS features didn’t agree otherwise Mongo would be gone and Rethink would be worth hundreds of millions of dollars.
  - crdoconnor 9y ago
    >Clearly people building simple crud websites with slick JS features didn’t agree I doubt it's that they didn't agree, it's more likely that the thought simply never occurred to them. Mongo's marketing is directed with laser like focus on the beginner developer seeking out tutorials to build a website, etc. Questions about data consistency simply never arise in that context. Later on that developer who was gently guided towards using mongo by all of the slick marketing will likely try to defend their decision when somebody attacks it ("their data consistency problems aren't that bad" or "data consistency isn't that important"), but that's something else.
  - gameswithgo 9y ago
    >Clearly people building simple crud websites with slick JS features didn’t agree otherwise Mongo would be gone Popularity is not always a good measure of what ideas are good ones.
makmanalp 9y ago
> build things people want, not the idealistic future some day version where we eventually get to a priority feature FWIW, I don't think this was what happened. RethinkDB started out as an SSD optimized database, and quickly repositioned itself (due to "is this what people want") to something more generally useful, and was one of the most feature-rich databases at the time, I thought. MongoDB however got first mover advantage and a bunch of cash that comes with it. They could afford to invest heavily in developer evangelism. Then they bought WiredTiger. If I sound bitter, I am a bit - not that Mongo did well in the end, but that RethinkDB went the way it did.

im_down_w_otp 9y ago

Clearly the evidence bears out the success of that strategy, but it's hard not to summarize it as, "Apparently a lot of developers, applications, and users don't need a database that works." But I don't know what that's really an indictment of exactly.

SCdF 9y ago

> MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. I have no idea how capable MongoDB is these days, as I haven't used Mongo in years (and even then it was not for long). However, I do not know any developers who, after living through the "hype first, features later" strategy, have been left with a positive enough opinion of MongoDB to ever want to use it again.

drcongo 9y ago
There's a whole 'nother generation of devs coming through who have never been burned by MongoDB though. Obviously they will be eventually, but by then another generation will come along to repeat the cycle.
- e12e 9y ago
  I love deriding mongodb as much as the next dev that hasn't used it much; but I'll just note that while I'd still be hard pressed to prefer mysql over postgres - there was a long period where mysql was put to tasks it was ill suited for, especially prior to around version 4.x. So while "hype first" might reap a deservedly abundant and bitter harvest of developer hatred - it doesn't preclude evolving into a genuinely useful product...
  - da_chicken 9y ago
    Both true, although I can completely understand why devs went with MySQL over PostgreSQL at that time. I remember that during the same time period that MySQL was drawing seemingly endless criticism for generally poor RDBMS behavior (3.x and 4.x), PostgreSQL was notorious for having poor performance due to insanely undersized default settings. Like out of the box it was sized to run with at most 10 MB of RAM or similar that was just unrealistic. I also remember it also had a lot of quirks and missing features prior to v8. I assume it was leftover cruft from Ingres, but I remember PostgreSQL v6 and v7 being unreasonably complicated to get configured just because the defaults were so off reality. One thing you can say about PostgreSQL, though, is that it's developers don't rest on their heels. Every major release packs in a ton of new features. They've gone from being fairly low or middling on the feature set to being pretty near the top. Even point releases have me saying, "Wow, that's really nice to have."

Gogogogirl 9y ago

Yes and Mongo helped us a lot to get thinks off the ground, but we're now moving everything to postgres with JSON for some time.

Analemma_ 9y ago

I was badly burned by Mongo hype back in the day, and as a result I won’t touch it with a 10-foot pole for the rest of my life, no matter how many times people say “No, really, it’s good now”. Falling for that was how I got into trouble in the first place. I know a lot of other devs like this. If they can be successful despite us, more power to em I suppose. I’m a little annoyed that their path to success was built on the flaming wreckage of so many products that fell apart because of Mongo, by using us as their beta testers instead of building a non-shitty product, and I’m at least going to get this comment in so we aren’t completely forgotten among the congratulation.

FPGAhacker 9y ago

There is a trend I noticed where I work. Most people that “get it right” the first time around do not get any recognition whatsoever. It is the people that screw up, release with big flaws that the customer then pressures the company about, that are heralded as heroes and bacon savers when the fix those flaws. After 3 years and as many releases.

tejasmanohar 9y ago
>Most people that “get it right” the first time around do not get any recognition whatsoever. I've heard similar complaints before. And, I get it, too-- at a glance, that person is playing the "superhero" by saving the project. But, good management will insist on root causing failures where this will unravel. If it's a recurring problem, you should bring it up with management.
- luckydata 9y ago
  My biggest gripe as a "lateral manager" (I don't manage engineers, I manage products) is that I see those things happen all the time, and I spend time coaching developers to interact effectively with their managers as much as I can. It's frustrating when I see people that should know better (because I know they heard me) not taking notes about serious issues they want to discuss with their superiors, not knowing how to escalate issues that threaten the well being of the product or the team but that their direct superior doesn't believe are urgent etc... Developers complain about management but tend to forget that managers are people just like everyone else, and we need to apply some skill to our interactions if we are to get the results we desire.
  - mycelium 9y ago
    This completely squares with my experiences as well — a lot of instances of complaints about management are hollow because developers aren't managing upward correctly. Their followup on their issues is missing, or non-actionable. Do you have any resources you've found helpful improving your skill at this?
- 9y ago

harryh 9y ago

It's linked in the RethinkDB essay, but it's always worth explicitly calling out the "Worse Is Better" essay: http://dreamsongs.com/RiseOfWorseIsBetter.html http://dreamsongs.com/RiseOfWorseIsBetter.html Ignore its lessons at your peril. Your job isn't to build an engineering masterpiece. Your job, as pg says, is to build something people want.

btilly 9y ago
I agree with the lessons in Worse Is Better, but I don't think that the author properly understood what he was observing. The result was a confused and confusing essay. The way that I understand it is that what is "Good" depends on how you measure it. When we measure in terms of technical quality, we get one answer. When we measure in terms of suited to be widely adopted, we get a different answer. We tend to idealize for technical quality, but popularity is what matters more. And once something is widely enough adopted, the technical inferiority tends to be fixable.
pweissbrod 9y ago
Shipping early and working on stability later may work for something like a video game but not for a database my system depends on thanks
- aneutron 9y ago
  If you're in nation-wide healthcare, that strategy quickly becomes unacceptable.
- overcast 9y ago
  I think many high profile games have over the last few years, have proven this does not work. Gamers are finally fighting back with their wallets.

zappo2938 9y ago

With all its problems, I built a MEAN (MongoDB, Express, Angular, Node) app from zero knowledge to production 2 years ago far faster than this React, Apollo, GraphQL, and Postgres app I'm building from zero knowledge.

gremlinsinc 9y ago
Speed isn't always a great thing... If it takes you 2x faster to build but 10x extra support/maintenance after the fact and eventually you need to migrate to postgres anyway because of acid features and stability.. then the time/money loss > benefits. Build something the right way first, even if it does take longer though I use rbdms(mysql or postgres) all the time with an ORM and the ORM does most of the heavy lifting. (Laravel/Eloquent in my case).. so I still develop pretty rapidly. I'm sure if you use pg+react on multiple projects eventually the speed to launch will increase...
- zappo2938 9y ago
  It is honestly a nightmare. I had to make a decision which framework to invest learning in with very limited funds. At the time, the big choices where Angular which had been established and backed by Google and React which was still very new with a much smaller community. I went with Angular and by the time I learned everything I needed, everyone wanted to hire React developers. Running out of money, I ended selling all my belongings, moving to a new city, and doing Backbone.js development. I've been working on learning React for the last several months not earning money and it is more difficult to learn than either Angular or Backbone because it isn't as opinionated driving me to have to learn each tool to decide which is best. My mind craves structure. Whereas most developers have 2 years React experience on me. I figure if I waited 6 months to learn JavaScript frameworks, React would have been the better choice and I would have been far better off than what happened. In a way the MEAN stack screwed me.

mooreds 9y ago

The flip side of 'hype first, features later' is that if you are a user who is burnt by mongodb (or another solution) you'll recommend against using it for a long time. So there's a knife edge to balance on--hype enough, but not so much that too many people get burned.

nemild 9y ago

I also quoted Rethink's post in "The Marketing Behind MongoDB" in part 3 of my series on MongoDB: > I sympathize with RethinkDB's team — they did what thoughtful engineers are trained to do. Engineering purity and humility is a tiny part of building a sustainable, venture-backed company. https://www.nemil.com/mongo/ https://www.nemil.com/mongo/

williamstein 9y ago
Despite their claims to the contrary, RethinkDB also released and claimed 'ready for production use' a version of the product that was pretty broken. I used it heavily and hit numerous serious bugs. The RethinkDB devs did do a very good job of tracking down and fixing them. Software is hard.

misterbowfinger 9y ago

From that post mortem: It was unfathomable to us why people would choose a system that barely does the thing it’s supposed to do (store data), has a big kernel lock, throws away errors at random, implements single node features that stop working when you shard, has a barely working sharding system despite it being one of the core features of the product, provides essentially no correctness guarantees, and exposes a hodge-podge of interfaces that have no discernible consistency or unity of vision. I mean... that's unfathomable to me too. He explains it later, that " MongoDB turned regular developers into heroes when people needed it" I have a hard time understanding why devs choose / chose MongoDB. Postgres with JSON columns gets you so far, why would you go with MongoDB, given the issues it's had?

s_kilk 9y ago
> I have a hard time understanding why devs choose / chose MongoDB. Postgres with JSON columns gets you so far, why would you go with MongoDB, given the issues it's had? Jsonb is a pretty recent addition to postgres, when compared with the MongoDB timeline. And even today postgres still doesn't have the replication/failover story that made MongoDB pretty compelling. I know, it's coming, whatever, but the point is that there was a time where if you wanted a json store that could stay alive through network issues, MongoDB was one of the only choices available, and postgres simply didn't have what was needed.
- tensor 9y ago
  The problem with that thinking is that the replication mattered. I'd argue it didn't, it was essentially a scam that people fell for. Who cares about failover when you're losing data due to a bad implementation? Who cares about replication when you can gain the same performance by using a performant database on a single node? Did mongodb truly allow anyone to really horizontally scale? Most places that need massive horizontal scaling using something like mysql as far as I know.
  - imtringued 9y ago
    The thing I don't understand about mongodb is that it makes a tradeoff for scalability. The secret ingredient in the horizontal scaling sauce is giving up inter node ACID transactions. Nothing prevents you from making the same tradeoff with mysql or postgresql.

vog 9y ago

On the other hand, PostgreSQL is a very good example of a successful implementation of the opposite strategy, that is, "correctness first". And since PostgreSQL fills that niche very well (correctness + real ACID + extensibility + decent performance), maybe it was really PostgreSQL who killed RethinkDB?

luckydata 9y ago
If you're playing the long game and not looking to make a profit that's fine, but PostgreSQL as a company would have been doomed a long time ago. You have to keep in mind the timelines of the business and what they need to do to keep the lights on. MongoDB has identified a real pain point: many developers don't like to use SQL to interface with a transactional database. I'm not going into the merits of SQL vs. NoSQL, I'm just stating that it's clear there's a need or they wouldn't have gotten any traction. Now they are maturing the product to the point it might be a safe bet for some use cases, it remains to be seen if their approach to product development will pay dividends or the reputation they have created for themselves has created a time bomb that will eventually kill them.
- williamstein 9y ago
  PostgreSQL as a company: https://www.citusdata.com https://www.citusdata.com
  - chipotle_coyote 9y ago
    There are several companies that are leveraging PostgreSQL for their own businesses, but that doesn't seem to me to be a rebuttal of the OP's assertion that PostgreSQL couldn't survive as a company itself. Citus Data is not "PostgreSQL as a company," it is "a company that exists because PostgreSQL already existed."
    - qaq 9y ago
      Well majority of key Postgres contributors work for 2ndQuadrant, EnterpriseDB, Crunchy Data, Citus etc. It basically means PostgreSQL is a distributed company and it would survive fine but being distributed it looks like it is able to innovate faster and is more resilient.

Fiahil 9y ago

While MongoDB might be a decent document store, I found that Elasticsearch is better at this job (as a secondary datastore). Its aggregation capabilities are juste far better than MongoDB's with the added bonus of being really good for all kind of searches.

btilly 9y ago

MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. I was going to say that I won't believe that it is on its way to being a decent database until after an article appears on https://aphyr.com/tags/jepsen https://aphyr.com/tags/jepsen saying that MongoDB actually delivers on what it claims. So I looked for the most recent analysis of MongoDB and found https://jepsen.io/analyses/mongodb-3-4-0-rc3 https://jepsen.io/analyses/mongodb-3-4-0-rc3. I still want to see verification of the latest release, and hear battle stories from it in production. But I'm provisionally optimistic that a lot of the glaring "it is a pile of shit that doesn't work when the chips are down" issues are now addressed. That said, I bet that it will be many years before most people who got burned by MongoDB ever rethink their attitudes about it. Once burned, twice shy. And it really was an overhyped steaming pile of shit for a very long time.

threeseed 9y ago
I have used MongoDB in production for a number of Fortune 100 sized companies. It has always been a unique database that was ideal for scenarios when your data model was document orientated. > was an overhyped steaming pile of shit for a very long time No it wasn't. This is something you heard from people who never really used it. It had its faults but it was never a pile of shit nor was it substantially worse than other databases.
- gaius 9y ago
  This is something you heard from people who never really used it I used it at a previous job. Project to move a multi-tera dataset from an Oracle box (24 CPUs, 24G RAM, SAN) to a MongoDB cluster (10 boxes, each with 48 cores, 96G RAM and internal SSD). MongoDB couldn't perform for shit, and it couldn't stay up in a usable state for more than a few hours at a time. This is with 20x the processors and 40x the memory of the system it was replacing. It's a complete joke of a product, sold on the basis of outright lies as far as what they told us and what it could actually do. Having been that badly burned I consider it an act of selfless public service to warn people off it. If you're just using it for a personal blog that gets 10 views a day, sure it might be barely adequate for that. But I'd still use Postgres.

overcast 9y ago

I loved working with RethinkDB, and the changefeed stuff was awesome. It gave me relational documents, which is all I wanted for most projects. Bummed that project has been basically slowed to nothing.

juancampa 9y ago

I'm still mourning RethinkDB. It's supposed to still be alive but the release cycle, or lack thereof, says otherwise.

Perseids 9y ago

> I have to admit them a certain respect for executing their strategy so successfully. But be certain not to conflate your respect as a business strategist with your judgment as a mindful developer. To speak clearly: By systematically playing a weak spot of ours [1] they have used countless of small teams as a stepping stone to sell their business contracts to large players, while hurting a lot of these small teams with an (at the time) inappropriate product for their needs. And through these huge costs they still made a product that is inferior to one that was designed properly. As a community (both as a startup, as well as a developer community) we should resent these tactics and try to find ways to protect us against players that abuse the common good of mindshare. And lest you say, that is the price you have to pay to at all get a product like MongoDB in harsh business environments: We could also lobby for open source funds that are organized like research funds, producing fundamental technology that benefit everyone. Not every technology fits the model of for-profit startup innovation. [1] Our community has very little defenses against marketing that comes from our midst, aiming to produce the (false) impression that a disproportionate amount of our fellows have evaluated the product and found it to be excellent. See https://www.nemil.com/mongo/3.html https://www.nemil.com/mongo/3.html for a discussion about MongoDb specifically (HN thread: https://news.ycombinator.com/item?id=15124306 https://news.ycombinator.com/item?id=15124306 )

Lunatic666 9y ago

A database which utterly fails the Jepsen test should not be considered for production. It might be good enough for a cache, but trusting it with real data is reckless.

eloff 9y ago
They have since passed the Jepsen test to be fair. But before then I fully agree, why people trusted MongoDB with their critical data is beyond me.

boubiyeah 9y ago

Agreed. It was a complete piece of crap when it was around version 1.4-1.6 but it's pretty good now!