14 ms·

MongoDB 3.4.0-rc3

226 points by aphyr 10y ago

kenwalger 10y ago
MongoDB 3.4 passes the rigorous and tough Jepsen test. Jepsen designs tests to make databases fail in terms of data consistency, correctness, and safety... MongoDB 3.4 passed through their newest tests. I think that this really shows how mature of a Database MongoDB is.
- siscia 10y ago
  > In addition, we show that the new v1 replication protocol has multiple bugs, allowing data loss in all versions up to MongoDB 3.2.11 and 3.4.0-rc4. While the v0 protocol remains broken, patches for v1 are available in MongoDB 3.2.12 and 3.4.0, and now pass the expanded Jepsen test suite. Wait, it is me not understanding what the abstract is trying to say?
  - wslh 10y ago
    I can only think that the article was a bit late and the new MongoDB version has the last word. Beyond this, the article is extremely valuable and it is mandatory to upgrade your MongoDB.
    - aphyr 10y ago
      This analysis was a collaboration with MongoDB over the past three months. MongoDB was able to fix these issues in 3.4.0 because I found them in rc3 and rc4.
      - wslh 10y ago
        Sorry, my fault.
  - ealexhudson 10y ago
    It's saying it was fixed in the final 3.4.0 release (or, maybe, that there's at least a patch available) for the v1 protocol, but that the v0 protocol is unfixable - so, 3.4.0 + V1 protocol and it's solid. Giving aphyr some money to do this makes me think much more of Mongo now; I wouldn't previously have considered it for serious use but this is an excellent demonstration of intent.
- untog 10y ago
  > I think that this really shows how mature of a Database MongoDB is. Or that it took this long for them to pass basic proficiency tests. How do other database systems fare with these tests?
  - jetpacktuxedo 10y ago
    You can check out the other databases he has tested here: http://jepsen.io/analyses http://jepsen.io/analyses That being said, none of the big players in sql are there, so you can't size it up against postgres or mysql.
    - mushi 10y ago
      isn't that because they are not distributed systems?
    - mat_keep 10y ago
      Galera was tested - enabling MySQL to behave like a distributed database
    - kedean 10y ago
      Postgres has been analyzed, I'm not sure why it's not on that list. https://aphyr.com/posts/282-call-me-maybe-postgres https://aphyr.com/posts/282-call-me-maybe-postgres
  - Joeri 10y ago
    If you look at past Jepsen test mostly they fail, and sometimes they fail spectacularly. This is one of those things where if you don't have an automated test suite to continually verify correctness you will end up with bugs.
- jhugg 10y ago
  Note it’s not really “The Jepsen Test” as the tests are all a bit custom and different. Sometimes they’re looking to validate different claims. For example, Kyle integrated some serializability tests into the VoltDB Jepsen testing that wouldn’t apply to Mongo.
- pvg 10y ago
  It's great that they're serious and committing resources to passing these tests. They just did, though, after nearly a decade of development. Which is terrific but it doesn't really show the 'how mature of a database MongoDB is'.
- aphyr 10y ago
  While you are correct that 3.4 now passes, I feel like your interpretation here is a bit optimistic. It's not like I tested Mongo and it passed out of the box--it failed Jepsen tests, and not just with a read anomaly--it lost majority-acknowledged inserts. The v0 protocol still fails--it's fundamentally broken. The v1 bugs are fixed now, but that's a consequence of our collaboration. I'd say the marker of maturity here is that MongoDB has put significant time and effort into correctness: they take clock skew and network partitions as serious failure modes, they've redesigned their replication protocol, added options for stronger reads, and invested in their own correctness test suite, and Jepsen tests, as a part of their CI process.
  - VeejayRampay 10y ago
    I'd like to use the occasion to thank you for the service you provide to the community. The tests themselves, the collaboration with vendors, the competition it fosters and the in-depth technical write-ups I think push the whole ecosystem forward, which everyone eventually benefits from. Much obliged.
    - aphyr 10y ago
      Thanks. :)
    - CJefferson 10y ago
      I agree -- I think the two most important people currently in software are aphyr for database testing, and lcamtuf for fuzzing (AFL). Both are pushing important fundamental software to pass the kind of tests it's embarrassing we haven't been doing for the past 20 years. I mean that in the most positive way, it's hard to get people to do things they've been neglecting for so long!
- digitalzombie 10y ago
  It's one test, paid for by MongoDB. It's good that they work with the dude. But there's only one test in town. I want to see more than just one test from more than one source and seeing their sketchy performance in the past only time will tell if this new version will be any better. > I think that this really shows how mature of a Database MongoDB is. Yeah no. It's a test. We have to wait and see in productions and see the data to prove that. It's also the first version to past this test. It means something but not a whole lot without real production data and other people attesting to it.
metheus 10y ago
I hadn't noticed this before, but aphyr has an elaborate and articulate ethics policy: http://jepsen.io/ethics http://jepsen.io/ethics I like it.
- tptacek 10y ago
  It's really good. I feel like a lot of security consulting shops could benefit from it as well.
- lmm 10y ago
  So a client can get the analysis done and leave it unpublished if they don't like the results? Maybe that's standard practice but it doesn't seem great for users.
  - cjbprime 10y ago
    Where does the policy say that? I see: > "Once I've prepared a written analysis, the client may defer publication by up to three months. This gives the client a chance to understand the faults I've identified, attempt to fix bugs, update documentation, and deploy fixes to users." So, a client can get the analysis done and attempt to defer publication for three months, after which time it'll be published. No?
  - deleted 10y ago
    [deleted]
  - dataminded 10y ago
    Isn't their policy just responsible disclosure? If you find a security bug, shouldn't the company have a chance to fix it before it goes public?
  - aphyr 10y ago
    I agree. In my ideal world I publish everything immediately. There's no "standard practice" that I know of--very few people are doing this kind of work. Behind every one of these analyses is weeks of contract negotiation where I try to convince assorted lawyers & CFOs to go along with my weird, idealistic ethical policies. And conversely, those lawyers and CFOs do their best to balance the desire for correctness with their duty to protect the company. This policy outlines how we compromise. Originally I did offer a client veto, because clients weren't willing to sign without some assurance of control over the outcome. My current standard contract for analyses actually drops that client veto: I have final say on the analysis content and publication. There's still a grace period to allow the vendor to fix bugs & get things in order. I'm adopting this as the standard going forward, but it's been a long road to get there. That said, I do perform private consulting--usually for in-house systems, sometimes for clients that can't afford a full analysis, and sometimes as a precursor to more involved work. That means that yes, vendors may be aware of bugs and I won't have told you about them--but I promise that my public work remains honest and forthcoming.
jlaustill 10y ago
This is great news. Historically I've accepted the risk of data loss and coded checks when needed. I will never rely on my database, regardless of which one I am using for complete data consistency. It is, however, nice to see strides being made towards even better robustness. Go MongoDB!
- koolba 10y ago
  > I will never rely on my database, regardless of which one I am using for complete data consistency. I can't imagine developing any software that involves relationships between entities that does not have data consistency. Check constraints, foreign keys, and data type validation all provide a minimum sanity level of the underlying data that allows your mind to focus on more important things. Otherwise you're entire application is going to be littered with those same sanity checks. I'm not saying that all apps need that type of data store or that there isn't room in this world for NoSQL stores, I mean specifically that complicated interdependencies and validation checks lend themselves well to the relational model.
  - mushi 10y ago
    I think maybe the doubt is based on case like https://aphyr.com/posts/282-jepsen-postgres https://aphyr.com/posts/282-jepsen-postgres which all systems are subject to.
    - continuational 10y ago
      That's not a case of inconsistency. There are just three outcomes for a transaction: Committed, Aborted and Unknown. The client will never experience Committed unless it really is committed, and thus never report that it was successful to the user.
    - throwawayish 10y ago
      Uh, no. That article is a bit of a long winded way to say that 2PC with timeouts is 2PC with timeouts.
tbrock 10y ago
MongoDB receives a fair amount of criticism here but the company is a fantastic place to work. I'm proud that I was able to learn and grow as a developer alongside all of those who have been trying (and succeeding) to make a great database. The team at MongoDB really cares a lot about making the best database product possible. I knew it when I was there and still think so after I've left.
- tyre 10y ago
  When building a company, I care more about culture than anything else. It's the people that make the company great. When choosing a database, caring is only one aspect of it. It's necessary, but not sufficient.
- diziet 10y ago
  MongoDB as a product evolved tremendously in the last few years. All the improvements introduced with WiredTiger and other major releases have seriously improved the performance and reliability of the product. It's not a perfect solution, but the majority of criticism is something that was written about version 1.8.
- bogomipz 10y ago
  >"MongoDB receives a fair amount of criticism here but the company is a fantastic place to work" None of the criticisms I have ever seen or heard about MongoDB were related to Mongo Inc and their office culture but rather their product.
  - omginternets 10y ago
    Exactly, which makes ... >The team at MongoDB really cares a lot about making the best database product possible. ... some what ironic.
    - bogomipz 10y ago
      Right I would hope that that would be implicit :)
  - 10y ago
jasondc 10y ago
Bigger news is that Jepsen tests are now part of the MongoDB continuous integration suite: https://evergreen.mongodb.com/build/mongodb_mongo_master_ubuntu1404_jepsen_bf4385aed5e528a8cf1edb7955c8c2164dda04f0_16_10_28_14_33_06 https://evergreen.mongodb.com/build/mongodb_mongo_master_ubu... Open and available for everyone to see, for every build of MongoDB. Is there another database that has this much transparency? (for every build)
- cjbprime 10y ago
  And big news for me is the existence of that Evergreen tool as an open source project. Looks great! Is anyone else using it? Is there a comparison to Jenkins available?
  - metheus 10y ago
    > Is there a comparison to Jenkins available? Not exactly, but we published a blog post about why we took the time to write a new CI system, and that post does touch upon why we didn't use Jenkins + plugins. https://engineering.mongodb.com/post/evergreen-continuous-integration-why-we-reinvented-the-wheel/ https://engineering.mongodb.com/post/evergreen-continuous-in...
- radicalbyte 10y ago
  Given their start point (a product unfit for public consumption) that is the absolute minimum they need to do. I at least will never trust Mongo for anything but a toy project. There are so many better options out there, options whose technical capabilities are as good as Mongo's marketing.
  - BillFinchDba 10y ago
    I think if you look back objectively, there are very few database platforms that were absolutely "fit for public consumption" right out of the box. Look at all the SQL Server shops out there (mine included) that won't even roll out a new version of SQL Server until it hits SP 1 at a minimum... For MongoDB, If you look forward based on what they are doing now rather than at how early adopters may have had a sub-optimal experience way back when, you'll see a mature product that is consistently improving and is demonstrably reliable. Can you give an example of another option you are referring to?
jdcarter 10y ago
It's been a long way from the "Call Me Maybe: MongoDB" post from years back. Aphyr/Kyle took them to task in so many ways for playing fast and loose with data integrity, and rightly so. MongoDB could have said, "that guy's full of BS, ignore him," but instead they did the smart thing and paid Kyle to help solve the problem. n.b. I can't find the original "Call Me Maybe" post, but this later one [1] is similar. [1]: https://aphyr.com/posts/284-jepsen-mongodb https://aphyr.com/posts/284-jepsen-mongodb
- edsouza 10y ago
  Call Me Maybe post here (updated in Jan 2017): https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-reads https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-read... HN Discussion: https://news.ycombinator.com/item?id=9417773 https://news.ycombinator.com/item?id=9417773
- omginternets 10y ago
  >paid Kyle to help solve the problem. Any more information on this? How did he "solve" the problem? Edit: straight from the horse's mouth [0] [0] https://news.ycombinator.com/user?id=aphyr https://news.ycombinator.com/user?id=aphyr
  - omginternets 10y ago
    Whoops, wrong link. Here's the correct one: https://news.ycombinator.com/item?id=13591048 https://news.ycombinator.com/item?id=13591048 It seems the problems were indeed "solved" and not solved.
    - aphyr 10y ago
      I'm not exactly sure what you're trying to say, but perhaps I can help provide context: In 2013, I performed an unpaid analysis, in my nights and weekends, of MongoDB. I found a bug leading to the loss of acknowledged writes with majority write concern. Mongo fixed this bug within a few weeks. In 2015, I performed a followup test as a part of my work at Stripe. I confirmed dirty reads (which were already documented, though perhaps not well-appreciated), and discovered stale reads (which ran counter to MongoDB's documentation). MongoDB wasn't enthusiastic about that report initially, but got things sorted out and started work on adding majority and linearizable read concerns. I found that writes appeared linearizable. In spring and summer of 2016, MongoDB paid me to help expand the Jepsen tests and hook them up to their internal CI system, so they could use the Jepsen tests to help verify their ongoing work towards linearizable reads. I privately confirmed that MongoDB still failed to prevent stale reads, but the dirty-read failures I'd seen in 2015 appeared to be prevented by majority reads. In fall 2016, MongoDB announced they were almost ready to release 3.4.0 with support for linearizable reads, and paid me to perform a full analysis so they could be more confident in the results. It passed the linearizability tests I had initially written in 2015, but I offered to expand the tests to be more aggressive. Our collaboration resulted in the present analysis, uncovering design flaws in v0 and implementation bugs in v1. MongoDB worked to develop patches prior to 3.4.0's release, and that's why it passes now. :)
geodel 10y ago
I think this is 10th year of MongoDB's existence. This year they seem to be having not just popular but good product. It will be interesting to see if ratio for Good/Popularity is similar for most non-traditional databases.
rocky1138 10y ago
Why would anyone use MongoDB when RethinkDB is available?
- mikegioia 10y ago
  Is RethinkDB even actively developed anymore? I saw they closed up not too long ago.
  - rocky1138 10y ago
    Yes. Development is very active. https://rethinkdb.com/blog/rethinkdb-joins-linux-foundation/ https://rethinkdb.com/blog/rethinkdb-joins-linux-foundation/
- redtree 10y ago
  "Over a third of the Fortune 100 and many of the most successful and innovative web companies rely on MongoDB. " taken from their page : https://www.mongodb.com/mongodb-scale https://www.mongodb.com/mongodb-scale This should give people some insight on why MongoDB is being used in the industry.
  - mat_keep 10y ago
    thats an old stat - its over 50% of the Fortune 100 now
  - h1d 10y ago
    You could replace the word MongoDB to "some crap C code from 20 years ago" but it wouldn't make you want to have it.
- Kiro 10y ago
  Is there anything like mlab.com for RethinkDB? I don't want to manage my own server.
  - atnnn 10y ago
    There is https://www.compose.com/rethinkdb https://www.compose.com/rethinkdb
lowbloodsugar 10y ago
Is protocol v1 something that existing users can migrate to with just a client library swap, or is it a rewrite effort?
- aphyr 10y ago
  Protocol v1 should be largely transparent to clients (at least for standard kv operations). It's a change you make on the server, at the replica set level.
mhoeller 10y ago
Good news to hear! We just finished a fairly big IoT project where we needed to ship a replicated database which was fast, easy to deploy and reliable. We tried other DBs but none was so stable to be shipped with a standalone application completely black boxed to the customer. I was septic in the beginning but after 800 installations in 2 month we have no complains. Also not from early adopters which run now for appr. a year.
ClayFerguson 10y ago
These 'from the ground up' totally all-new-code approaches to DBs are just a scary proposition. Think of the thousands of man-years of effort that went into building MySQL, testing its codebase, and perfecting it's robustness (fail-proof-ness). What does MongoDB bring that couldn't have 'built on top' of MySQL codebase, and used MySQL transational layer as it's underpinnings. Sure, MongoDB gets all its performance gains from delaying writes to the DB (eventually consistent), caching in memory, and dispensing with ACID, however there is nothing about MongoDB that couldn't have been written to use the DB layer of MySQL at the point where it actually does its writes to disk. In this way, MongoDB would have revolutionized the world rather than mostly "fragmenting" the storage space. I guess there are those who will say that even using batch-commits, that MongoDB could never have achieved the performance it currently does (by bypassing any ACID) if it was built on top of MySQL. But regardless, why not focus efforts on improving MySQL batch processing performance rather than throwing it all out, starting from scratch, and writing directly to disk. I know MongoDB became a success, but I think that is 'in spite of' their decision to start from scratch and not 'because of' it. Also think of the power that would be available if there were some relational table capability (true real MySQL ACID) right inside MongoDB whenever it was needed, if they were the 'same animal', rather than having to use two totally and completely separate DBs if you need NoSQL and also ACID in an app, which 99% of apps DO NEED, at some point, once an app grows beyond the round-one funding startup-toy phase and MongoDB falls flat in it's RDB capabilities.
- hmottestad 10y ago
  Probably because there has been a lot of development in distributed databases in the last few years. Raft is fairly new, for one. Taking really old code and trying to make it do new things is very very hard. After you've invested all that time and effort making mysql better, it's still a database owned by oracle. Think how Java would have turned out if Sun made it to be cross compiled to C rather than run in a VM.
  - ClayFerguson 10y ago
    I haven't seen the codebase of MySQL, but it's open source and not "owned" by Oracle. The only reason anyone should want to start over is if that codebase itself was crappy code. I bet it isn't. Implementations of a robust RDBMS takes decades. MySQL is very mature. That doesn't act against it, it acts in favor of it. The better analogy is Linux. I'd be more likely to say "Linux is bad because of age" long before I'd ever say "MySQL is bad only due to age." There are plenty of examples in the Language world also: Go, Rust, etc. All those new languages SHOULD have built on top of Java, for the same reason Mongo should have built on top of MySQL. BTW, Java DID leverage C++. The Java compiler IS written in C. Mongo didn't use any part of MySQL, but Java uses EVERY part of C.
dirkg 10y ago
I see many people say that Mongo is unfot for a production db, but it is actually used in production by many companies. Obviously this is true of many products, but how serious is it in practice? The criteria for 'good enough' can't be passing Jespen because very few db's do. Storing schemaless json is a very valid use case. And the fact is there aren't really many proper NoSql alternatives, other than RethinkDB, which I hope becomes popular, and maybe Couch. e.g. Cassandra is often touted but its a k-v store, not a document db. There's a reason Mongo became and continues to be popular. I still think Rethink is superior in every way.
ahosny 10y ago
This is the first time I hear of Jepsen. I think I'm immersed completely in Java than databases!
_Codemonkeyism 10y ago
I know there are a lot of bad vibes on HN about MongoDB, but I also know several companies which MongoDB enabled with the right libraries to much faster iterate on "schema" and business requirements (in their growth phase) than with other databases at that time (some years ago). The schemaless approach of MongoDB suited their fast changing needs much better than e.g. RDBMs. What currently still makes MongoDB nicer than e.g. Postgres JSON are the libraries embracing schemaless while JSON in PG libs still feels tacked on. Recently I had some bad experience with MongoDB support though. No experience with RethinkDB.

226 points by aphyr 10y ago

MongoDB 3.4 passes the rigorous and tough Jepsen test. Jepsen designs tests to make databases fail in terms of data consistency, correctness, and safety... MongoDB 3.4 passed through their newest tests. I think that this really shows how mature of a Database MongoDB is.

siscia 10y ago
> In addition, we show that the new v1 replication protocol has multiple bugs, allowing data loss in all versions up to MongoDB 3.2.11 and 3.4.0-rc4. While the v0 protocol remains broken, patches for v1 are available in MongoDB 3.2.12 and 3.4.0, and now pass the expanded Jepsen test suite. Wait, it is me not understanding what the abstract is trying to say?
- wslh 10y ago
  I can only think that the article was a bit late and the new MongoDB version has the last word. Beyond this, the article is extremely valuable and it is mandatory to upgrade your MongoDB.
  - aphyr 10y ago
    This analysis was a collaboration with MongoDB over the past three months. MongoDB was able to fix these issues in 3.4.0 because I found them in rc3 and rc4.
    - wslh 10y ago
      Sorry, my fault.
- ealexhudson 10y ago
  It's saying it was fixed in the final 3.4.0 release (or, maybe, that there's at least a patch available) for the v1 protocol, but that the v0 protocol is unfixable - so, 3.4.0 + V1 protocol and it's solid. Giving aphyr some money to do this makes me think much more of Mongo now; I wouldn't previously have considered it for serious use but this is an excellent demonstration of intent.
untog 10y ago
> I think that this really shows how mature of a Database MongoDB is. Or that it took this long for them to pass basic proficiency tests. How do other database systems fare with these tests?
- jetpacktuxedo 10y ago
  You can check out the other databases he has tested here: http://jepsen.io/analyses http://jepsen.io/analyses That being said, none of the big players in sql are there, so you can't size it up against postgres or mysql.
  - mushi 10y ago
    isn't that because they are not distributed systems?
  - mat_keep 10y ago
    Galera was tested - enabling MySQL to behave like a distributed database
  - kedean 10y ago
    Postgres has been analyzed, I'm not sure why it's not on that list. https://aphyr.com/posts/282-call-me-maybe-postgres https://aphyr.com/posts/282-call-me-maybe-postgres
- Joeri 10y ago
  If you look at past Jepsen test mostly they fail, and sometimes they fail spectacularly. This is one of those things where if you don't have an automated test suite to continually verify correctness you will end up with bugs.
jhugg 10y ago
Note it’s not really “The Jepsen Test” as the tests are all a bit custom and different. Sometimes they’re looking to validate different claims. For example, Kyle integrated some serializability tests into the VoltDB Jepsen testing that wouldn’t apply to Mongo.
pvg 10y ago
It's great that they're serious and committing resources to passing these tests. They just did, though, after nearly a decade of development. Which is terrific but it doesn't really show the 'how mature of a database MongoDB is'.
aphyr 10y ago
While you are correct that 3.4 now passes, I feel like your interpretation here is a bit optimistic. It's not like I tested Mongo and it passed out of the box--it failed Jepsen tests, and not just with a read anomaly--it lost majority-acknowledged inserts. The v0 protocol still fails--it's fundamentally broken. The v1 bugs are fixed now, but that's a consequence of our collaboration. I'd say the marker of maturity here is that MongoDB has put significant time and effort into correctness: they take clock skew and network partitions as serious failure modes, they've redesigned their replication protocol, added options for stronger reads, and invested in their own correctness test suite, and Jepsen tests, as a part of their CI process.
- VeejayRampay 10y ago
  I'd like to use the occasion to thank you for the service you provide to the community. The tests themselves, the collaboration with vendors, the competition it fosters and the in-depth technical write-ups I think push the whole ecosystem forward, which everyone eventually benefits from. Much obliged.
  - aphyr 10y ago
    Thanks. :)
  - CJefferson 10y ago
    I agree -- I think the two most important people currently in software are aphyr for database testing, and lcamtuf for fuzzing (AFL). Both are pushing important fundamental software to pass the kind of tests it's embarrassing we haven't been doing for the past 20 years. I mean that in the most positive way, it's hard to get people to do things they've been neglecting for so long!
digitalzombie 10y ago
It's one test, paid for by MongoDB. It's good that they work with the dude. But there's only one test in town. I want to see more than just one test from more than one source and seeing their sketchy performance in the past only time will tell if this new version will be any better. > I think that this really shows how mature of a Database MongoDB is. Yeah no. It's a test. We have to wait and see in productions and see the data to prove that. It's also the first version to past this test. It means something but not a whole lot without real production data and other people attesting to it.

metheus 10y ago

I hadn't noticed this before, but aphyr has an elaborate and articulate ethics policy: http://jepsen.io/ethics http://jepsen.io/ethics I like it.

tptacek 10y ago
It's really good. I feel like a lot of security consulting shops could benefit from it as well.
lmm 10y ago
So a client can get the analysis done and leave it unpublished if they don't like the results? Maybe that's standard practice but it doesn't seem great for users.
- cjbprime 10y ago
  Where does the policy say that? I see: > "Once I've prepared a written analysis, the client may defer publication by up to three months. This gives the client a chance to understand the faults I've identified, attempt to fix bugs, update documentation, and deploy fixes to users." So, a client can get the analysis done and attempt to defer publication for three months, after which time it'll be published. No?
- deleted 10y ago
  [deleted]
- dataminded 10y ago
  Isn't their policy just responsible disclosure? If you find a security bug, shouldn't the company have a chance to fix it before it goes public?
- aphyr 10y ago
  I agree. In my ideal world I publish everything immediately. There's no "standard practice" that I know of--very few people are doing this kind of work. Behind every one of these analyses is weeks of contract negotiation where I try to convince assorted lawyers & CFOs to go along with my weird, idealistic ethical policies. And conversely, those lawyers and CFOs do their best to balance the desire for correctness with their duty to protect the company. This policy outlines how we compromise. Originally I did offer a client veto, because clients weren't willing to sign without some assurance of control over the outcome. My current standard contract for analyses actually drops that client veto: I have final say on the analysis content and publication. There's still a grace period to allow the vendor to fix bugs & get things in order. I'm adopting this as the standard going forward, but it's been a long road to get there. That said, I do perform private consulting--usually for in-house systems, sometimes for clients that can't afford a full analysis, and sometimes as a precursor to more involved work. That means that yes, vendors may be aware of bugs and I won't have told you about them--but I promise that my public work remains honest and forthcoming.

jlaustill 10y ago

This is great news. Historically I've accepted the risk of data loss and coded checks when needed. I will never rely on my database, regardless of which one I am using for complete data consistency. It is, however, nice to see strides being made towards even better robustness. Go MongoDB!

koolba 10y ago
> I will never rely on my database, regardless of which one I am using for complete data consistency. I can't imagine developing any software that involves relationships between entities that does not have data consistency. Check constraints, foreign keys, and data type validation all provide a minimum sanity level of the underlying data that allows your mind to focus on more important things. Otherwise you're entire application is going to be littered with those same sanity checks. I'm not saying that all apps need that type of data store or that there isn't room in this world for NoSQL stores, I mean specifically that complicated interdependencies and validation checks lend themselves well to the relational model.
- mushi 10y ago
  I think maybe the doubt is based on case like https://aphyr.com/posts/282-jepsen-postgres https://aphyr.com/posts/282-jepsen-postgres which all systems are subject to.
  - continuational 10y ago
    That's not a case of inconsistency. There are just three outcomes for a transaction: Committed, Aborted and Unknown. The client will never experience Committed unless it really is committed, and thus never report that it was successful to the user.
  - throwawayish 10y ago
    Uh, no. That article is a bit of a long winded way to say that 2PC with timeouts is 2PC with timeouts.

tbrock 10y ago

MongoDB receives a fair amount of criticism here but the company is a fantastic place to work. I'm proud that I was able to learn and grow as a developer alongside all of those who have been trying (and succeeding) to make a great database. The team at MongoDB really cares a lot about making the best database product possible. I knew it when I was there and still think so after I've left.

tyre 10y ago
When building a company, I care more about culture than anything else. It's the people that make the company great. When choosing a database, caring is only one aspect of it. It's necessary, but not sufficient.
diziet 10y ago
MongoDB as a product evolved tremendously in the last few years. All the improvements introduced with WiredTiger and other major releases have seriously improved the performance and reliability of the product. It's not a perfect solution, but the majority of criticism is something that was written about version 1.8.
bogomipz 10y ago
>"MongoDB receives a fair amount of criticism here but the company is a fantastic place to work" None of the criticisms I have ever seen or heard about MongoDB were related to Mongo Inc and their office culture but rather their product.
- omginternets 10y ago
  Exactly, which makes ... >The team at MongoDB really cares a lot about making the best database product possible. ... some what ironic.
  - bogomipz 10y ago
    Right I would hope that that would be implicit :)
- 10y ago

jasondc 10y ago

Bigger news is that Jepsen tests are now part of the MongoDB continuous integration suite: https://evergreen.mongodb.com/build/mongodb_mongo_master_ubuntu1404_jepsen_bf4385aed5e528a8cf1edb7955c8c2164dda04f0_16_10_28_14_33_06 https://evergreen.mongodb.com/build/mongodb_mongo_master_ubu... Open and available for everyone to see, for every build of MongoDB. Is there another database that has this much transparency? (for every build)

cjbprime 10y ago
And big news for me is the existence of that Evergreen tool as an open source project. Looks great! Is anyone else using it? Is there a comparison to Jenkins available?
- metheus 10y ago
  > Is there a comparison to Jenkins available? Not exactly, but we published a blog post about why we took the time to write a new CI system, and that post does touch upon why we didn't use Jenkins + plugins. https://engineering.mongodb.com/post/evergreen-continuous-integration-why-we-reinvented-the-wheel/ https://engineering.mongodb.com/post/evergreen-continuous-in...
radicalbyte 10y ago
Given their start point (a product unfit for public consumption) that is the absolute minimum they need to do. I at least will never trust Mongo for anything but a toy project. There are so many better options out there, options whose technical capabilities are as good as Mongo's marketing.
- BillFinchDba 10y ago
  I think if you look back objectively, there are very few database platforms that were absolutely "fit for public consumption" right out of the box. Look at all the SQL Server shops out there (mine included) that won't even roll out a new version of SQL Server until it hits SP 1 at a minimum... For MongoDB, If you look forward based on what they are doing now rather than at how early adopters may have had a sub-optimal experience way back when, you'll see a mature product that is consistently improving and is demonstrably reliable. Can you give an example of another option you are referring to?

jdcarter 10y ago

It's been a long way from the "Call Me Maybe: MongoDB" post from years back. Aphyr/Kyle took them to task in so many ways for playing fast and loose with data integrity, and rightly so. MongoDB could have said, "that guy's full of BS, ignore him," but instead they did the smart thing and paid Kyle to help solve the problem. n.b. I can't find the original "Call Me Maybe" post, but this later one [1] is similar. [1]: https://aphyr.com/posts/284-jepsen-mongodb https://aphyr.com/posts/284-jepsen-mongodb

edsouza 10y ago
Call Me Maybe post here (updated in Jan 2017): https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-reads https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-read... HN Discussion: https://news.ycombinator.com/item?id=9417773 https://news.ycombinator.com/item?id=9417773
omginternets 10y ago
>paid Kyle to help solve the problem. Any more information on this? How did he "solve" the problem? Edit: straight from the horse's mouth [0] [0] https://news.ycombinator.com/user?id=aphyr https://news.ycombinator.com/user?id=aphyr
- omginternets 10y ago
  Whoops, wrong link. Here's the correct one: https://news.ycombinator.com/item?id=13591048 https://news.ycombinator.com/item?id=13591048 It seems the problems were indeed "solved" and not solved.
  - aphyr 10y ago
    I'm not exactly sure what you're trying to say, but perhaps I can help provide context: In 2013, I performed an unpaid analysis, in my nights and weekends, of MongoDB. I found a bug leading to the loss of acknowledged writes with majority write concern. Mongo fixed this bug within a few weeks. In 2015, I performed a followup test as a part of my work at Stripe. I confirmed dirty reads (which were already documented, though perhaps not well-appreciated), and discovered stale reads (which ran counter to MongoDB's documentation). MongoDB wasn't enthusiastic about that report initially, but got things sorted out and started work on adding majority and linearizable read concerns. I found that writes appeared linearizable. In spring and summer of 2016, MongoDB paid me to help expand the Jepsen tests and hook them up to their internal CI system, so they could use the Jepsen tests to help verify their ongoing work towards linearizable reads. I privately confirmed that MongoDB still failed to prevent stale reads, but the dirty-read failures I'd seen in 2015 appeared to be prevented by majority reads. In fall 2016, MongoDB announced they were almost ready to release 3.4.0 with support for linearizable reads, and paid me to perform a full analysis so they could be more confident in the results. It passed the linearizability tests I had initially written in 2015, but I offered to expand the tests to be more aggressive. Our collaboration resulted in the present analysis, uncovering design flaws in v0 and implementation bugs in v1. MongoDB worked to develop patches prior to 3.4.0's release, and that's why it passes now. :)

geodel 10y ago

I think this is 10th year of MongoDB's existence. This year they seem to be having not just popular but good product. It will be interesting to see if ratio for Good/Popularity is similar for most non-traditional databases.

rocky1138 10y ago

Why would anyone use MongoDB when RethinkDB is available?

mikegioia 10y ago
Is RethinkDB even actively developed anymore? I saw they closed up not too long ago.
- rocky1138 10y ago
  Yes. Development is very active. https://rethinkdb.com/blog/rethinkdb-joins-linux-foundation/ https://rethinkdb.com/blog/rethinkdb-joins-linux-foundation/
redtree 10y ago
"Over a third of the Fortune 100 and many of the most successful and innovative web companies rely on MongoDB. " taken from their page : https://www.mongodb.com/mongodb-scale https://www.mongodb.com/mongodb-scale This should give people some insight on why MongoDB is being used in the industry.
- mat_keep 10y ago
  thats an old stat - its over 50% of the Fortune 100 now
- h1d 10y ago
  You could replace the word MongoDB to "some crap C code from 20 years ago" but it wouldn't make you want to have it.
Kiro 10y ago
Is there anything like mlab.com for RethinkDB? I don't want to manage my own server.
- atnnn 10y ago
  There is https://www.compose.com/rethinkdb https://www.compose.com/rethinkdb

lowbloodsugar 10y ago

Is protocol v1 something that existing users can migrate to with just a client library swap, or is it a rewrite effort?

aphyr 10y ago
Protocol v1 should be largely transparent to clients (at least for standard kv operations). It's a change you make on the server, at the replica set level.

mhoeller 10y ago

Good news to hear! We just finished a fairly big IoT project where we needed to ship a replicated database which was fast, easy to deploy and reliable. We tried other DBs but none was so stable to be shipped with a standalone application completely black boxed to the customer. I was septic in the beginning but after 800 installations in 2 month we have no complains. Also not from early adopters which run now for appr. a year.

ClayFerguson 10y ago

These 'from the ground up' totally all-new-code approaches to DBs are just a scary proposition. Think of the thousands of man-years of effort that went into building MySQL, testing its codebase, and perfecting it's robustness (fail-proof-ness). What does MongoDB bring that couldn't have 'built on top' of MySQL codebase, and used MySQL transational layer as it's underpinnings. Sure, MongoDB gets all its performance gains from delaying writes to the DB (eventually consistent), caching in memory, and dispensing with ACID, however there is nothing about MongoDB that couldn't have been written to use the DB layer of MySQL at the point where it actually does its writes to disk. In this way, MongoDB would have revolutionized the world rather than mostly "fragmenting" the storage space. I guess there are those who will say that even using batch-commits, that MongoDB could never have achieved the performance it currently does (by bypassing any ACID) if it was built on top of MySQL. But regardless, why not focus efforts on improving MySQL batch processing performance rather than throwing it all out, starting from scratch, and writing directly to disk. I know MongoDB became a success, but I think that is 'in spite of' their decision to start from scratch and not 'because of' it. Also think of the power that would be available if there were some relational table capability (true real MySQL ACID) right inside MongoDB whenever it was needed, if they were the 'same animal', rather than having to use two totally and completely separate DBs if you need NoSQL and also ACID in an app, which 99% of apps DO NEED, at some point, once an app grows beyond the round-one funding startup-toy phase and MongoDB falls flat in it's RDB capabilities.

hmottestad 10y ago
Probably because there has been a lot of development in distributed databases in the last few years. Raft is fairly new, for one. Taking really old code and trying to make it do new things is very very hard. After you've invested all that time and effort making mysql better, it's still a database owned by oracle. Think how Java would have turned out if Sun made it to be cross compiled to C rather than run in a VM.
- ClayFerguson 10y ago
  I haven't seen the codebase of MySQL, but it's open source and not "owned" by Oracle. The only reason anyone should want to start over is if that codebase itself was crappy code. I bet it isn't. Implementations of a robust RDBMS takes decades. MySQL is very mature. That doesn't act against it, it acts in favor of it. The better analogy is Linux. I'd be more likely to say "Linux is bad because of age" long before I'd ever say "MySQL is bad only due to age." There are plenty of examples in the Language world also: Go, Rust, etc. All those new languages SHOULD have built on top of Java, for the same reason Mongo should have built on top of MySQL. BTW, Java DID leverage C++. The Java compiler IS written in C. Mongo didn't use any part of MySQL, but Java uses EVERY part of C.

dirkg 10y ago

I see many people say that Mongo is unfot for a production db, but it is actually used in production by many companies. Obviously this is true of many products, but how serious is it in practice? The criteria for 'good enough' can't be passing Jespen because very few db's do. Storing schemaless json is a very valid use case. And the fact is there aren't really many proper NoSql alternatives, other than RethinkDB, which I hope becomes popular, and maybe Couch. e.g. Cassandra is often touted but its a k-v store, not a document db. There's a reason Mongo became and continues to be popular. I still think Rethink is superior in every way.

ahosny 10y ago

This is the first time I hear of Jepsen. I think I'm immersed completely in Java than databases!

_Codemonkeyism 10y ago

I know there are a lot of bad vibes on HN about MongoDB, but I also know several companies which MongoDB enabled with the right libraries to much faster iterate on "schema" and business requirements (in their growth phase) than with other databases at that time (some years ago). The schemaless approach of MongoDB suited their fast changing needs much better than e.g. RDBMs. What currently still makes MongoDB nicer than e.g. Postgres JSON are the libraries embracing schemaless while JSON in PG libs still feels tacked on. Recently I had some bad experience with MongoDB support though. No experience with RethinkDB.