10 ms·

Bigger news is that Jepsen tests are now part of the MongoDB continuous integration suite: https://evergreen.mongodb.com/build/mongodb_mongo_master_ubuntu1404_j

by jasondc 10y ago

Bigger news is that Jepsen tests are now part of the MongoDB continuous integration suite: https://evergreen.mongodb.com/build/mongodb_mongo_master_ubuntu1404_jepsen_bf4385aed5e528a8cf1edb7955c8c2164dda04f0_16_10_28_14_33_06 https://evergreen.mongodb.com/build/mongodb_mongo_master_ubu... Open and available for everyone to see, for every build of MongoDB. Is there another database that has this much transparency? (for every build)

cjbprime 10y ago
And big news for me is the existence of that Evergreen tool as an open source project. Looks great! Is anyone else using it? Is there a comparison to Jenkins available?
- metheus 10y ago
  > Is there a comparison to Jenkins available? Not exactly, but we published a blog post about why we took the time to write a new CI system, and that post does touch upon why we didn't use Jenkins + plugins. https://engineering.mongodb.com/post/evergreen-continuous-integration-why-we-reinvented-the-wheel/ https://engineering.mongodb.com/post/evergreen-continuous-in...
radicalbyte 10y ago
Given their start point (a product unfit for public consumption) that is the absolute minimum they need to do. I at least will never trust Mongo for anything but a toy project. There are so many better options out there, options whose technical capabilities are as good as Mongo's marketing.
- BillFinchDba 10y ago
  I think if you look back objectively, there are very few database platforms that were absolutely "fit for public consumption" right out of the box. Look at all the SQL Server shops out there (mine included) that won't even roll out a new version of SQL Server until it hits SP 1 at a minimum... For MongoDB, If you look forward based on what they are doing now rather than at how early adopters may have had a sub-optimal experience way back when, you'll see a mature product that is consistently improving and is demonstrably reliable. Can you give an example of another option you are referring to?
  - digitalzombie 10y ago
    > Can you give an example of another option you are referring to? That depends on the data. What type of data you have and what you want to do with it. MongoDB isn't data specific and it claim to fame is flexible data structure. If you want fast write and look up with very little relation Cassandra is good. If you want searchable text document then anything that is base on Lucene is good (ES, Solr, Raven). If you want time series there are few out there but it's a niche. Likewise if you want graph data then there are NodeJS, Titan, etc.. MongoDB at most company I worked with was use because they don't think about what type of data it is and what performance they want. They want to store unstructure data cause it's easy. I personally think it's a cop out, especially as a statistician/programmer.
    - dijit 10y ago
      For me, mongodb was like a document-store on tmpfs. And, I can't sell that to people I work with.
    - elvinyung 10y ago
      > if you want graph data then there are NodeJS I think you meant Neo4j.
    - mabn 10y ago
      What if I want filtering by several criteria (on a table with 1k columns) and simple aggregations, but I don't need full-text search* ? I'm still looking :( * I only want starts-with and contains on strings.
  - ralfn 10y ago
    Not the OP, but RethinkDB is superior in many ways including stability, integrity and the feature set for pretty much every use case you would consider using MongoDB. But with Jespen tests MongoDB can finally be considered a contender. Its not like competent teams were using it in production. Right?
    - hendzen 10y ago
- tigershark 10y ago
  In the same NoSql space as mongo db? I can't remember not even one that passed fully jepsen test. Can you post some links to the better NoSql options?
  - GiorgioG 10y ago
    RethinkDB https://aphyr.com/posts/329-jepsen-rethinkdb-2-1-5 https://aphyr.com/posts/329-jepsen-rethinkdb-2-1-5
  - fictioncircle 10y ago
    https://aphyr.com/posts/329-jepsen-rethinkdb-2-1-5 https://aphyr.com/posts/329-jepsen-rethinkdb-2-1-5 > As far as I can ascertain, RethinkDB’s safety claims are accurate. You can lose updates if you write with anything less than majority, and see assorted read anomalies with single or outdated reads, but majority/majority appears linearizable. The safety claims are accurate but like all databases, its never 100% perfect as there are tradeoffs. https://aphyr.com/posts/330-jepsen-rethinkdb-2-2-3-reconfiguration https://aphyr.com/posts/330-jepsen-rethinkdb-2-2-3-reconfigu... > That reordering was only possible because of a special workaround for a bug in an older version of RethinkDB. > What are the risks to users? The RethinkDB team and I suspect it’s unlikely this bug will occur outside of stress testing. Cluster reconfiguration is typically infrequent, and users would need a specific series of network failures or other message delays which happen to cut the replicas apart—in a way which allows both network components to find independent majorities for their respective table configurations. In Jepsen tests, it usually takes tens to hundreds of partition/reconfigure rounds to trigger this bug. This isn't really an architectural failure (as you see repeatedly with MongoDB) but an implementation bug. Compared to the failures in MongoDB that are frequently architectural (rather than implementation bugs), RethinkDB passed easily and quickly fixed the implementation bugs.
  - radicalbyte 10y ago
    Rethink and Cassandra both fare far better in Jepsen. Depending on the exact requirements Postgres (in the right configuration) and Redis are better options. I'd argue that for 90% of MongoDB users - the group whose requirements were met by MySQL/MyISAM - would be better served by Postgres or MySQL with a proper storage engine.
- diminoten 10y ago
  I think you're going to end up on the wrong side of this one, if you're not willing to change your opinion based on new info. It's a big deal, his second to last sentence in the intro: > While the v0 protocol remains broken, patches for v1 are available in MongoDB 3.2.12 and 3.4.0, and now pass the expanded Jepsen test suite.
  - radicalbyte 10y ago
    I'll change my opinion once they've actually fixed it. The same way my opinion of MySQL changed when it finally matured (around 5.0 when they added proper transactions).
    - aphyr 10y ago
      Er, to be clear, they have actually fixed those issues in the v1 protocol. There may be other bugs, but I haven't been able to find them yet.
- chx 10y ago
  We built examiner.com on MongoDB in 2010. Yes, that was before the journal was introduced. Would you call a then Quantcast Top 100 site a "toy project"? It served well into 2016.
  - radicalbyte 10y ago
    I said that I would only use it for a toy project. Or at least a project for whose requirements resulted in MongoDB being a good choice. For example a content site where losing some data doesn't matter and whose hot datasets would never expand above the memory available by my budget. Given the projects I chose to work on, MongoDB would be a bad choice. So I would only use it for a toy project right now. In 3-5 years time I expect that it will be a very good choice, given that they've pretty much won and can hire all of the good engineering talent. The parallels with MySQL are really astounding; I just hope that they don't end up being sold to Oracle or Microsoft.
francoisdevlin 10y ago
sqlite. It's truly magnificent https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html

by jasondc 10y ago

cjbprime 10y ago

And big news for me is the existence of that Evergreen tool as an open source project. Looks great! Is anyone else using it? Is there a comparison to Jenkins available?

metheus 10y ago
> Is there a comparison to Jenkins available? Not exactly, but we published a blog post about why we took the time to write a new CI system, and that post does touch upon why we didn't use Jenkins + plugins. https://engineering.mongodb.com/post/evergreen-continuous-integration-why-we-reinvented-the-wheel/ https://engineering.mongodb.com/post/evergreen-continuous-in...

radicalbyte 10y ago

Given their start point (a product unfit for public consumption) that is the absolute minimum they need to do. I at least will never trust Mongo for anything but a toy project. There are so many better options out there, options whose technical capabilities are as good as Mongo's marketing.

BillFinchDba 10y ago
I think if you look back objectively, there are very few database platforms that were absolutely "fit for public consumption" right out of the box. Look at all the SQL Server shops out there (mine included) that won't even roll out a new version of SQL Server until it hits SP 1 at a minimum... For MongoDB, If you look forward based on what they are doing now rather than at how early adopters may have had a sub-optimal experience way back when, you'll see a mature product that is consistently improving and is demonstrably reliable. Can you give an example of another option you are referring to?
- digitalzombie 10y ago
  > Can you give an example of another option you are referring to? That depends on the data. What type of data you have and what you want to do with it. MongoDB isn't data specific and it claim to fame is flexible data structure. If you want fast write and look up with very little relation Cassandra is good. If you want searchable text document then anything that is base on Lucene is good (ES, Solr, Raven). If you want time series there are few out there but it's a niche. Likewise if you want graph data then there are NodeJS, Titan, etc.. MongoDB at most company I worked with was use because they don't think about what type of data it is and what performance they want. They want to store unstructure data cause it's easy. I personally think it's a cop out, especially as a statistician/programmer.
  - dijit 10y ago
    For me, mongodb was like a document-store on tmpfs. And, I can't sell that to people I work with.
  - elvinyung 10y ago
    > if you want graph data then there are NodeJS I think you meant Neo4j.
  - mabn 10y ago
    What if I want filtering by several criteria (on a table with 1k columns) and simple aggregations, but I don't need full-text search* ? I'm still looking :( * I only want starts-with and contains on strings.
- ralfn 10y ago
  Not the OP, but RethinkDB is superior in many ways including stability, integrity and the feature set for pretty much every use case you would consider using MongoDB. But with Jespen tests MongoDB can finally be considered a contender. Its not like competent teams were using it in production. Right?
  - hendzen 10y ago
tigershark 10y ago
In the same NoSql space as mongo db? I can't remember not even one that passed fully jepsen test. Can you post some links to the better NoSql options?
- GiorgioG 10y ago
  RethinkDB https://aphyr.com/posts/329-jepsen-rethinkdb-2-1-5 https://aphyr.com/posts/329-jepsen-rethinkdb-2-1-5
- fictioncircle 10y ago
  https://aphyr.com/posts/329-jepsen-rethinkdb-2-1-5 https://aphyr.com/posts/329-jepsen-rethinkdb-2-1-5 > As far as I can ascertain, RethinkDB’s safety claims are accurate. You can lose updates if you write with anything less than majority, and see assorted read anomalies with single or outdated reads, but majority/majority appears linearizable. The safety claims are accurate but like all databases, its never 100% perfect as there are tradeoffs. https://aphyr.com/posts/330-jepsen-rethinkdb-2-2-3-reconfiguration https://aphyr.com/posts/330-jepsen-rethinkdb-2-2-3-reconfigu... > That reordering was only possible because of a special workaround for a bug in an older version of RethinkDB. > What are the risks to users? The RethinkDB team and I suspect it’s unlikely this bug will occur outside of stress testing. Cluster reconfiguration is typically infrequent, and users would need a specific series of network failures or other message delays which happen to cut the replicas apart—in a way which allows both network components to find independent majorities for their respective table configurations. In Jepsen tests, it usually takes tens to hundreds of partition/reconfigure rounds to trigger this bug. This isn't really an architectural failure (as you see repeatedly with MongoDB) but an implementation bug. Compared to the failures in MongoDB that are frequently architectural (rather than implementation bugs), RethinkDB passed easily and quickly fixed the implementation bugs.
- radicalbyte 10y ago
  Rethink and Cassandra both fare far better in Jepsen. Depending on the exact requirements Postgres (in the right configuration) and Redis are better options. I'd argue that for 90% of MongoDB users - the group whose requirements were met by MySQL/MyISAM - would be better served by Postgres or MySQL with a proper storage engine.
diminoten 10y ago
I think you're going to end up on the wrong side of this one, if you're not willing to change your opinion based on new info. It's a big deal, his second to last sentence in the intro: > While the v0 protocol remains broken, patches for v1 are available in MongoDB 3.2.12 and 3.4.0, and now pass the expanded Jepsen test suite.
- radicalbyte 10y ago
  I'll change my opinion once they've actually fixed it. The same way my opinion of MySQL changed when it finally matured (around 5.0 when they added proper transactions).
  - aphyr 10y ago
    Er, to be clear, they have actually fixed those issues in the v1 protocol. There may be other bugs, but I haven't been able to find them yet.
chx 10y ago
We built examiner.com on MongoDB in 2010. Yes, that was before the journal was introduced. Would you call a then Quantcast Top 100 site a "toy project"? It served well into 2016.
- radicalbyte 10y ago
  I said that I would only use it for a toy project. Or at least a project for whose requirements resulted in MongoDB being a good choice. For example a content site where losing some data doesn't matter and whose hot datasets would never expand above the memory available by my budget. Given the projects I chose to work on, MongoDB would be a bad choice. So I would only use it for a toy project right now. In 3-5 years time I expect that it will be a very good choice, given that they've pretty much won and can hire all of the good engineering talent. The parallels with MySQL are really astounding; I just hope that they don't end up being sold to Oracle or Microsoft.

francoisdevlin 10y ago

sqlite. It's truly magnificent https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html