13 ms·

Jepsen: Dgraph 1.0.2

183 points by aphyr 8y ago

Sir_Cmpwn 8y ago
Important: contrary to the marketing material, dgraph is not open source - it uses the Commons Clause we've been discussing over the past few days. Discussion here: https://github.com/dgraph-io/dgraph/issues/2545 https://github.com/dgraph-io/dgraph/issues/2545
- jadbox 8y ago
  Doesn't open source just mean you have access to read and modify the software? It's the stronger term Libre that implies that you grant all distribution/selling rights to the public.
  - Sir_Cmpwn 8y ago
    No, open source is defined by the Open Source Definition, published by the OSI: https://opensource.org/osd https://opensource.org/osd
    - kbenson 8y ago
      Is the contention by OSI that when they coined the label "open source" on 1998-02-03[1] that it was both unused previously and/or that they have sole claim to that label? Because that's never been how I understood it. I'm pretty sure I understood it to be, at least in it's most encompassing sense, that the source was visible long before the time OSI says it created the label. Edit: s/weakest/encompassing/ 1: https://opensource.org/history https://opensource.org/history
      - Sir_Cmpwn 8y ago
        The generally accepted definition of open source is the OSI definition. OSI also audits and keeps a list of software which qualifies under that definition, which is accepted as the canonical list of open source licenses by everyone. Even if OSI is not your cup of tea (they're a very netural organization, so I don't see why anyone would protest their authority), the FSF is another venerable organization that provides a similar definition of open source and maintains a similar list of open source licenses. You would be hard pressed to find any other respected authority which defines open source in a manner consistent with the Commons Clause. It's deliberately misleading people to ape this terminology to promote software which does not guarantee the fundamental freedoms of open source. If you want to use a proprietary license, own up to it, don't try to get the mindshare of open source when you aren't. The ancient Greek empire might have included Italy in the past, but an Italian who says they're from Greece today is lying.
        kbenson 8y ago
        > It's deliberately misleading people to ape this terminology to promote software which does not guarantee the fundamental freedoms of open source. That may be, but the terminology chosen by OSI is easily misunderstood, exacerbating this problem. When I see "open source", I think mostly of the source and can I see it, possibly whether I am allowed to alter it for myself. Like an "open book" (hey, I can freely annotate my own copy). Whether something is free to sell, or redistribute, or alter and redistribute always seemed like variations on what the easily inferred (even if incorrectly inferred) core meaning of open source. Interestingly, I think the vast majority of people probably think open source means the source is visible, regardless of whether it's an OSI approved license, and regardless of whether they are happy with the license. If that's true, and common usage is at odds with OSI intention, where does that leave us? I think I could easily argue either side in that case.
        simonw 8y ago
        > Interestingly, I think the vast majority of people probably think open source means the source is visible, regardless of whether it's an OSI approved license, and regardless of whether they are happy with the license. It leaves us with an education problem. Ten years ago I don't recall ever seeing people confuse the term Open Source with source-code-available - but the Open Source community was much, much smaller then so I guess the people involved were all on the same page. Now that Open Source has genuinely won (when's the last time you hard a company say they have a policy of NOT using open source?) it seems we have a new terminology problem that I was previously unaware of.
      - cjbprime 8y ago
        Yes, I think so. If you believe that it was used previously, show evidence?
        kbenson 8y ago
        I can't seem to find any, so apparently I'm misremembering. So what were we referring to Linux and the different software running on it back then as? I was running Linux back in 1996, and I can't seem to remember any other way it was referred to.
        dragonwriter 8y ago
        > So what were we referring to Linux and the different software running on it back then as Free Software. Sometimes with “free as in speech” to distinguish from the also common “free as in beer” no-cost but restrictively licensed (or, often at the time, with no express license) software.
  - brazzledazzle 8y ago
    I don’t have any problems with the commons clause in principle but I think they (the commons coalition or whatever they are) should just create a new license. Like commons-apache2.0 or something. Saying you’re Apache licensed is kind of dishonest because it’s not really.
    - Sir_Cmpwn 8y ago
      If they do I'd rather they not use the commons name, which (perhaps intentionally) causes confusion alongside Apache Commons: https://commons.apache.org/ https://commons.apache.org/
      - dragonwriter 8y ago
        I'd rather they didn't use the Apache trademark, too; just write a clear license of your own rather than trying to hijack a license with a radically different purpose but existing goodwill and then reversing it's meaning with an add-on clause.
        reitanqild 8y ago
        It is not a radically different purpose as far as I understand; it is the same purpose: an extremely permissive and business friendly license with one, intentionally very narrow exception.
        dragonwriter 8y ago
        > an extremely permissive and business friendly license In software licensing a “permissive license” is a well-established term for a free or open source license that does not restrict downstream derivatives to use the same license (or the license family) for the downstream contributions, as opposed to a copyleft license which does impose such a requirement. Apache 2.0 is, indeed, a permissive license, and that is a central purpose of its intent. (Anything) + Commons Clause is not a permissive license, and that too is a central purpose of its intent. > with one, intentionally very narrow exception. The restriction in the Common Clause is not “very narrow” (or with very clear boundaries, which makes the area of legal risk larger.)
- tunesmith 8y ago
  When did the definition of "open source" change? I always took "open source" to be an umbrella term that contained both Apache-type licenses and GPL/FSF-type licenses. I get that Commons Clause is more restrictive in ways than GPL, but GPL is definitely open source. So what makes Commons Clause not open source, if GPL is?
  - ealexhudson 8y ago
    You can sell GPL'd software (on its own or as a service), you can't sell Commons Cause software as a service.
  - brazzledazzle 8y ago
    It’s not really Apache though. That’s the problem I see with it. If you tell me you’re Apache licensed I shouldn’t be surprised when I open your license file.
  - Sir_Cmpwn 8y ago
    The definition of "open source" never changed. Apache and the GPL are both open source licenses. Appending the Commons Clause removes the zeroth freedom (FSF lingo) or the first clause (OSI lingo). https://opensource.org/osd https://opensource.org/osd https://www.gnu.org/philosophy/free-sw.en.html https://www.gnu.org/philosophy/free-sw.en.html
    - eecc 8y ago
      Which in today’s world where “we were planning to fund our lives offering professional services around a production deployment” is kindly railroaded by rock bottom AWS prices is a-ok... That’s the server side version of the anti-tivo clause that made the GPLv3. I see a v4 coming
      - Sir_Cmpwn 8y ago
        Withholding judgement on whether or not using a non-open license is the appropriate choice for Dgraph, the problem persists that their language is deliberately misleading. Also, GPLv3 already prevents you from tacking on the Commons Clause, any software using any GPL-family license cannot include the Commons Clause. Finally, AGPL was explicitly designed years ago to address the case you raise, i.e. the server-side Tivo problem.
- mrjn 8y ago
  We changed the references to say liberally licensed. I'll try and write a bigger blog post about the reasoning behind Commons Clause and Dgraph and other companies' adoption of it.
  - Sir_Cmpwn 8y ago
    I think it's disengenuous to say liberally licensed. Liberally licensed might apply to MIT or even Apache alone, but not when you include the Commons Clause. But it's a subjective measure, so you're not making factually incorrect statements even if you are being dishonest. Your updated statement also says "Dgraph is available under an Apache v2.0 based liberal license", which I think is unfairly using the Apache 2.0 name without clarifying that your license in fact is not open source. The language is still dishonest.
    - mrjn 8y ago
      First, to be clear, we are the authors of this software, so our responsibility is to choose our license terms and clearly communicate them to those who want to use the software. We have done that and we are obviously not trying to mislead anyone. We think our licensing statement is easy to understand. We have not changed the Apache license terms. We have added an additional term (Commons Clause) and clearly stated that we are applying the two together. If we had changed the Apache license, we would not have referred to it by that name. But given we have not changed the Apache terms, it seems to us more confusing to arbitrarily refer to those terms by a different name. My take is that Apache + Commons Clause is more liberal than AGPL, something I can expand on in my blog post -- that's clear from our users, who actually appreciated the switch from AGPL to the current license.
      - Sir_Cmpwn 8y ago
        You are free to license your software as you wish, and I am not questioning your right to. There are three separate issues here. >our responsibility is to choose our license terms and clearly communicate them to those who want to use the software You are not doing this. Your website reads "Dgraph is available under an Apache v2.0 based liberal license", which is technically true but also very dishonest. Your software does not use the Apache 2.0 license, and users of it should not expect to enjoy the freedoms granted to them by the Apache 2.0 license. >We have not changed the Apache license terms This is not true. The Apache license explicitly grants freedoms which the following >We have added an additional term changes. Don't wrangle words into passable lies, you're not talking to idiots. Your wording is misleading and dishonest, which are two qualities you should be trying very hard not to associate with your brand. So far you're not doing well. No one is questioning your right to license the software as you wish. But you are not proud of your choice and are using slimy and dishonest language on your marketing material to hide your shame and trick potential users. A separate issue is the subjective one, which is whether or not the commons clause is liberal. I think you are being dishonest here. There are widely accepted definitions for "liberal" among the open source community. The main differentiating factor of a liberal license is that it's not viral. Most people informed on this subject can then neatly and objectively file licenses into liberal and restrictive categories, with licenses like MIT, BSD, and Apache in the liberal camp, and the GPL family of licenses in the restrictive camp. However, the distinction has always been subjective, and nothing like the OSD exists. Additionally, the distinction is only ever applied in the comparison of open source licenses, of which yours is not a member. The realm of proprietary licenses masquarading as open source licenses is unexplored territory, and I'm drawing a line: no matter how restrictive a license like AGPL is, a license which is not even open source in the first place is far from liberal. The third issue is whether or not using the Commons Clause is the right thing for Dgraph to do. In my opinion, it's a slap in the face to everyone who ever used or especially to anyone who ever contributed to Dgraph. In fact, as I was researching dgraph a bit more to see how you approached this issue, it also occurs to me that I can't find any discussion where existing Dgraph contributors were consulted on the matter of relicensing their work, which seems to me that your license change was not only distasteful and harmful to your project, but also illegal. Were all of the contributors consulted and their permission obtained to change the license? I'll certainly be consulting them if not.
- Meai 8y ago
  Well I guess I learned something new today, open source is apparently a term that was defined by somebody and has a counter intuitive meaning. To me it has always meant that the code is available for viewing - nothing more. I assumed this without knowing anything about the OSI. Reference: https://opensource.org/osd https://opensource.org/osd
  - slavak 8y ago
    Yup. Open-source is a whole philosophy. What you're talking about is called source-available software.
- sanderjd 8y ago
  What does this have to do with the Jepsen test discussed here? I'm pretty tired of so many comments sections turning into philosophical debates of open source licensing terms the past few days. Maybe this will naturally bubble down so that the far more interesting technical discussions can bubble to the top, but I fear this sub-thread will eventually drown out the on-topic discussion.
  - Sir_Cmpwn 8y ago
    I normally wouldn't comment, but in this case we're talking about a promotional article that leads readers down a conversion funnel. Because some steps at that conversion funnel are dishonest, I want to make sure that people are aware of what they're actually looking at.
    - sanderjd 8y ago
      This is fair, and it looks like the more interesting discussions have bubbled up above this, so my concern appears to have been misplaced. Cheers.
BMarkmann 8y ago
I honestly don't know that there are any posts I get more excited about on HN than Jepsen analyses. The number of landmines distributed systems create and the ability to suss them out in such detail / depth is really incredible. Kudos.
- striking 8y ago
  Totally agreed. My one and only wish is that they might offer an RSS feed.
  - aphyr 8y ago
    Oh, yeah, that's something I could put together. In the meantime, there's a mailing list and twitter you can subscribe to! https://jepsen.io https://jepsen.io
- amelius 8y ago
  Agree. But ... I sure hope that the new way of developing distributed systems doesn't follow this pattern: 1. write some code 2. submit to Jepsen 3. if errors, goto 1 The problem with this approach is obvious.
  - BMarkmann 8y ago
    You forgot step zero, which is "find terrible example code on SO and copy it"... then 3 goes back to step 0.
  - yashap 8y ago
    With most of the DBs he tests, the authors have very good understandings of distributed systems, and make very carefully considered design choices. They still make mistakes though, because they’re human. Very thorough testing is simply a great way to make any software more robust, distributed systems or otherwise.
  - namibj 8y ago
    I'd hope they find themselves one or more with a master in math on logic proving in ways that a computer can help, and actually prove that their code, at least if compiled to LLVM bytecode, is adhering to the high-level architectural proof. Considering that they'd only need to prove consitency, nothing more, and especially not every little piece of code they are writing, but just the core that handles transactional isolation, not even that the code actually resolves in all cases (being stranded with a stuck cluster that is made to handle a hard reboot without problems is better than a cluster that silently violates constraints), it should not be an infeasible goal. The main issue is that distributed database consistency without trivial, performance-killing locking schemes is too complex to prove when writing or using any trivial, local methods based on e.g. SMT solvers or so. If something like cockroachdb would be proved-consistent on that level, it would be used for applications currently employing pessimistic locking due to a lack of trust in their database, or scaling vertically without really needing to (there are cases which make horizontal scaling cost-prohibitive due to the dependency chains in the algorithms that can solve them, but they can be replaced most of the time).
makmanalp 8y ago
Slightly tangential but dang, this is the best graph and accompanying explanations of consistency models I've ever seen: https://jepsen.io/consistency https://jepsen.io/consistency
jdcarter 8y ago
Does anybody else miss the style of Aphyr's older posts, with animated Barbie dolls and such mocking the product under scrutiny? Examples: https://aphyr.com/posts/317-jepsen-elasticsearch https://aphyr.com/posts/317-jepsen-elasticsearch https://aphyr.com/posts/284-jepsen-mongodb https://aphyr.com/posts/284-jepsen-mongodb I understand this analysis was paid for by Dgraph so GIFs and memes aren't appropriate... but I do miss them.
- mberning 8y ago
  No
- aphyr 8y ago
  Oh, I certainly took no shortage of flack for those early analyses, but I'm glad you appreciated them! Some of my clients have even expressed disappointment that my jepsen.io analyses for them weren't as snarky as the aphyr.com ones! It's a tough situation though--if I were to write snarky analyses, even for unpaid clients, it would raise the possibility that clients could pay for favorable treatment. So... I go through an extensive process, often involving weeks of review with my client and peers, to produce as accurate, professional, and neutral a report as I can. There's another aspect to this: now that I've established a reputation and people take my work seriously, I don't have to be snarky or aggressive to get attention drawn to these issues. And the DB landscape has changed a lot in the last five years! Engineers want to provide formalized safety guarantees. Vendors are taking these failure modes seriously instead of dismissing them as irrelevant! Since I'm getting paid, I can invest more time into making these analyses real collaborations with the vendors. So I see my role now as more about helping people reach their safety goals, and a little less about shaming folks for making systems that didn't live up to their claims. My other motivations--letting users know how to work with the databases they've chosen, and giving case studies to help other engineers test and improve their own systems, well, those are unchanged. :)
  - kjeetgill 8y ago
    Thank you for all of your work. I dove into those posts at the start of my career and they've shaped how I think about my work tremendously. (Along with Brendan Gregg's work!) I still reread this and pass it around at least once our twice a year: https://aphyr.com/posts/313-strong-consistency-models https://aphyr.com/posts/313-strong-consistency-models It's dense and takes some time to digest but I consider it required reading for anyone working on or with distributed systems.
mrjn 8y ago
Author of https://dgraph.io https://dgraph.io here. I'd say this is one of the best collaborations we've done. Due to Jepsen tests, we were able to proactively identify and fix issues in a complex real time distributed graph DB, that is Dgraph. Dgraph's graph sharding, replication and transaction implementation is unique and it took more work to make it all work correctly, while still achieving good performance, and without utilizing any third party system. I'd like to thank Kyle for his work on Dgraph. We'll continue to expand Jepsen tests to try and identify more issues and fix the remaining ones. I'm around if you have questions! Cheers, Manish
- jimktrains2 8y ago
  > Dgraph has made significant progress, but 4 of the 23 issues we identified remain unresolved, including the corruption of data in healthy clusters. This work was funded by Dgraph, Between this and your comment above, it restores a little faith in humanity that there are people out there who honestly want to make their work as good as possible and not paper over cracks.
  - manigandham 8y ago
    If you're talking about funding, many of the recent analyses were funded by the company whose product is being tested. Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen.
    - jimktrains2 8y ago
      > Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen. Yes, but having it published so that everyone sees the actual results without sugar coating or outright ignoring problems is what I meant. I didn't mean that the testing itself was unique, but people funding research and letting the researcher publish it is. (OK, "unique" is much too strong as that's what much of Jepsen does, but I feel like whenever I'm watching the news and there's a public health study talking about how good or not bad X is, it was often funded by an X industry group and reading more about it find it subject to a lot of issues.)
deleted 8y ago
[deleted]
staticassertion 8y ago
I've been using dgraph for a side project. I chose it because I have a really high performance requirement and I find the query language feels 'right'. Glad to see the progress made, and bugs found and fixed, for these Jepsen tests. As a side note, interesting to see a largely Go-based, real-world project plagued with race conditions. Quite a few bugs read "but it was done in a goroutine" or "but mutable capturing in a goroutine", and even a segfault.
- baq 8y ago
  Go probably is the easiest language to get something working with any kind of performance, but it doesn't help much to get something working correctly. After dabbling with Rust I'm amazed that multithreaded programs work at all...
gazarsgo 8y ago
For each Jepsen release my favorite thing is the code for the tests themselves -- https://github.com/jepsen-io/jepsen/tree/master/dgraph/src/jepsen/dgraph https://github.com/jepsen-io/jepsen/tree/master/dgraph/src/j... -- which seems surprisingly unreferenced in the post itself.
- aphyr 8y ago
  Links to the code are throughout the "test design" sections of each report. :)
rdxm 8y ago
gotta say, if you do distributed systems, subject yourself to Jepsen analysis and publish it, you win a truck-load of points in my book.
steven1982 8y ago
Dgraph has been bragging about being used by big companies. Jepsen test clearly shows that there were critical bugs and no sane company can use it in production as primary database, unless they don't care about their data.
- vemv 8y ago
  Same could be said about almost every system tested by Jepsen, so your observation seems unfair.
  - deleted 8y ago
    [deleted]
  - deleted 8y ago
    [deleted]
steven1982 8y ago
Dgraph released 1.0.6 and replication stopped working, I gave up on dgraph. It's just not reliable and immature. It's expected since there is only single developer making commits from months.
- mrjn 8y ago
  So, you created an account on HN half an hour ago and all your comment activity is just Dgraph bashing -- sounds legit!
antpls 8y ago
Was a set of testing tools used to build the analysis? It would be nice to be able to reproduce the results.
drej 8y ago
FYI: There's a very recent podcast with Manish, the author of DGraph, https://www.dataengineeringpodcast.com/dgraph-with-manish-jain-episode-44/ https://www.dataengineeringpodcast.com/dgraph-with-manish-ja...

183 points by aphyr 8y ago

Important: contrary to the marketing material, dgraph is not open source - it uses the Commons Clause we've been discussing over the past few days. Discussion here: https://github.com/dgraph-io/dgraph/issues/2545 https://github.com/dgraph-io/dgraph/issues/2545

jadbox 8y ago
Doesn't open source just mean you have access to read and modify the software? It's the stronger term Libre that implies that you grant all distribution/selling rights to the public.
- Sir_Cmpwn 8y ago
  No, open source is defined by the Open Source Definition, published by the OSI: https://opensource.org/osd https://opensource.org/osd
  - kbenson 8y ago
    Is the contention by OSI that when they coined the label "open source" on 1998-02-03[1] that it was both unused previously and/or that they have sole claim to that label? Because that's never been how I understood it. I'm pretty sure I understood it to be, at least in it's most encompassing sense, that the source was visible long before the time OSI says it created the label. Edit: s/weakest/encompassing/ 1: https://opensource.org/history https://opensource.org/history
    - Sir_Cmpwn 8y ago
      The generally accepted definition of open source is the OSI definition. OSI also audits and keeps a list of software which qualifies under that definition, which is accepted as the canonical list of open source licenses by everyone. Even if OSI is not your cup of tea (they're a very netural organization, so I don't see why anyone would protest their authority), the FSF is another venerable organization that provides a similar definition of open source and maintains a similar list of open source licenses. You would be hard pressed to find any other respected authority which defines open source in a manner consistent with the Commons Clause. It's deliberately misleading people to ape this terminology to promote software which does not guarantee the fundamental freedoms of open source. If you want to use a proprietary license, own up to it, don't try to get the mindshare of open source when you aren't. The ancient Greek empire might have included Italy in the past, but an Italian who says they're from Greece today is lying.
      - kbenson 8y ago
        > It's deliberately misleading people to ape this terminology to promote software which does not guarantee the fundamental freedoms of open source. That may be, but the terminology chosen by OSI is easily misunderstood, exacerbating this problem. When I see "open source", I think mostly of the source and can I see it, possibly whether I am allowed to alter it for myself. Like an "open book" (hey, I can freely annotate my own copy). Whether something is free to sell, or redistribute, or alter and redistribute always seemed like variations on what the easily inferred (even if incorrectly inferred) core meaning of open source. Interestingly, I think the vast majority of people probably think open source means the source is visible, regardless of whether it's an OSI approved license, and regardless of whether they are happy with the license. If that's true, and common usage is at odds with OSI intention, where does that leave us? I think I could easily argue either side in that case.
        simonw 8y ago
        > Interestingly, I think the vast majority of people probably think open source means the source is visible, regardless of whether it's an OSI approved license, and regardless of whether they are happy with the license. It leaves us with an education problem. Ten years ago I don't recall ever seeing people confuse the term Open Source with source-code-available - but the Open Source community was much, much smaller then so I guess the people involved were all on the same page. Now that Open Source has genuinely won (when's the last time you hard a company say they have a policy of NOT using open source?) it seems we have a new terminology problem that I was previously unaware of.
    - cjbprime 8y ago
      Yes, I think so. If you believe that it was used previously, show evidence?
      - kbenson 8y ago
        I can't seem to find any, so apparently I'm misremembering. So what were we referring to Linux and the different software running on it back then as? I was running Linux back in 1996, and I can't seem to remember any other way it was referred to.
        dragonwriter 8y ago
        > So what were we referring to Linux and the different software running on it back then as Free Software. Sometimes with “free as in speech” to distinguish from the also common “free as in beer” no-cost but restrictively licensed (or, often at the time, with no express license) software.
- brazzledazzle 8y ago
  I don’t have any problems with the commons clause in principle but I think they (the commons coalition or whatever they are) should just create a new license. Like commons-apache2.0 or something. Saying you’re Apache licensed is kind of dishonest because it’s not really.
  - Sir_Cmpwn 8y ago
    If they do I'd rather they not use the commons name, which (perhaps intentionally) causes confusion alongside Apache Commons: https://commons.apache.org/ https://commons.apache.org/
    - dragonwriter 8y ago
      I'd rather they didn't use the Apache trademark, too; just write a clear license of your own rather than trying to hijack a license with a radically different purpose but existing goodwill and then reversing it's meaning with an add-on clause.
      - reitanqild 8y ago
        It is not a radically different purpose as far as I understand; it is the same purpose: an extremely permissive and business friendly license with one, intentionally very narrow exception.
        dragonwriter 8y ago
        > an extremely permissive and business friendly license In software licensing a “permissive license” is a well-established term for a free or open source license that does not restrict downstream derivatives to use the same license (or the license family) for the downstream contributions, as opposed to a copyleft license which does impose such a requirement. Apache 2.0 is, indeed, a permissive license, and that is a central purpose of its intent. (Anything) + Commons Clause is not a permissive license, and that too is a central purpose of its intent. > with one, intentionally very narrow exception. The restriction in the Common Clause is not “very narrow” (or with very clear boundaries, which makes the area of legal risk larger.)
tunesmith 8y ago
When did the definition of "open source" change? I always took "open source" to be an umbrella term that contained both Apache-type licenses and GPL/FSF-type licenses. I get that Commons Clause is more restrictive in ways than GPL, but GPL is definitely open source. So what makes Commons Clause not open source, if GPL is?
- ealexhudson 8y ago
  You can sell GPL'd software (on its own or as a service), you can't sell Commons Cause software as a service.
- brazzledazzle 8y ago
  It’s not really Apache though. That’s the problem I see with it. If you tell me you’re Apache licensed I shouldn’t be surprised when I open your license file.
- Sir_Cmpwn 8y ago
  The definition of "open source" never changed. Apache and the GPL are both open source licenses. Appending the Commons Clause removes the zeroth freedom (FSF lingo) or the first clause (OSI lingo). https://opensource.org/osd https://opensource.org/osd https://www.gnu.org/philosophy/free-sw.en.html https://www.gnu.org/philosophy/free-sw.en.html
  - eecc 8y ago
    Which in today’s world where “we were planning to fund our lives offering professional services around a production deployment” is kindly railroaded by rock bottom AWS prices is a-ok... That’s the server side version of the anti-tivo clause that made the GPLv3. I see a v4 coming
    - Sir_Cmpwn 8y ago
      Withholding judgement on whether or not using a non-open license is the appropriate choice for Dgraph, the problem persists that their language is deliberately misleading. Also, GPLv3 already prevents you from tacking on the Commons Clause, any software using any GPL-family license cannot include the Commons Clause. Finally, AGPL was explicitly designed years ago to address the case you raise, i.e. the server-side Tivo problem.
mrjn 8y ago
We changed the references to say liberally licensed. I'll try and write a bigger blog post about the reasoning behind Commons Clause and Dgraph and other companies' adoption of it.
- Sir_Cmpwn 8y ago
  I think it's disengenuous to say liberally licensed. Liberally licensed might apply to MIT or even Apache alone, but not when you include the Commons Clause. But it's a subjective measure, so you're not making factually incorrect statements even if you are being dishonest. Your updated statement also says "Dgraph is available under an Apache v2.0 based liberal license", which I think is unfairly using the Apache 2.0 name without clarifying that your license in fact is not open source. The language is still dishonest.
  - mrjn 8y ago
    First, to be clear, we are the authors of this software, so our responsibility is to choose our license terms and clearly communicate them to those who want to use the software. We have done that and we are obviously not trying to mislead anyone. We think our licensing statement is easy to understand. We have not changed the Apache license terms. We have added an additional term (Commons Clause) and clearly stated that we are applying the two together. If we had changed the Apache license, we would not have referred to it by that name. But given we have not changed the Apache terms, it seems to us more confusing to arbitrarily refer to those terms by a different name. My take is that Apache + Commons Clause is more liberal than AGPL, something I can expand on in my blog post -- that's clear from our users, who actually appreciated the switch from AGPL to the current license.
    - Sir_Cmpwn 8y ago
      You are free to license your software as you wish, and I am not questioning your right to. There are three separate issues here. >our responsibility is to choose our license terms and clearly communicate them to those who want to use the software You are not doing this. Your website reads "Dgraph is available under an Apache v2.0 based liberal license", which is technically true but also very dishonest. Your software does not use the Apache 2.0 license, and users of it should not expect to enjoy the freedoms granted to them by the Apache 2.0 license. >We have not changed the Apache license terms This is not true. The Apache license explicitly grants freedoms which the following >We have added an additional term changes. Don't wrangle words into passable lies, you're not talking to idiots. Your wording is misleading and dishonest, which are two qualities you should be trying very hard not to associate with your brand. So far you're not doing well. No one is questioning your right to license the software as you wish. But you are not proud of your choice and are using slimy and dishonest language on your marketing material to hide your shame and trick potential users. A separate issue is the subjective one, which is whether or not the commons clause is liberal. I think you are being dishonest here. There are widely accepted definitions for "liberal" among the open source community. The main differentiating factor of a liberal license is that it's not viral. Most people informed on this subject can then neatly and objectively file licenses into liberal and restrictive categories, with licenses like MIT, BSD, and Apache in the liberal camp, and the GPL family of licenses in the restrictive camp. However, the distinction has always been subjective, and nothing like the OSD exists. Additionally, the distinction is only ever applied in the comparison of open source licenses, of which yours is not a member. The realm of proprietary licenses masquarading as open source licenses is unexplored territory, and I'm drawing a line: no matter how restrictive a license like AGPL is, a license which is not even open source in the first place is far from liberal. The third issue is whether or not using the Commons Clause is the right thing for Dgraph to do. In my opinion, it's a slap in the face to everyone who ever used or especially to anyone who ever contributed to Dgraph. In fact, as I was researching dgraph a bit more to see how you approached this issue, it also occurs to me that I can't find any discussion where existing Dgraph contributors were consulted on the matter of relicensing their work, which seems to me that your license change was not only distasteful and harmful to your project, but also illegal. Were all of the contributors consulted and their permission obtained to change the license? I'll certainly be consulting them if not.
Meai 8y ago
Well I guess I learned something new today, open source is apparently a term that was defined by somebody and has a counter intuitive meaning. To me it has always meant that the code is available for viewing - nothing more. I assumed this without knowing anything about the OSI. Reference: https://opensource.org/osd https://opensource.org/osd
- slavak 8y ago
  Yup. Open-source is a whole philosophy. What you're talking about is called source-available software.
sanderjd 8y ago
What does this have to do with the Jepsen test discussed here? I'm pretty tired of so many comments sections turning into philosophical debates of open source licensing terms the past few days. Maybe this will naturally bubble down so that the far more interesting technical discussions can bubble to the top, but I fear this sub-thread will eventually drown out the on-topic discussion.
- Sir_Cmpwn 8y ago
  I normally wouldn't comment, but in this case we're talking about a promotional article that leads readers down a conversion funnel. Because some steps at that conversion funnel are dishonest, I want to make sure that people are aware of what they're actually looking at.
  - sanderjd 8y ago
    This is fair, and it looks like the more interesting discussions have bubbled up above this, so my concern appears to have been misplaced. Cheers.

BMarkmann 8y ago

I honestly don't know that there are any posts I get more excited about on HN than Jepsen analyses. The number of landmines distributed systems create and the ability to suss them out in such detail / depth is really incredible. Kudos.

striking 8y ago
Totally agreed. My one and only wish is that they might offer an RSS feed.
- aphyr 8y ago
  Oh, yeah, that's something I could put together. In the meantime, there's a mailing list and twitter you can subscribe to! https://jepsen.io https://jepsen.io
amelius 8y ago
Agree. But ... I sure hope that the new way of developing distributed systems doesn't follow this pattern: 1. write some code 2. submit to Jepsen 3. if errors, goto 1 The problem with this approach is obvious.
- BMarkmann 8y ago
  You forgot step zero, which is "find terrible example code on SO and copy it"... then 3 goes back to step 0.
- yashap 8y ago
  With most of the DBs he tests, the authors have very good understandings of distributed systems, and make very carefully considered design choices. They still make mistakes though, because they’re human. Very thorough testing is simply a great way to make any software more robust, distributed systems or otherwise.
- namibj 8y ago
  I'd hope they find themselves one or more with a master in math on logic proving in ways that a computer can help, and actually prove that their code, at least if compiled to LLVM bytecode, is adhering to the high-level architectural proof. Considering that they'd only need to prove consitency, nothing more, and especially not every little piece of code they are writing, but just the core that handles transactional isolation, not even that the code actually resolves in all cases (being stranded with a stuck cluster that is made to handle a hard reboot without problems is better than a cluster that silently violates constraints), it should not be an infeasible goal. The main issue is that distributed database consistency without trivial, performance-killing locking schemes is too complex to prove when writing or using any trivial, local methods based on e.g. SMT solvers or so. If something like cockroachdb would be proved-consistent on that level, it would be used for applications currently employing pessimistic locking due to a lack of trust in their database, or scaling vertically without really needing to (there are cases which make horizontal scaling cost-prohibitive due to the dependency chains in the algorithms that can solve them, but they can be replaced most of the time).

makmanalp 8y ago

Slightly tangential but dang, this is the best graph and accompanying explanations of consistency models I've ever seen: https://jepsen.io/consistency https://jepsen.io/consistency

jdcarter 8y ago

Does anybody else miss the style of Aphyr's older posts, with animated Barbie dolls and such mocking the product under scrutiny? Examples: https://aphyr.com/posts/317-jepsen-elasticsearch https://aphyr.com/posts/317-jepsen-elasticsearch https://aphyr.com/posts/284-jepsen-mongodb https://aphyr.com/posts/284-jepsen-mongodb I understand this analysis was paid for by Dgraph so GIFs and memes aren't appropriate... but I do miss them.

mberning 8y ago
No
aphyr 8y ago
Oh, I certainly took no shortage of flack for those early analyses, but I'm glad you appreciated them! Some of my clients have even expressed disappointment that my jepsen.io analyses for them weren't as snarky as the aphyr.com ones! It's a tough situation though--if I were to write snarky analyses, even for unpaid clients, it would raise the possibility that clients could pay for favorable treatment. So... I go through an extensive process, often involving weeks of review with my client and peers, to produce as accurate, professional, and neutral a report as I can. There's another aspect to this: now that I've established a reputation and people take my work seriously, I don't have to be snarky or aggressive to get attention drawn to these issues. And the DB landscape has changed a lot in the last five years! Engineers want to provide formalized safety guarantees. Vendors are taking these failure modes seriously instead of dismissing them as irrelevant! Since I'm getting paid, I can invest more time into making these analyses real collaborations with the vendors. So I see my role now as more about helping people reach their safety goals, and a little less about shaming folks for making systems that didn't live up to their claims. My other motivations--letting users know how to work with the databases they've chosen, and giving case studies to help other engineers test and improve their own systems, well, those are unchanged. :)
- kjeetgill 8y ago
  Thank you for all of your work. I dove into those posts at the start of my career and they've shaped how I think about my work tremendously. (Along with Brendan Gregg's work!) I still reread this and pass it around at least once our twice a year: https://aphyr.com/posts/313-strong-consistency-models https://aphyr.com/posts/313-strong-consistency-models It's dense and takes some time to digest but I consider it required reading for anyone working on or with distributed systems.

mrjn 8y ago

Author of https://dgraph.io https://dgraph.io here. I'd say this is one of the best collaborations we've done. Due to Jepsen tests, we were able to proactively identify and fix issues in a complex real time distributed graph DB, that is Dgraph. Dgraph's graph sharding, replication and transaction implementation is unique and it took more work to make it all work correctly, while still achieving good performance, and without utilizing any third party system. I'd like to thank Kyle for his work on Dgraph. We'll continue to expand Jepsen tests to try and identify more issues and fix the remaining ones. I'm around if you have questions! Cheers, Manish

jimktrains2 8y ago
> Dgraph has made significant progress, but 4 of the 23 issues we identified remain unresolved, including the corruption of data in healthy clusters. This work was funded by Dgraph, Between this and your comment above, it restores a little faith in humanity that there are people out there who honestly want to make their work as good as possible and not paper over cracks.
- manigandham 8y ago
  If you're talking about funding, many of the recent analyses were funded by the company whose product is being tested. Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen.
  - jimktrains2 8y ago
    > Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen. Yes, but having it published so that everyone sees the actual results without sugar coating or outright ignoring problems is what I meant. I didn't mean that the testing itself was unique, but people funding research and letting the researcher publish it is. (OK, "unique" is much too strong as that's what much of Jepsen does, but I feel like whenever I'm watching the news and there's a public health study talking about how good or not bad X is, it was often funded by an X industry group and reading more about it find it subject to a lot of issues.)

deleted 8y ago

[deleted]

staticassertion 8y ago

I've been using dgraph for a side project. I chose it because I have a really high performance requirement and I find the query language feels 'right'. Glad to see the progress made, and bugs found and fixed, for these Jepsen tests. As a side note, interesting to see a largely Go-based, real-world project plagued with race conditions. Quite a few bugs read "but it was done in a goroutine" or "but mutable capturing in a goroutine", and even a segfault.

baq 8y ago
Go probably is the easiest language to get something working with any kind of performance, but it doesn't help much to get something working correctly. After dabbling with Rust I'm amazed that multithreaded programs work at all...

gazarsgo 8y ago

For each Jepsen release my favorite thing is the code for the tests themselves -- https://github.com/jepsen-io/jepsen/tree/master/dgraph/src/jepsen/dgraph https://github.com/jepsen-io/jepsen/tree/master/dgraph/src/j... -- which seems surprisingly unreferenced in the post itself.

aphyr 8y ago
Links to the code are throughout the "test design" sections of each report. :)

rdxm 8y ago

gotta say, if you do distributed systems, subject yourself to Jepsen analysis and publish it, you win a truck-load of points in my book.

steven1982 8y ago

Dgraph has been bragging about being used by big companies. Jepsen test clearly shows that there were critical bugs and no sane company can use it in production as primary database, unless they don't care about their data.

vemv 8y ago
Same could be said about almost every system tested by Jepsen, so your observation seems unfair.
- deleted 8y ago
  [deleted]
- deleted 8y ago
  [deleted]

steven1982 8y ago

Dgraph released 1.0.6 and replication stopped working, I gave up on dgraph. It's just not reliable and immature. It's expected since there is only single developer making commits from months.

mrjn 8y ago
So, you created an account on HN half an hour ago and all your comment activity is just Dgraph bashing -- sounds legit!

antpls 8y ago

Was a set of testing tools used to build the analysis? It would be nice to be able to reproduce the results.

drej 8y ago

FYI: There's a very recent podcast with Manish, the author of DGraph, https://www.dataengineeringpodcast.com/dgraph-with-manish-jain-episode-44/ https://www.dataengineeringpodcast.com/dgraph-with-manish-ja...