5 ms·

Author of https://dgraph.io https://dgraph.io here. I'd say this is one of the best collaborations we've done. Due to Jepsen tests, we were able to proactively

by mrjn 8y ago

Author of https://dgraph.io https://dgraph.io here. I'd say this is one of the best collaborations we've done. Due to Jepsen tests, we were able to proactively identify and fix issues in a complex real time distributed graph DB, that is Dgraph. Dgraph's graph sharding, replication and transaction implementation is unique and it took more work to make it all work correctly, while still achieving good performance, and without utilizing any third party system. I'd like to thank Kyle for his work on Dgraph. We'll continue to expand Jepsen tests to try and identify more issues and fix the remaining ones. I'm around if you have questions! Cheers, Manish

jimktrains2 8y ago
> Dgraph has made significant progress, but 4 of the 23 issues we identified remain unresolved, including the corruption of data in healthy clusters. This work was funded by Dgraph, Between this and your comment above, it restores a little faith in humanity that there are people out there who honestly want to make their work as good as possible and not paper over cracks.
- manigandham 8y ago
  If you're talking about funding, many of the recent analyses were funded by the company whose product is being tested. Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen.
  - jimktrains2 8y ago
    > Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen. Yes, but having it published so that everyone sees the actual results without sugar coating or outright ignoring problems is what I meant. I didn't mean that the testing itself was unique, but people funding research and letting the researcher publish it is. (OK, "unique" is much too strong as that's what much of Jepsen does, but I feel like whenever I'm watching the news and there's a public health study talking about how good or not bad X is, it was often funded by an X industry group and reading more about it find it subject to a lot of issues.)
- mrjn 8y ago
  Thank you! This is the best acknowledgment of our work I've seen so far. A lot of blood, sweat, hairs, frustrations, and tears has gone into building Dgraph -- and this comment by you makes me feel proud of what we're building, and the community of people using it.
  - beagledude 8y ago
    I'll echo that as well. You all are doing it right!
beagledude 8y ago
Were there any badger related issues? There doesn't seem to be reading the article.
- mrjn 8y ago
  None were found based on Jepsen testing done. We had done a separate testing for Badger before Jepsen, where we had identified and fixed a few issues related to crashes. You can read that blog post here: https://blog.dgraph.io/post/alice/ https://blog.dgraph.io/post/alice/
- aphyr 8y ago
  Yeah, we did talk about Badger a bit, but I think Dgraph's internal testing was likely going to stress it more than I could over the network. We have occasionally found storage-engine level bugs with Jepsen (Tendermint, for instance), but usually the low request rates mask race conditions in storage libraries. :)
billrobertson42 8y ago
I assume more testing will occur after the four bugs are addressed. Will that be published on the Jepsen blog?
- steven1982 8y ago
  No man, the bugs are open from long time. The team is no more, don't know why. Only the founder is doing commits from months. I had so much hopes on dgraph, it was way better than other graph dbs but immature
  - billrobertson42 8y ago
    Could that be the nature of Github? Doesn't it give credit to whomever merges pull requests?
    - steven1982 8y ago
      GitHub differentiates author and the one who committed if they are not same.
      - billrobertson42 8y ago
        Well, that's unfortunate. DGraph seemed like it was pretty cool. I wish them the best of luck.
- mrjn 8y ago
  Yeah, we're working on fixing the existing bugs and working with Kit Patella (https://github.com/mkcp https://github.com/mkcp) to further extend Jepsen's Dgraph tests. The Snapshot Isolation bank test bug, in particular, is proving a lot harder to reproduce now -- it takes many hours (even up to 8h) to show a violation; which makes debugging harder obviously. The rest 2-3 are sort of related and easier to fix. I doubt Jepsen blog post would be updated, but once all the remaining bugs are fixed, we'll write a blog post about it on https://blog.dgraph.io https://blog.dgraph.io.
truth_seeker 8y ago
Are these fixes also applicable for OpenCypher engine in Dgraph ?
- mrjn 8y ago
  Dgraph currently does not support OpenCypher.

by mrjn 8y ago

jimktrains2 8y ago

> Dgraph has made significant progress, but 4 of the 23 issues we identified remain unresolved, including the corruption of data in healthy clusters. This work was funded by Dgraph, Between this and your comment above, it restores a little faith in humanity that there are people out there who honestly want to make their work as good as possible and not paper over cracks.

manigandham 8y ago
If you're talking about funding, many of the recent analyses were funded by the company whose product is being tested. Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen.
- jimktrains2 8y ago
  > Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen. Yes, but having it published so that everyone sees the actual results without sugar coating or outright ignoring problems is what I meant. I didn't mean that the testing itself was unique, but people funding research and letting the researcher publish it is. (OK, "unique" is much too strong as that's what much of Jepsen does, but I feel like whenever I'm watching the news and there's a public health study talking about how good or not bad X is, it was often funded by an X industry group and reading more about it find it subject to a lot of issues.)
mrjn 8y ago
Thank you! This is the best acknowledgment of our work I've seen so far. A lot of blood, sweat, hairs, frustrations, and tears has gone into building Dgraph -- and this comment by you makes me feel proud of what we're building, and the community of people using it.
- beagledude 8y ago
  I'll echo that as well. You all are doing it right!

beagledude 8y ago

Were there any badger related issues? There doesn't seem to be reading the article.

mrjn 8y ago
None were found based on Jepsen testing done. We had done a separate testing for Badger before Jepsen, where we had identified and fixed a few issues related to crashes. You can read that blog post here: https://blog.dgraph.io/post/alice/ https://blog.dgraph.io/post/alice/
aphyr 8y ago
Yeah, we did talk about Badger a bit, but I think Dgraph's internal testing was likely going to stress it more than I could over the network. We have occasionally found storage-engine level bugs with Jepsen (Tendermint, for instance), but usually the low request rates mask race conditions in storage libraries. :)

billrobertson42 8y ago

I assume more testing will occur after the four bugs are addressed. Will that be published on the Jepsen blog?

steven1982 8y ago
No man, the bugs are open from long time. The team is no more, don't know why. Only the founder is doing commits from months. I had so much hopes on dgraph, it was way better than other graph dbs but immature
- billrobertson42 8y ago
  Could that be the nature of Github? Doesn't it give credit to whomever merges pull requests?
  - steven1982 8y ago
    GitHub differentiates author and the one who committed if they are not same.
    - billrobertson42 8y ago
      Well, that's unfortunate. DGraph seemed like it was pretty cool. I wish them the best of luck.
mrjn 8y ago
Yeah, we're working on fixing the existing bugs and working with Kit Patella (https://github.com/mkcp https://github.com/mkcp) to further extend Jepsen's Dgraph tests. The Snapshot Isolation bank test bug, in particular, is proving a lot harder to reproduce now -- it takes many hours (even up to 8h) to show a violation; which makes debugging harder obviously. The rest 2-3 are sort of related and easier to fix. I doubt Jepsen blog post would be updated, but once all the remaining bugs are fixed, we'll write a blog post about it on https://blog.dgraph.io https://blog.dgraph.io.

truth_seeker 8y ago

Are these fixes also applicable for OpenCypher engine in Dgraph ?

mrjn 8y ago
Dgraph currently does not support OpenCypher.