I'm very excited by this work. As they say, testing for distributed systems leaves a great deal to be desired, but there work is a great step! I was excited fo

I'm very excited by this work. As they say, testing for distributed systems leaves a great deal to be desired, but there work is a great step! I was excited for FoundationDB, who put a great deal of focus on testing[1], but they were acquired into mystery by Apple. To give you an idea of how well tested they were, Aphyr of "Call Me Maybe" fame didn't test them "because their testing appears to be waaaay more rigorous than mine"[2]. FoundationDB ran Jepsen internally anyway[3]. DEMi is a really nice tool that has me excited. I've worked with similar test systems for non-blocking IO models and they're hugely useful. For non-blocking IO, you can artificially accelerate time as you can leap to the next interrupt, speeding up fuzzing substantially (see: Flow from FoundationDB in [1]). @ikneaddough, the demi-applications repo notes Spark, have you tested it so far? I quite like Spark but had some very temperamental issues at scale. [1]: https://www.youtube.com/watch?v=4fFDFbi3toc https://www.youtube.com/watch?v=4fFDFbi3toc [2]: https://twitter.com/aphyr/status/405017101804396546 https://twitter.com/aphyr/status/405017101804396546 [3]: https://web.archive.org/web/20150325003511/http://blog.foundationdb.com/call-me-maybe-foundationdb-vs-jepsen https://web.archive.org/web/20150325003511/http://blog.found...