4 ms·

Numbers look great, but so do /dev/null's. What guarantees does it make? Has it been through Jepsen yet?

by mappu 11y ago

acconsta 11y ago
Honestly, Cassandra's Jepsen didn't set a high bar: https://aphyr.com/posts/294-call-me-maybe-cassandra/ https://aphyr.com/posts/294-call-me-maybe-cassandra/
- cbsmith 11y ago
  Except that problem has been largely addressed now.
  - saurik 11y ago
    I really really really want to see Aphyr attack the patched version to see if he thinks the fix actually worked.
    - gegtik 11y ago
      Datastax is presenting on the topic at their Summit on thursday http://cassandrasummit-datastax.com/agenda/testing-cassandra-guarantees-under-diverse-failure-modes-with-jepsen/ http://cassandrasummit-datastax.com/agenda/testing-cassandra...
  - acconsta 11y ago
    Right, I should add that it was two years ago. My point is that the age of a project has nothing to do with the correctness of its Paxos implementation.
    - cbsmith 11y ago
      Jepsen's finding wasn't that there was a bug in Paxos. It was in how it handled conflicts.
      - acconsta 11y ago
        "So you confer with DataStax for a while, and they manage to reproduce and fix the bug: #6029 (Lightweight transactions race render primary key useless), and #5985 (Paxos replay of in progress update is incorrect). You start building patched versions of Cassandra." "Cassandra lightweight transactions are not even close to correct. Depending on throughput, they may drop anywhere from 1-5% of acknowledged writes–and this doesn’t even require a network partition to demonstrate. It’s just a broken implementation of Paxos. In addition to the deadlock bug, these Jepsen tests revealed #6012 (Cassandra may accept multiple proposals for a single Paxos round) and #6013 (unnecessarily high false negative probabilities)." That's four bugs independent of the conflict resolution issue.
        cbsmith 11y ago
        Oh LWT's are a mess, indeed. Thankfully, you don't normally need them.
  - sargun 11y ago
    How so? The fundamental flaw was using timestamps.
dorlaor 11y ago
It's planned. However, I don't believe we'll pass it today. We're targeting GA for Jan and we'll give it our best shot.
- acconsta 11y ago
  Yeah, I was wondering about that. It looks like you guys have done some brilliant work with the storage engine, but reimplementing all the distributed logic is another (possibly bigger) project.

Numbers look great, but so do /dev/null's. What guarantees does it make? Has it been through Jepsen yet?

by mappu 11y ago

Numbers look great, but so do /dev/null's. What guarantees does it make? Has it been through Jepsen yet?

Honestly, Cassandra's Jepsen didn't set a high bar: https://aphyr.com/posts/294-call-me-maybe-cassandra/ https://aphyr.com/posts/294-call-me-maybe-cassandra/

cbsmith 11y ago
Except that problem has been largely addressed now.
- saurik 11y ago
  I really really really want to see Aphyr attack the patched version to see if he thinks the fix actually worked.
  - gegtik 11y ago
    Datastax is presenting on the topic at their Summit on thursday http://cassandrasummit-datastax.com/agenda/testing-cassandra-guarantees-under-diverse-failure-modes-with-jepsen/ http://cassandrasummit-datastax.com/agenda/testing-cassandra...
- acconsta 11y ago
  Right, I should add that it was two years ago. My point is that the age of a project has nothing to do with the correctness of its Paxos implementation.
  - cbsmith 11y ago
    Jepsen's finding wasn't that there was a bug in Paxos. It was in how it handled conflicts.
    - acconsta 11y ago
      "So you confer with DataStax for a while, and they manage to reproduce and fix the bug: #6029 (Lightweight transactions race render primary key useless), and #5985 (Paxos replay of in progress update is incorrect). You start building patched versions of Cassandra." "Cassandra lightweight transactions are not even close to correct. Depending on throughput, they may drop anywhere from 1-5% of acknowledged writes–and this doesn’t even require a network partition to demonstrate. It’s just a broken implementation of Paxos. In addition to the deadlock bug, these Jepsen tests revealed #6012 (Cassandra may accept multiple proposals for a single Paxos round) and #6013 (unnecessarily high false negative probabilities)." That's four bugs independent of the conflict resolution issue.
      - cbsmith 11y ago
        Oh LWT's are a mess, indeed. Thankfully, you don't normally need them.
- sargun 11y ago
  How so? The fundamental flaw was using timestamps.

dorlaor 11y ago

It's planned. However, I don't believe we'll pass it today. We're targeting GA for Jan and we'll give it our best shot.

acconsta 11y ago
Yeah, I was wondering about that. It looks like you guys have done some brilliant work with the storage engine, but reimplementing all the distributed logic is another (possibly bigger) project.