4 ms·

For this to work as easily as portrayed in the article it would imply that distributed consensus must be possible in the presence of failures. But such consens

For this to work as easily as portrayed in the article it would imply that distributed consensus must be possible in the presence of failures. But such consensus is unsolvable [0] so it seems that this approach is just moving the problem to another place. It's difficult from reading this article to understand exactly what that place is. I would guess that it must at least in part involve limitation on the behavior of applications to eliminate category 1 application generated aborts. If so what do those applications look like? [0] https://en.wikipedia.org/wiki/Consensus_(computer_science) https://en.wikipedia.org/wiki/Consensus_(computer_science)

orangepenguin 8y ago
Right. Talking about "Calvin", he says: > Nonetheless, it was able to bring the processing of that transaction to completion without having to abort it. It accomplished this via restarting the transaction from the same original input. I may be misunderstanding, but if you have transactions recorded in such a way that any worker can replay from any point in time and reach the correct database state, then the transaction log must absorb all of the complexity. The problem has moved, but not changed.
- freels 8y ago
  The big difference is that it's a much easier problem to solve in the context of a log (see Paxos, Raft, ZAB, et al), where the consensus membership is more or less fixed, as opposed to having to manage it among an adhoc membership set.
abadid 8y ago
No. Each process is independently checking the value of a non-changing snapshot of the state of the input data and we therefore do not run into the impossibility result.
- orangepenguin 8y ago
  So then whatever process maintains the snapshots of input data must absorb all of the complexity eliminated from the workers, no?
  - abadid 8y ago
    Multi-versioned systems can read from a snapshot at negligible complexity (just read the correct version).
    - orangepenguin 8y ago
      And is the snapshot system also distributed and highly available? If so, wouldn't it take on the challenges of committing changes?
      - abadid 8y ago
        There is no separate snapshot system. Each partition just maintains multiple versions of all data it stores locally.
    - acqq 8y ago
      Do I understand then correctly that the: temp = Do_Remote_Read(Y) if (temp > 0) X = 42 Is actually temp = Do_Remote_Read(Y_i) if (temp > 0) X = 42 where Y_i is the value of Y in the version i that matches the version for which the coordinator created the code? Otherwise Y can be something else? And isn't then there a need for a synchronization with the keeper of Y, if Y_i is still not available?
      - abadid 8y ago
        Yes, but it is possible to architect the system such that Y_i is available.
        acqq 8y ago
        > it is possible to architect the system such that Y_i is available. I'd really like to know how? How can we be sure that the keeper of Y is ready (or even online) at the Read moment? (As you see I'm not familiar with the systems you worked on but I'd really like to understand the context in which you present the solution).

orangepenguin 8y ago

Right. Talking about "Calvin", he says: > Nonetheless, it was able to bring the processing of that transaction to completion without having to abort it. It accomplished this via restarting the transaction from the same original input. I may be misunderstanding, but if you have transactions recorded in such a way that any worker can replay from any point in time and reach the correct database state, then the transaction log must absorb all of the complexity. The problem has moved, but not changed.

freels 8y ago
The big difference is that it's a much easier problem to solve in the context of a log (see Paxos, Raft, ZAB, et al), where the consensus membership is more or less fixed, as opposed to having to manage it among an adhoc membership set.

abadid 8y ago

No. Each process is independently checking the value of a non-changing snapshot of the state of the input data and we therefore do not run into the impossibility result.

orangepenguin 8y ago
So then whatever process maintains the snapshots of input data must absorb all of the complexity eliminated from the workers, no?
- abadid 8y ago
  Multi-versioned systems can read from a snapshot at negligible complexity (just read the correct version).
  - orangepenguin 8y ago
    And is the snapshot system also distributed and highly available? If so, wouldn't it take on the challenges of committing changes?
    - abadid 8y ago
      There is no separate snapshot system. Each partition just maintains multiple versions of all data it stores locally.
  - acqq 8y ago
    Do I understand then correctly that the: temp = Do_Remote_Read(Y) if (temp > 0) X = 42 Is actually temp = Do_Remote_Read(Y_i) if (temp > 0) X = 42 where Y_i is the value of Y in the version i that matches the version for which the coordinator created the code? Otherwise Y can be something else? And isn't then there a need for a synchronization with the keeper of Y, if Y_i is still not available?
    - abadid 8y ago
      Yes, but it is possible to architect the system such that Y_i is available.
      - acqq 8y ago
        > it is possible to architect the system such that Y_i is available. I'd really like to know how? How can we be sure that the keeper of Y is ready (or even online) at the Read moment? (As you see I'm not familiar with the systems you worked on but I'd really like to understand the context in which you present the solution).