4 ms·

> Transactions are applied independently on each storage host I’m not talking about transaction application, but obtaining your slot in the global transaction

> Transactions are applied independently on each storage host I’m not talking about transaction application, but obtaining your slot in the global transaction log. How does a replica know it isn’t missing a transaction from some other shard if that shard is offline come its turn to declare transactions in the log? It must at least receive a message saying no transactions involving it were declared by that shard, no? This is pretty core to the Calvin approach, unless I misunderstand it. > I'm not at liberty to say. I think scalability is something that is a function of both theoretical expectations and practical demonstration.

bitbckt 4y ago
> How does a replica know it isn’t missing a transaction from some other shard if that shard is offline come its turn to declare transactions in the log? It must at least receive a message saying no transactions involving it were declared by that shard, no? A transaction is committed to the log with sufficient information necessary for the log (and each storage host) to know whether any particular host is involved in that transaction. There's no need for a message to prove a host's absence from that transaction - it has all it needs in the log to determine that it isn't involved. There is some complexity around how that information maps to the cluster's physical topology, but hosts which aren't involved in a transaction don't process that transaction and need not check with any other host to know that fact. [ETA] That information is derived from the read set when constructing the transaction to propose to the log. Logical partitions that weren't a part of that read set don't need to know that they weren't involved - that fact is obvious from their absence. > I think scalability is something that is a function of both theoretical expectations and practical demonstration. I agree with you, though as a commercial product operated as a service, that practical demonstration is naturally limited in this forum.
- _benedict 4y ago
  You misunderstand my point. A replica may of course safely assume it is not involved in a transaction it hasn’t witnessed so long as it has a “complete” log (as in, the portion involving it). > that fact is obvious from their absence. Yes, but absence from what? There must be some message that contains the relevant portion of the log declared by each other shard. If that shard is offline, so it cannot declare its transactions at all, how does the system continue? This absence is one step back from the one you are discussing. If a shard’s leader is offline, a new leader must be elected before its slot comes around for processing - and until this happens all transactions in later slots must wait, as it might have declared a transaction that interferes with those a later slot would declare. If no leader can be elected (because a majority of replicas are offline) then the entire log stops, no?
  - bitbckt 4y ago
    It's true: I've had trouble determining what exactly your point is, as you seem to think you know more about FaunaDB's architecture than you do. You're under the mistaken impression that we have the same data/log architecture as described in the Calvin paper, which requires that every host talk with every other host. That is not true of FaunaDB.
    - _benedict 4y ago
      What you describe is a very important distinction from Calvin, as this mechanism for deriving a global log is core to its design. Have you published this new approach in any public venue? If not, we’re now in an unfortunate situation where neither your practical nor theoretical capabilities are public knowledge. > every host talk with every other host Only every shard > you seem to think you know more about FaunaDB's architecture than you do You are perhaps being uncharitable. The only public statements I am aware of about Fauna's capabilities relate to its Calvin heritage. I have only been asking if my understanding is correct regarding both Calvin and its application to Fauna. However, none of the prior issues you responded to were problems with the original Calvin paper either, at least by my reading.
      - bitbckt 4y ago
        LMGTFY: https://fauna.com/blog/consistency-without-clocks-faunadb-transaction-protocol https://fauna.com/blog/consistency-without-clocks-faunadb-tr...
        _benedict 4y ago
        Thanks > for scalability, this insertion process happens in batches, and the log itself is replicated and partitioned This is the link's total exposition on this topic, but it is consistent with my reading of the Calvin paper. If any log partition becomes unavailable, the log in its entirety becomes unavailable until that partition recovers[1]. Anyway, it feels like we’ve wasted enough of each others’ time without making much progress. Might as well leave it there. [1] ... and every log partition must communicate with every replica, hence NxN (or MxN)

bitbckt 4y ago

> How does a replica know it isn’t missing a transaction from some other shard if that shard is offline come its turn to declare transactions in the log? It must at least receive a message saying no transactions involving it were declared by that shard, no? A transaction is committed to the log with sufficient information necessary for the log (and each storage host) to know whether any particular host is involved in that transaction. There's no need for a message to prove a host's absence from that transaction - it has all it needs in the log to determine that it isn't involved. There is some complexity around how that information maps to the cluster's physical topology, but hosts which aren't involved in a transaction don't process that transaction and need not check with any other host to know that fact. [ETA] That information is derived from the read set when constructing the transaction to propose to the log. Logical partitions that weren't a part of that read set don't need to know that they weren't involved - that fact is obvious from their absence. > I think scalability is something that is a function of both theoretical expectations and practical demonstration. I agree with you, though as a commercial product operated as a service, that practical demonstration is naturally limited in this forum.

_benedict 4y ago
You misunderstand my point. A replica may of course safely assume it is not involved in a transaction it hasn’t witnessed so long as it has a “complete” log (as in, the portion involving it). > that fact is obvious from their absence. Yes, but absence from what? There must be some message that contains the relevant portion of the log declared by each other shard. If that shard is offline, so it cannot declare its transactions at all, how does the system continue? This absence is one step back from the one you are discussing. If a shard’s leader is offline, a new leader must be elected before its slot comes around for processing - and until this happens all transactions in later slots must wait, as it might have declared a transaction that interferes with those a later slot would declare. If no leader can be elected (because a majority of replicas are offline) then the entire log stops, no?
- bitbckt 4y ago
  It's true: I've had trouble determining what exactly your point is, as you seem to think you know more about FaunaDB's architecture than you do. You're under the mistaken impression that we have the same data/log architecture as described in the Calvin paper, which requires that every host talk with every other host. That is not true of FaunaDB.
  - _benedict 4y ago
    What you describe is a very important distinction from Calvin, as this mechanism for deriving a global log is core to its design. Have you published this new approach in any public venue? If not, we’re now in an unfortunate situation where neither your practical nor theoretical capabilities are public knowledge. > every host talk with every other host Only every shard > you seem to think you know more about FaunaDB's architecture than you do You are perhaps being uncharitable. The only public statements I am aware of about Fauna's capabilities relate to its Calvin heritage. I have only been asking if my understanding is correct regarding both Calvin and its application to Fauna. However, none of the prior issues you responded to were problems with the original Calvin paper either, at least by my reading.
    - bitbckt 4y ago
      LMGTFY: https://fauna.com/blog/consistency-without-clocks-faunadb-transaction-protocol https://fauna.com/blog/consistency-without-clocks-faunadb-tr...
      - _benedict 4y ago
        Thanks > for scalability, this insertion process happens in batches, and the log itself is replicated and partitioned This is the link's total exposition on this topic, but it is consistent with my reading of the Calvin paper. If any log partition becomes unavailable, the log in its entirety becomes unavailable until that partition recovers[1]. Anyway, it feels like we’ve wasted enough of each others’ time without making much progress. Might as well leave it there. [1] ... and every log partition must communicate with every replica, hence NxN (or MxN)