3 ms·

The inversion of authority in distributed & decentralized systems, from data-centric to agent-centric.

by pjkundert 2y ago

caprock 2y ago
Very interesting. Could you provide an example of this being applied?
- pjkundert 2y ago
  Traditionally, distributed systems must maintain a globally agreed upon state, via locking, two-phase commit, etc. Of course, Byzantine faults caused problems. Bitcoin's PoW provided reliable Byzantine Fault Tolerant consensus in un-permissioned distributed systems. It was a breakthrough, but provided only statistical finality. Hashgraph improved on that to provide asynchronous BFT (aBFT) and deterministic finality and blocking on detection of lack of quorum, but: global consensus still necessarily fails either Availability or Consistency on Partitioning. By imitating physical distributed, decentralized systems where each agent only demands consensus with those agent(s) directly interacting with it, each "Agent" only maintains (and cryptographically proves authority for through PKI signing) its own state, and publishes that state to a shared best-effort DHT (where each party hosting an entry validates the entry by running the purported shared application "entry validation" code, rejecting the entry and "warranting" the offending node on detection of validation failure). Just like an immune system response. Any traditional (a)BFT consensus algorithm can be implemented at whatever scale is appropriate for consensus, but usually only tiny portions of the global state (between a few parties) needs consensus, dramatically reducing the total complexity. Usually, simplistic algorithms ("everybody sign this entry and append it to their state. OK, everyone done?") are sufficient, and trivial to implement. But Hashgraph aBFT may be implemented, if you need it. But, overall, the global system proceeds unimpeded by even large-scale Partition failure. You can glue the system together with "SD cards taped to pigeons", if necessary -- it just slightly increases your transaction times (from milliseconds to hours), but the shared system maintains internal consistency at all times.

The inversion of authority in distributed & decentralized systems, from data-centric to agent-centric.

caprock 2y ago

Very interesting. Could you provide an example of this being applied?

pjkundert 2y ago
Traditionally, distributed systems must maintain a globally agreed upon state, via locking, two-phase commit, etc. Of course, Byzantine faults caused problems. Bitcoin's PoW provided reliable Byzantine Fault Tolerant consensus in un-permissioned distributed systems. It was a breakthrough, but provided only statistical finality. Hashgraph improved on that to provide asynchronous BFT (aBFT) and deterministic finality and blocking on detection of lack of quorum, but: global consensus still necessarily fails either Availability or Consistency on Partitioning. By imitating physical distributed, decentralized systems where each agent only demands consensus with those agent(s) directly interacting with it, each "Agent" only maintains (and cryptographically proves authority for through PKI signing) its own state, and publishes that state to a shared best-effort DHT (where each party hosting an entry validates the entry by running the purported shared application "entry validation" code, rejecting the entry and "warranting" the offending node on detection of validation failure). Just like an immune system response. Any traditional (a)BFT consensus algorithm can be implemented at whatever scale is appropriate for consensus, but usually only tiny portions of the global state (between a few parties) needs consensus, dramatically reducing the total complexity. Usually, simplistic algorithms ("everybody sign this entry and append it to their state. OK, everyone done?") are sufficient, and trivial to implement. But Hashgraph aBFT may be implemented, if you need it. But, overall, the global system proceeds unimpeded by even large-scale Partition failure. You can glue the system together with "SD cards taped to pigeons", if necessary -- it just slightly increases your transaction times (from milliseconds to hours), but the shared system maintains internal consistency at all times.