4 ms·

has anyone ever done the modeling on the dynamic limits of decentralized network consensus? my intuition has always been that getting, say, 100,000 globally d

has anyone ever done the modeling on the dynamic limits of decentralized network consensus? my intuition has always been that getting, say, 100,000 globally distributed voting nodes to agree on even a simple truth value will quickly run into exponential (or worse) latency bottlenecks by way of metcalfe's law. the speed of light, as a hard limit, starts to become insurmountable when your consensus pathway has to be traversed the equivalent of tens of millions of km of signal path in order for every node to have an equal influence on the validity of every other node's vote. these high tps coins all seem to get around this by weakening decentralization, whether by shortening this path, either because they have far fewer nodes, or are deciding to sum over some approximation of node consensus by sampling or creating privileged paths, or because geography already creates privileged paths (all the nodes are in the same general area, or even in the same datacenter). I suspect the reason TX speed tends to stay mired in the tens per second region is that no amount of consensus accounting gimmickry will overcome the underlying physical limitations of true global scale decentralization of consensus.

Taek 5y ago
Bitcoin's Proof of Work algorithm has linear scaling. Basically 100,000 nodes all trying to build consensus using Nakamoto PoW can do it in O(100,000) messages. The scaling factor on the number of miners is not what is a bottleneck here. The main bottleneck for cryptocurrencies is that every single node has to validate every single transaction. So your global throughput is effectively limited to what a single node can process. Technologies like STARKs can improve these bottlenecks without introducing new trust layers or trust assumptions but still carry data availability requirements which once again require every node to have all the data, even if they don't have to actually process all the data. There are additional techniques you can use to minimize the data availability impact but then you start introducing trust assumptions again. It's a tricky problem and there's a good reason none of the major chains have adopted a solution, but it's not as dire as your post suggests, nor does your post highlight any of the fundamental issues at play.
- ChainOfFools 5y ago
  > The main bottleneck for cryptocurrencies is that every single node has to validate every single transaction. So your global throughput is effectively limited to what a single node can process literally what I said in my post. the mining aspect was your addition, not mine. I am describing an idealized model which doesn't even consider the added complication of mining to incentivise playing by the rules(let's assume coins are issued based on signs from, say, zeus). just simple consensus on a single 0/1 truth value. this is the hard scalability problem, and people claiming to fix it are ultimately bound to create hierarchies of truth authority, undermining decentralization and recreating the existing concentric power structure (with themselves in the center) they claim to obsolete.
  - randomopining 5y ago
    Yeah I was thinking the same thing. I know nothing compared to many of the posters on this thread. But it seems like decentralization and gas fees/tps are going to be inversely correlated no?
- buckie 5y ago
  There is an example of directly scaling PoW that's been in prod for 1.5 years now: https://explorer.chainweb.com/mainnet https://explorer.chainweb.com/mainnet Instead of having X PH of PoW difficulty for a single blockchain, you have N parallel blockchains with ~X/N PH difficulty. Same energy needs, N*single chain performance. The coolest bit being that N can increase in an upgrade w/o needing to increase energy consumption, should the network gain traction and need more throughput. It upgraded from 10 to 20 chains last summer. Other cool bit is that each chain runs a formally verifiable LISP (Pact)... I jokingly refer to Kadena as "a multithreaded LISP machine in the sky". Disclaimer: I designed the Chainweb consensus algorithm
- tromp 5y ago
  What are the data availability requirements in a chain whose header commits to UTXO set Merkle Mountain Range peaks, where tx inputs come with Merkle proofs up to an MMR peak, and the miner adds a STARK proof of existence of a valid history?
billytetrud 5y ago
> has anyone ever done the modeling on the dynamic limits of decentralized network consensus? I did in the paper this post is about (in the context of bitcoin). Look at the section on Miner Centralization Pressure: https://github.com/fresheneesz/bitcoinThroughputAnalysis#mining-centralization-pressure https://github.com/fresheneesz/bitcoinThroughputAnalysis#min... > these high tps coins all seem to get around this by weakening decentralization Yes, that's true. > in order for every node to have an equal influence on the validity of every other node's vote The trick is to ensure that distance does not play a substantial role in a node's "vote". That's one reason that blocks must take so much time to be mined in bitcoin (10 minutes). Proof of work systems that have less-than-a-minute block times are just not very safe. It takes time for lots of entities to come to a consensus across the whole world. > I suspect the reason TX speed tends to stay mired in the tens per second region is that no amount of consensus accounting gimmickry will overcome the underlying physical limitations of true global scale decentralization of consensus. Yeah, there are physical limits. The speed of light is the ultimate limit here. You can't come to a consensus with someone faster than you can send and receive a response from them. And the minimum requirements for consensus probably require a lot more than one message and one response.

Taek 5y ago

Bitcoin's Proof of Work algorithm has linear scaling. Basically 100,000 nodes all trying to build consensus using Nakamoto PoW can do it in O(100,000) messages. The scaling factor on the number of miners is not what is a bottleneck here. The main bottleneck for cryptocurrencies is that every single node has to validate every single transaction. So your global throughput is effectively limited to what a single node can process. Technologies like STARKs can improve these bottlenecks without introducing new trust layers or trust assumptions but still carry data availability requirements which once again require every node to have all the data, even if they don't have to actually process all the data. There are additional techniques you can use to minimize the data availability impact but then you start introducing trust assumptions again. It's a tricky problem and there's a good reason none of the major chains have adopted a solution, but it's not as dire as your post suggests, nor does your post highlight any of the fundamental issues at play.

ChainOfFools 5y ago
> The main bottleneck for cryptocurrencies is that every single node has to validate every single transaction. So your global throughput is effectively limited to what a single node can process literally what I said in my post. the mining aspect was your addition, not mine. I am describing an idealized model which doesn't even consider the added complication of mining to incentivise playing by the rules(let's assume coins are issued based on signs from, say, zeus). just simple consensus on a single 0/1 truth value. this is the hard scalability problem, and people claiming to fix it are ultimately bound to create hierarchies of truth authority, undermining decentralization and recreating the existing concentric power structure (with themselves in the center) they claim to obsolete.
- randomopining 5y ago
  Yeah I was thinking the same thing. I know nothing compared to many of the posters on this thread. But it seems like decentralization and gas fees/tps are going to be inversely correlated no?
buckie 5y ago
There is an example of directly scaling PoW that's been in prod for 1.5 years now: https://explorer.chainweb.com/mainnet https://explorer.chainweb.com/mainnet Instead of having X PH of PoW difficulty for a single blockchain, you have N parallel blockchains with ~X/N PH difficulty. Same energy needs, N*single chain performance. The coolest bit being that N can increase in an upgrade w/o needing to increase energy consumption, should the network gain traction and need more throughput. It upgraded from 10 to 20 chains last summer. Other cool bit is that each chain runs a formally verifiable LISP (Pact)... I jokingly refer to Kadena as "a multithreaded LISP machine in the sky". Disclaimer: I designed the Chainweb consensus algorithm
tromp 5y ago
What are the data availability requirements in a chain whose header commits to UTXO set Merkle Mountain Range peaks, where tx inputs come with Merkle proofs up to an MMR peak, and the miner adds a STARK proof of existence of a valid history?

billytetrud 5y ago

> has anyone ever done the modeling on the dynamic limits of decentralized network consensus? I did in the paper this post is about (in the context of bitcoin). Look at the section on Miner Centralization Pressure: https://github.com/fresheneesz/bitcoinThroughputAnalysis#mining-centralization-pressure https://github.com/fresheneesz/bitcoinThroughputAnalysis#min... > these high tps coins all seem to get around this by weakening decentralization Yes, that's true. > in order for every node to have an equal influence on the validity of every other node's vote The trick is to ensure that distance does not play a substantial role in a node's "vote". That's one reason that blocks must take so much time to be mined in bitcoin (10 minutes). Proof of work systems that have less-than-a-minute block times are just not very safe. It takes time for lots of entities to come to a consensus across the whole world. > I suspect the reason TX speed tends to stay mired in the tens per second region is that no amount of consensus accounting gimmickry will overcome the underlying physical limitations of true global scale decentralization of consensus. Yeah, there are physical limits. The speed of light is the ultimate limit here. You can't come to a consensus with someone faster than you can send and receive a response from them. And the minimum requirements for consensus probably require a lot more than one message and one response.