5 ms·

I have seen description like this so many times, but no guide seems to describe exactly how everything is done in decentralised way. "The validators for each s

I have seen description like this so many times, but no guide seems to describe exactly how everything is done in decentralised way. "The validators for each slot are assigned to committees". By whom? Also what if there is network partition for these 12 seconds. It's not very unlikely that some country could be completely cutoff from the internet for some time due to anything from natural or political reasons. Could there be recovery after this? At least in PoW, it is clear on what could be the possible scenarios.

RL_Quine 4y ago
Generally we've described this sort of research as "review resistant", rather than being secure in some classical sense. How Ethereum and other cryptocurrencies actually work is often eagerly explained but completely unfathomable on closer inspection. Concerns raised are fixed with additional complexity repeatedly until the reviewer grows weary of attempting to piece it together. Consequently there's sufficient conflicting descriptions that I don't think anybody has any hope of successfully understanding the system as a whole, much less being able to come to conclusions about its security.
- whatisweb3 4y ago
  The spec is open and documented pretty clearly.[1] [1] https://notes.ethereum.org/@vbuterin/Sys3GLJbD#Misc https://notes.ethereum.org/@vbuterin/Sys3GLJbD#Misc
  - capableweb 4y ago
    Considering that Eth2 also have about 7 clients that work today (with more being worked on), the specification is clearly understandable by at least 7 different entities, further eroding the argument that Eth2 specification is so complicated no one actually understands it.
    - ben-schaaf 4y ago
      Most of the linked spec is pseudocode - actually it looks like valid python code. So to be fair I highly doubt you'd need any understanding of what it's actually doing to successfully implement that spec. Just like I still have no clue how Quaternions work, yet I've implemented them multiple times.
      - freemint 4y ago
        How much would one need to pay you to test your hypothesis? That is you try to implement a standard complaint new Eth2 implementation from scratch ( and possible record yourself while doing so.
  - YetAnotherNick 4y ago
    That's just a random function description. The main thing here is how is seed calculated and analysis of whether it seed is safe from network partition, DDoS, malicious attack, nodes being offline, forks etc.
    - whatisweb3 4y ago
      Also clearly documented and open to researchers to examine. The full code for the various Eth2 clients is open source. [1] https://eth2book.info/altair/part2/building_blocks/randomness https://eth2book.info/altair/part2/building_blocks/randomnes... [2] https://eth2.incessant.ink/book/06__building-blocks/02__randomness.html https://eth2.incessant.ink/book/06__building-blocks/02__rand...
- capableweb 4y ago
  That's a gross misrepresentation of how it works in the real-world. Do you believe the same about cryptography? Since you can't understand it, and few really understand the full system, algorithms and functions, it must be "review resistant" rather than actually safe to use?
  - josh2600 4y ago
    Cryptography has mathematical proofs of the underlying math that are rigorous. There are constraints such as breaking if a proof of an attack against the discrete log were to emerge or other constraints related to p=np. The implementations are also very close to formally verified if not fully formally verified. Nothing is perfect but cryptographic code has to be pretty bulletproof or a lot of systems would get owned. The descriptions of how to verify cryptographic systems are academically and professionally rigorous especially compared to eth. Caveat: sometimes the underlying systems that cryptographic implementations utilize change degrading their guarantees. There have been times where the compiler will get updated and cause what was a branchless process to branch which could lead to information leaks against a dedicated attacker. Thankfully such examples are rare in the literature.
    - capableweb 4y ago
      > The implementations are also very close to formally verified if not fully formally verified. So is Eth2, see "Formal Verification of the Ethereum 2.0 Beacon Chain" by Franck Cassez, Joanne Fuller, Aditya Asgaonkar, paper (https://arxiv.org/abs/2110.12909 https://arxiv.org/abs/2110.12909) and source code (https://github.com/ConsenSys/eth2.0-dafny https://github.com/ConsenSys/eth2.0-dafny). More efforts to formally verify Eth2 is ongoing as well, by different entities. > Nothing is perfect but cryptographic code has to be pretty bulletproof or a lot of systems would get owned Same with Ethereum. The chance of having a major impact with a vulnerability is even higher I'd argue, as you can easily extract currency you can trade for USD, and the entire network is inter-connected, so finding targets to exploit becomes even easier. Point still stands that cryptography goes over a lot of peoples head, but you don't hear those people complaining that because they don't understand it, no one does.
- bowsamic 4y ago
  "I don't get it so it must be false" that's what you sound like right now
- douglaswlance 4y ago
  All popular Ethereum clients are open-source. If you really want to know how it works, go read the code.
- pa7x1 4y ago
  How can this argument be made in good faith? The spec is open source, all the clients are open source. Furthermore, it's implemented and running. You could argue that someone in particular has made a poor or incomplete explanation of how it works but not that they way it works is unfathomable or being patched as your speak. It's already implemented, go look at the code if you don't want to believe what anyone says.
  - jayd16 4y ago
    If the spec is open source, how can you say all the clients are open source? Or do you simply mean there are open source clients one can inspect and use?
    - pa7x1 4y ago
      There is a specification of the protocol, which is open source: https://github.com/ethereum/consensus-specs https://github.com/ethereum/consensus-specs And there are over 5 interoperable, independent and open source implementations of said specification: - https://github.com/sigp/lighthouse/ https://github.com/sigp/lighthouse/ - https://github.com/ChainSafe/lodestar https://github.com/ChainSafe/lodestar - https://github.com/status-im/nimbus-eth2 https://github.com/status-im/nimbus-eth2 - https://github.com/prysmaticlabs/prysm https://github.com/prysmaticlabs/prysm - https://github.com/ConsenSys/teku https://github.com/ConsenSys/teku - https://github.com/sifraitech/grandine https://github.com/sifraitech/grandine ...
      - deleted 4y ago
        [deleted]
- Humdeee 4y ago
  But isn't that a great description of our current banking system as it is today? Not everyone knows everything, somehow it works, complex, etc. However, still so woefully insecure.
- aaaaaaaaata 4y ago
  Spoken like somebody who doesn't read Buterin.
3np 4y ago
As for the source of randomness, they mention it briefly in the appendix: > RANDAO: the random number generation scheme used to select block proposers, sort validators into committees, etc Spec and implementation: https://github.com/randao/randao https://github.com/randao/randao
rgrmrts 4y ago
Heres my understanding of it: committee assignment is random, as in baked into the protocol, similar to how a validator is selected for a block (i.e. randomly). While only one validator creates the block, others create attestations which I understand as duplicate work done to check the chosen validators produced block. If an entire country goes down, those validators fall out of the active set and assuming some majority is still active they'll produce and verify blocks. Incorrect state is rejected by the majority, and the minority is punished (slashing their stake). Full disclosure, I'm not an expert - just an enthusiast interested in these protocols. Would love for someone more knowledgeable to correct my mistakes. I'm also not arguing for or against anything here.
- bhaak 4y ago
  > Incorrect state is rejected by the majority, and the minority is punished (slashing their stake). That's wrong. Also the state of those validators that are separated from the larger network is not wrong it's just not the same as what the majority considers the state. They will be on a forked chain of the network. But for being on a fork, no validator is slashed. They will only lose out on the rewards they would have gotten on the majority chain. Validators are only slashed if they try to vote for the same slot/blockheight on different forks of the chain.
  - rgrmrts 4y ago
    Ah, thanks. How does the forked chain reconcile with the rest of the network once the networks sync back up?
    - bhaak 4y ago
      I think that depends on how long the validators were cut off from the majority of validators. Ethereum's POS has a concept called finality which means that under certain conditions the blockchain gets checkpoints that can't ever be reverted. Somewhat simplified but the the blocks are finalized if 2/3rd of the validators validated the last 64 blocks. So, normally after about 13 minutes you can be sure if the transactions are set in stone or not. If the cut off validators are reunited after not too long, they won't have been able to finalize any blocks and then they will just throw away the part of the chain they have been building since they were separated from the majority. As far as I understand it that's quite similar to a reorg on a POW blockchain. The other case, if they've been separated for a very long time, the situation is a bit different. If you don't follow your duties as a validator, you get penalized. A little of your staked ETH is burned. Note, that's not the same as slashing (which is much harsher) and if your staked ETH balance falls below 16 ETH, you're kicked off the list of validators. This is so that active validators will be able to reach again a 2/3rd majority after a while and be able to finalize blocks. If this happens to our cut off validators, even if they get access to the other validators data again, they can't reorg the blockchain as their blocks have been finalized. In this case, the admins need to manually reset the data so the validators can rejoin the correct chain. This is a known issue which POW networks don't have. There you can always rely on the chain that has the most work done. But the work is cheap and fast in POS networks so you sometimes need to get external info and verify it by hand.
whatisweb3 4y ago
According to Eth docs: > One validator is randomly selected to be a block proposer in every slot. This validator is responsible for creating a new block and sending it out to other nodes on the network. Also in every slot, a committee of validators is randomly chosen, whose votes are used to determine the validity of the block being proposed. The annotated code for this can be found in [2]. [1] https://github.com/ethereum/annotated-spec/blob/master/phase0/beacon-chain.md#compute_proposer_index https://github.com/ethereum/annotated-spec/blob/master/phase... [2] https://notes.ethereum.org/@vbuterin/Sys3GLJbD#Misc https://notes.ethereum.org/@vbuterin/Sys3GLJbD#Misc
- niemandhier 4y ago
  Maybe I am stupid, but at least to me its not obvious how one would randomly select a member in a decentralized way, assuming some actors are malicious?
  - capableweb 4y ago
    There are tons of different ways of doing so, with their own set of pros/cons. I'm not familiar with exactly how Eth2 does it, but I suppose your curiosity is a general one, not specific to Eth2, so here is how it works for Algorand as an example using VRFs (Verifiable Random Function): > The VRF takes a secret key and a value and produces a pseudorandom output, with a proof that anyone can use to verify the result. The VRF functions similar to a lottery and is used to choose leaders to propose a block and committee members to vote on a block. This VRF output, when executed for an account, is used to sample from a binomial distribution to emulate a call for every Algo in a user’s account. The more Algos in an account, the better chance the account has of winning – it’s as if every Algo in an account gets its own lottery. This method ensures that a user does not gain any advantage by creating multiple accounts. https://www.algorand.com/Algorand%20Protocol.pdf https://www.algorand.com/Algorand%20Protocol.pdf The search term you're looking for is "decentralized leader election", searching for that on Google or Google Scholar gives you bunch of results ranging from random musings to academic papers.
    - 4y ago
ricochet11 4y ago
https://benjaminion.xyz/eth2-annotated-spec/phase0/beacon-chain/#compute_committee https://benjaminion.xyz/eth2-annotated-spec/phase0/beacon-ch...
norswap 4y ago
Would recommend reading Ben Edgington's annotated spec: https://benjaminion.xyz/eth2-annotated-spec/phase0/beacon-chain/ https://benjaminion.xyz/eth2-annotated-spec/phase0/beacon-ch... It's not entirely up to date (none of the annotated specs are) but out of the three annotated specs I'm aware of, it's the one that is most focused on explaining how things work. Vitalik's annotation (linked in another comment) places a particular focus on the reason specific constants where chosen.
0xfoobar 4y ago
Article author here. Great questions, should have explored the randomness beacon more. Ethereum uses [RANDAO](https://github.com/randao/randao https://github.com/randao/randao), which is a distributed commit-reveal scheme where participants in the generation post a hash of their data on the commit portion and then at a later timestamp reveal the data preimage, and get slashed if they do not reveal a correct preimage. Then all participant data is aggregated together. This means if there is at least one honest participant the generation will be random. A supermajority (2/3rds) of validators is required to finalize a block, in case of a 50-50 network partition blocks would stop being finalized and attestation rewards would stop. Non-participating validators would slowly leak stake through the inactivity leak until online validators once again had a supermajority. This is the "self-healing" mechanism that allows both safety and liveness.

RL_Quine 4y ago

Generally we've described this sort of research as "review resistant", rather than being secure in some classical sense. How Ethereum and other cryptocurrencies actually work is often eagerly explained but completely unfathomable on closer inspection. Concerns raised are fixed with additional complexity repeatedly until the reviewer grows weary of attempting to piece it together. Consequently there's sufficient conflicting descriptions that I don't think anybody has any hope of successfully understanding the system as a whole, much less being able to come to conclusions about its security.

whatisweb3 4y ago
The spec is open and documented pretty clearly.[1] [1] https://notes.ethereum.org/@vbuterin/Sys3GLJbD#Misc https://notes.ethereum.org/@vbuterin/Sys3GLJbD#Misc
- capableweb 4y ago
  Considering that Eth2 also have about 7 clients that work today (with more being worked on), the specification is clearly understandable by at least 7 different entities, further eroding the argument that Eth2 specification is so complicated no one actually understands it.
  - ben-schaaf 4y ago
    Most of the linked spec is pseudocode - actually it looks like valid python code. So to be fair I highly doubt you'd need any understanding of what it's actually doing to successfully implement that spec. Just like I still have no clue how Quaternions work, yet I've implemented them multiple times.
    - freemint 4y ago
      How much would one need to pay you to test your hypothesis? That is you try to implement a standard complaint new Eth2 implementation from scratch ( and possible record yourself while doing so.
- YetAnotherNick 4y ago
  That's just a random function description. The main thing here is how is seed calculated and analysis of whether it seed is safe from network partition, DDoS, malicious attack, nodes being offline, forks etc.
  - whatisweb3 4y ago
    Also clearly documented and open to researchers to examine. The full code for the various Eth2 clients is open source. [1] https://eth2book.info/altair/part2/building_blocks/randomness https://eth2book.info/altair/part2/building_blocks/randomnes... [2] https://eth2.incessant.ink/book/06__building-blocks/02__randomness.html https://eth2.incessant.ink/book/06__building-blocks/02__rand...
capableweb 4y ago
That's a gross misrepresentation of how it works in the real-world. Do you believe the same about cryptography? Since you can't understand it, and few really understand the full system, algorithms and functions, it must be "review resistant" rather than actually safe to use?
- josh2600 4y ago
  Cryptography has mathematical proofs of the underlying math that are rigorous. There are constraints such as breaking if a proof of an attack against the discrete log were to emerge or other constraints related to p=np. The implementations are also very close to formally verified if not fully formally verified. Nothing is perfect but cryptographic code has to be pretty bulletproof or a lot of systems would get owned. The descriptions of how to verify cryptographic systems are academically and professionally rigorous especially compared to eth. Caveat: sometimes the underlying systems that cryptographic implementations utilize change degrading their guarantees. There have been times where the compiler will get updated and cause what was a branchless process to branch which could lead to information leaks against a dedicated attacker. Thankfully such examples are rare in the literature.
  - capableweb 4y ago
    > The implementations are also very close to formally verified if not fully formally verified. So is Eth2, see "Formal Verification of the Ethereum 2.0 Beacon Chain" by Franck Cassez, Joanne Fuller, Aditya Asgaonkar, paper (https://arxiv.org/abs/2110.12909 https://arxiv.org/abs/2110.12909) and source code (https://github.com/ConsenSys/eth2.0-dafny https://github.com/ConsenSys/eth2.0-dafny). More efforts to formally verify Eth2 is ongoing as well, by different entities. > Nothing is perfect but cryptographic code has to be pretty bulletproof or a lot of systems would get owned Same with Ethereum. The chance of having a major impact with a vulnerability is even higher I'd argue, as you can easily extract currency you can trade for USD, and the entire network is inter-connected, so finding targets to exploit becomes even easier. Point still stands that cryptography goes over a lot of peoples head, but you don't hear those people complaining that because they don't understand it, no one does.
bowsamic 4y ago
"I don't get it so it must be false" that's what you sound like right now
douglaswlance 4y ago
All popular Ethereum clients are open-source. If you really want to know how it works, go read the code.
pa7x1 4y ago
How can this argument be made in good faith? The spec is open source, all the clients are open source. Furthermore, it's implemented and running. You could argue that someone in particular has made a poor or incomplete explanation of how it works but not that they way it works is unfathomable or being patched as your speak. It's already implemented, go look at the code if you don't want to believe what anyone says.
- jayd16 4y ago
  If the spec is open source, how can you say all the clients are open source? Or do you simply mean there are open source clients one can inspect and use?
  - pa7x1 4y ago
    There is a specification of the protocol, which is open source: https://github.com/ethereum/consensus-specs https://github.com/ethereum/consensus-specs And there are over 5 interoperable, independent and open source implementations of said specification: - https://github.com/sigp/lighthouse/ https://github.com/sigp/lighthouse/ - https://github.com/ChainSafe/lodestar https://github.com/ChainSafe/lodestar - https://github.com/status-im/nimbus-eth2 https://github.com/status-im/nimbus-eth2 - https://github.com/prysmaticlabs/prysm https://github.com/prysmaticlabs/prysm - https://github.com/ConsenSys/teku https://github.com/ConsenSys/teku - https://github.com/sifraitech/grandine https://github.com/sifraitech/grandine ...
    - deleted 4y ago
      [deleted]
Humdeee 4y ago
But isn't that a great description of our current banking system as it is today? Not everyone knows everything, somehow it works, complex, etc. However, still so woefully insecure.
aaaaaaaaata 4y ago
Spoken like somebody who doesn't read Buterin.

3np 4y ago

As for the source of randomness, they mention it briefly in the appendix: > RANDAO: the random number generation scheme used to select block proposers, sort validators into committees, etc Spec and implementation: https://github.com/randao/randao https://github.com/randao/randao

rgrmrts 4y ago

Heres my understanding of it: committee assignment is random, as in baked into the protocol, similar to how a validator is selected for a block (i.e. randomly). While only one validator creates the block, others create attestations which I understand as duplicate work done to check the chosen validators produced block. If an entire country goes down, those validators fall out of the active set and assuming some majority is still active they'll produce and verify blocks. Incorrect state is rejected by the majority, and the minority is punished (slashing their stake). Full disclosure, I'm not an expert - just an enthusiast interested in these protocols. Would love for someone more knowledgeable to correct my mistakes. I'm also not arguing for or against anything here.

bhaak 4y ago
> Incorrect state is rejected by the majority, and the minority is punished (slashing their stake). That's wrong. Also the state of those validators that are separated from the larger network is not wrong it's just not the same as what the majority considers the state. They will be on a forked chain of the network. But for being on a fork, no validator is slashed. They will only lose out on the rewards they would have gotten on the majority chain. Validators are only slashed if they try to vote for the same slot/blockheight on different forks of the chain.
- rgrmrts 4y ago
  Ah, thanks. How does the forked chain reconcile with the rest of the network once the networks sync back up?
  - bhaak 4y ago
    I think that depends on how long the validators were cut off from the majority of validators. Ethereum's POS has a concept called finality which means that under certain conditions the blockchain gets checkpoints that can't ever be reverted. Somewhat simplified but the the blocks are finalized if 2/3rd of the validators validated the last 64 blocks. So, normally after about 13 minutes you can be sure if the transactions are set in stone or not. If the cut off validators are reunited after not too long, they won't have been able to finalize any blocks and then they will just throw away the part of the chain they have been building since they were separated from the majority. As far as I understand it that's quite similar to a reorg on a POW blockchain. The other case, if they've been separated for a very long time, the situation is a bit different. If you don't follow your duties as a validator, you get penalized. A little of your staked ETH is burned. Note, that's not the same as slashing (which is much harsher) and if your staked ETH balance falls below 16 ETH, you're kicked off the list of validators. This is so that active validators will be able to reach again a 2/3rd majority after a while and be able to finalize blocks. If this happens to our cut off validators, even if they get access to the other validators data again, they can't reorg the blockchain as their blocks have been finalized. In this case, the admins need to manually reset the data so the validators can rejoin the correct chain. This is a known issue which POW networks don't have. There you can always rely on the chain that has the most work done. But the work is cheap and fast in POS networks so you sometimes need to get external info and verify it by hand.

whatisweb3 4y ago

According to Eth docs: > One validator is randomly selected to be a block proposer in every slot. This validator is responsible for creating a new block and sending it out to other nodes on the network. Also in every slot, a committee of validators is randomly chosen, whose votes are used to determine the validity of the block being proposed. The annotated code for this can be found in [2]. [1] https://github.com/ethereum/annotated-spec/blob/master/phase0/beacon-chain.md#compute_proposer_index https://github.com/ethereum/annotated-spec/blob/master/phase... [2] https://notes.ethereum.org/@vbuterin/Sys3GLJbD#Misc https://notes.ethereum.org/@vbuterin/Sys3GLJbD#Misc

niemandhier 4y ago
Maybe I am stupid, but at least to me its not obvious how one would randomly select a member in a decentralized way, assuming some actors are malicious?
- capableweb 4y ago
  There are tons of different ways of doing so, with their own set of pros/cons. I'm not familiar with exactly how Eth2 does it, but I suppose your curiosity is a general one, not specific to Eth2, so here is how it works for Algorand as an example using VRFs (Verifiable Random Function): > The VRF takes a secret key and a value and produces a pseudorandom output, with a proof that anyone can use to verify the result. The VRF functions similar to a lottery and is used to choose leaders to propose a block and committee members to vote on a block. This VRF output, when executed for an account, is used to sample from a binomial distribution to emulate a call for every Algo in a user’s account. The more Algos in an account, the better chance the account has of winning – it’s as if every Algo in an account gets its own lottery. This method ensures that a user does not gain any advantage by creating multiple accounts. https://www.algorand.com/Algorand%20Protocol.pdf https://www.algorand.com/Algorand%20Protocol.pdf The search term you're looking for is "decentralized leader election", searching for that on Google or Google Scholar gives you bunch of results ranging from random musings to academic papers.
  - 4y ago

ricochet11 4y ago

https://benjaminion.xyz/eth2-annotated-spec/phase0/beacon-chain/#compute_committee https://benjaminion.xyz/eth2-annotated-spec/phase0/beacon-ch...

norswap 4y ago

Would recommend reading Ben Edgington's annotated spec: https://benjaminion.xyz/eth2-annotated-spec/phase0/beacon-chain/ https://benjaminion.xyz/eth2-annotated-spec/phase0/beacon-ch... It's not entirely up to date (none of the annotated specs are) but out of the three annotated specs I'm aware of, it's the one that is most focused on explaining how things work. Vitalik's annotation (linked in another comment) places a particular focus on the reason specific constants where chosen.

0xfoobar 4y ago

Article author here. Great questions, should have explored the randomness beacon more. Ethereum uses [RANDAO](https://github.com/randao/randao https://github.com/randao/randao), which is a distributed commit-reveal scheme where participants in the generation post a hash of their data on the commit portion and then at a later timestamp reveal the data preimage, and get slashed if they do not reveal a correct preimage. Then all participant data is aggregated together. This means if there is at least one honest participant the generation will be random. A supermajority (2/3rds) of validators is required to finalize a block, in case of a 50-50 network partition blocks would stop being finalized and attestation rewards would stop. Non-participating validators would slowly leak stake through the inactivity leak until online validators once again had a supermajority. This is the "self-healing" mechanism that allows both safety and liveness.