7 ms·

Safety, liveness and fault tolerance—the consensus choices

66 points by soroushjp 12y ago

taspeotis 12y ago
I don't know what Stellar is. I had a look at their website. It's a decentralised [1] currency something or other. But this blog post states: > We were able to replay most of these rolled back transactions on chain B to minimize the impact And > To ensure no ledger forks going forward in Stellar, we have decided to temporarily only run one validating node until the new consensus algorithm is live I don't see the decentralisation here? [1] https://www.stellar.org/learn/#Decentralized_network https://www.stellar.org/learn/#Decentralized_network "This means that the Stellar network does not depend on any single entity"
- bhouston 12y ago
  They have de-decentralized so that there are no more ledger splits. Sort of embarrassing but understandable. It is understood that they will re-decentralize once they get a new consensus algorithm I guess.
  - joyce 12y ago
    Joyce from Stellar here. Yes, that is correct. When given the choice between temporary centralization and guaranteeing the security of the protocol and therefore user funds, the choice is obvious. Once the new consensus algorithm is complete, it will be safe to run with more than one node again.
    - taspeotis 12y ago
      How'd you centralize your decentralized system? It sounds like something that shouldn't be possible unless it's actually centralized to begin with. I'm a layperson when it comes to crypto currencies, but my impression is that most people would consider the fact that you can centralize a bug (separate to the hiccup you had in the article).
      - polymathist 12y ago
        I'm just guessing here, and I hope that Joyce or someone from Stellar could correct me if I'm wrong. It could be as simple as announcing that you're going to make this change to de-decentralize, describing what that process entails, and asking all the invested parties (in this case validating nodes) to follow suit. I imagine that the process is similar to when bitcoin has done a hard fork in the past. The bitcoin core developers see the need for a hard fork, announce the hard fork, and ask everyone to update. If the majority of nodes agree than the hard fork was successful. So in Stellar's case, they could be asking people who run validating nodes to update their software, or to simply modify their UNL to point to only a single node (similar to seeding your bitcoin client with only a single peer). I think (and hope) that the Stellar Foundation doesn't have the ability to actually force people to de-decentralize.
        joyce 12y ago
        Howdy - so at the time of the ledger fork, the Foundation was running all 5 of the validating nodes and there were other parties not associated with us that were running non-validating nodes. We do not have the ability nor do we want the ability to control other people's nodes. Since Stellar only launched 4 months ago, the number nodes in the network was still small. In the future, when the network is on the new consensus system and able to run safely in a truly decentralized, then it would be up to individual nodes to decide what to do. Hope that clarifies things.
        themusicgod1 12y ago
        Why wasn't anyone else running a validating node? Where is the code for the validating node?
      - deleted 12y ago
        [deleted]
  - makomk 12y ago
    I don't think they were ever really decentralized except on paper anyway; apparently all five of the nodes making up the Stellar consensus were run by the foundation that created it.
- nly 12y ago
  Decentralisation is overrated. Email is decentralised, but there are still powerful actors (Gmail, Outlook, Yahoo, DNSBLs). In Bitcoin they're the miner pools, which, while currently diverse and seemingly quite good at self-regulation[0], have ultimate say over who gets to transact what. In many ways a centralised, but highly anonymous, digital cash system puts stronger practical limits on the power of a single individual. In such a system the the only way the central authority (bank etc) to doctor history, once money has moved out of their hands, is to drive the entire currency in to the ground and start over. [0] http://www.coindesk.com/ghash-commits-40-hashrate-cap-bitcoin-mining-summit/ http://www.coindesk.com/ghash-commits-40-hashrate-cap-bitcoi...
  - drcode 12y ago
    Decentralization is not overrated, just misunderstood. The primary concern should be whether a central entity can change the agreed-upon communication protocol at will (i.e. arbitrarily change the "rules" of the money) With Bitcoin, such a change is very difficult. With stellar, it is easy for the central Corp to change the rules at it's whim.
  - joosters 12y ago
    What? Without decentralisation, Bitcoin has nothing to offer over everyone trusting some random entity offering to keep a ledger of account balances. And we already have plenty of those around.
ghshephard 12y ago
I'm somewhat familiar with Bitcoin, and I've heard of Stellar, and my sense is they are related - both a decentralized consensus based ledger system. I'm wondering if the problem that Stellar encountered is something that Bitcoin has some resistance to. Anybody have some insight?
- bachback 12y ago
  Stellar is not decentralized. If problems like this occur, there is the Stellar organization /corporation, to enforce consenus. Whenever there is a legal entity behind it, it's not de-central. Bitcoin has solved all these problems. Ripple was invented before Bitcoin. Stellar is a Ripple fork.
  - soroushjp 12y ago
    Stellar is not currently decentralized. The use of a single validating node is a temporary measure while they rewrite their consensus algorithm. This is obviously far from ideal, but the Stellar foundation has been transparent that this is a temporary measure. In the original design, and in their stated mission that they've never wavered from, this will be a open protocol where anyone can join as a validating node. But they needed a stop gap measure while they fixed this ledger fork issue. Again, far from ideal, but let's not jump to harsh conclusions either.
bachback 12y ago
The Ripple / Stellar protocol is bad, so this is not a surprise. Ripple was invented before Bitcoin. Ripple/Stellar requires explicit trust, where Bitcoin does not. Also the distribution was widely known to be gameable (I know many people involved who managed to get free stellar). The only surprising thing is that Stripe and Ravikant invested in a technology which is obviously inferior to Bitcoin. Prof. Mazieré and JedMcCaleb have no significant background in economic research and it shows.
- argonaut 12y ago
  I don't agree that Stellar is an "obviously inferior" technology. Stellar just makes different tradeoffs. I particularly enjoy this quote from http://adamierymenko.com/decentralization-i-want-to-believe/ http://adamierymenko.com/decentralization-i-want-to-believe/: A centralized alternative to Bitcoin would be a simple SQL database with a schema representing standard double-entry accounting and some meta-data fields. The entire transaction volume of the Bitcoin network could be handled by a Raspberry Pi in a shoebox. Instead, we have a transaction volume of a few hundred thousand database entries a day being handled by a compute cluster comparable to those used to simulate atomic bomb blasts with physically realistic voxel models or probabilistically describe a complete relationship graph for the human proteome.
  - drcode 12y ago
    I agree that stellar-like systems might have a role to play in the future, but surely a coin with an incident of corrupted transaction logs is inferior to one without such an incident, all else being equal. EDIT: To those pointing out the 2013 Bitcoin fork, good point. However, in cryptocurrency land, 2013 is ancient history, and I'd argue standards are higher already.
    - moreati 12y ago
      Yes, but note that Bitcoin had a blockchain fork in 2013 http://bitcoinmagazine.com/3668/bitcoin-network-shaken-by-blockchain-fork/ http://bitcoinmagazine.com/3668/bitcoin-network-shaken-by-bl.... Without knowing the details of Stellar's fork I make no comment on the relative severities/merits.
dsr_ 12y ago
" Any distributed consensus system on the Internet must sacrifice one of these features." is incomplete: a distributed consensus system (which is, at heart, a distributed database) can not have all three features... but there is no guarantee that a distributed database has any of those features. As with everything else, execution matters. If you're going to be recording history for financial transactions, you need to put immutability as your first goal. This is not compatible with unlimited space-time separation of trusted inputs, so the second thing you need is to decide how you're going to resolve inconsistent histories. Doing so always involves a centralized trusted system, even if it is fed from a distributed system: someone needs to decide what transactions really happened. You can claim that you have a distributed algorithm to do so (consensus) but that itself will always fall into the same distribution problem. And that's what seems to have happened here.
- drcode 12y ago
  > ...how you're going to resolve inconsistent histories. Doing so always involves a centralized trusted system False, the Bitcoin system resolves inconsistent histories in a completely decentralized manner.
  - wcoenen 12y ago
    One could argue that a block chain fork could invalidate an arbitrary number of blocks at any time, and therefore the Bitcoin protocol does not by itself have this property of being able to tell which transactions really happened. To limit the number of blocks that could be invalidated in this way, the bitcoin reference implementation contains checkpoint hashes. These are indeed decided centrally. If you accept the core assumption in bitcoin that an attacker will never control a majority of the hashing power, then there probably isn't a problem and the checkpoint s aren't needed.
    - carry_bit 12y ago
      Each block chain fork is in essence a difference currency, and so you can tell which transactions happened for a given currency. Which currency is the favored one depends on the client hence the user. The users then just need to make sure that they are selecting the currency that they want to select. The checkpoint hashes are not imposed on the users, but rather the users must choose to upgrade to the new checkpoint hashes. As such, you can argue that it is decentralized.
marco_salvatori 12y ago
"Prof. Mazières’s research indicated some risk that consensus could fail, though we were nor certain if the required circumstances for such a failure were realistic." I'm surprised to see the above statement in a press release; maybe it was not worded quite right. At the scale of 100s of thousands, or a millions of transactions a day, "some risk" will manifest itself on operation timescales itself. So when one is "not certain" it's always best to assume that problems will show up and it will take less time than expected. "We are still investigating the triggers for this consensus failure, but believe it is caused by the innate weaknesses of the Ripple/Stellar consensus system outlined above compounded by the number of accounts in the network." Also not great wording. Saying the system had "innate weaknesses" and we "believe" kind of implies engineers are still guessing on the trigger. Your building a financial corporation and if you lose you loose everything
yc1010 12y ago
Well it was to be expected, Ripple scammed thousands of users with similar promises They re-branded as "Stellar" + somehow got Stripe to give them a mention and the scam repeats
subnatant 12y ago
Scary! I just pulled my money out of ripple.
rfugger 12y ago
Ripple Labs' response: https://ripple.com/why-the-stellar-forking-issue-does-not-affect-ripple/ https://ripple.com/why-the-stellar-forking-issue-does-not-af... (Ripple Labs develops the software that Stellar modifies for their own use, and the original Ripple network still runs on unmodified Ripple Labs software. Stellar was started by Jed McCaleb, who founded Ripple Labs, but broke with the CEO last year.) The Ripple consensus protocol puts certain requirements on the topology of the network of transaction-validating nodes in order to work properly. They are still investigating, but it's possible Stellar's network fell outside the workable range. Ripple Labs manages their network topology more carefully than Stellar, and this incident may validate their approach. We'll have to see what actually happened.

66 points by soroushjp 12y ago

I don't know what Stellar is. I had a look at their website. It's a decentralised [1] currency something or other. But this blog post states: > We were able to replay most of these rolled back transactions on chain B to minimize the impact And > To ensure no ledger forks going forward in Stellar, we have decided to temporarily only run one validating node until the new consensus algorithm is live I don't see the decentralisation here? [1] https://www.stellar.org/learn/#Decentralized_network https://www.stellar.org/learn/#Decentralized_network "This means that the Stellar network does not depend on any single entity"

bhouston 12y ago
They have de-decentralized so that there are no more ledger splits. Sort of embarrassing but understandable. It is understood that they will re-decentralize once they get a new consensus algorithm I guess.
- joyce 12y ago
  Joyce from Stellar here. Yes, that is correct. When given the choice between temporary centralization and guaranteeing the security of the protocol and therefore user funds, the choice is obvious. Once the new consensus algorithm is complete, it will be safe to run with more than one node again.
  - taspeotis 12y ago
    How'd you centralize your decentralized system? It sounds like something that shouldn't be possible unless it's actually centralized to begin with. I'm a layperson when it comes to crypto currencies, but my impression is that most people would consider the fact that you can centralize a bug (separate to the hiccup you had in the article).
    - polymathist 12y ago
      I'm just guessing here, and I hope that Joyce or someone from Stellar could correct me if I'm wrong. It could be as simple as announcing that you're going to make this change to de-decentralize, describing what that process entails, and asking all the invested parties (in this case validating nodes) to follow suit. I imagine that the process is similar to when bitcoin has done a hard fork in the past. The bitcoin core developers see the need for a hard fork, announce the hard fork, and ask everyone to update. If the majority of nodes agree than the hard fork was successful. So in Stellar's case, they could be asking people who run validating nodes to update their software, or to simply modify their UNL to point to only a single node (similar to seeding your bitcoin client with only a single peer). I think (and hope) that the Stellar Foundation doesn't have the ability to actually force people to de-decentralize.
      - joyce 12y ago
        Howdy - so at the time of the ledger fork, the Foundation was running all 5 of the validating nodes and there were other parties not associated with us that were running non-validating nodes. We do not have the ability nor do we want the ability to control other people's nodes. Since Stellar only launched 4 months ago, the number nodes in the network was still small. In the future, when the network is on the new consensus system and able to run safely in a truly decentralized, then it would be up to individual nodes to decide what to do. Hope that clarifies things.
        themusicgod1 12y ago
        Why wasn't anyone else running a validating node? Where is the code for the validating node?
    - deleted 12y ago
      [deleted]
- makomk 12y ago
  I don't think they were ever really decentralized except on paper anyway; apparently all five of the nodes making up the Stellar consensus were run by the foundation that created it.
nly 12y ago
Decentralisation is overrated. Email is decentralised, but there are still powerful actors (Gmail, Outlook, Yahoo, DNSBLs). In Bitcoin they're the miner pools, which, while currently diverse and seemingly quite good at self-regulation[0], have ultimate say over who gets to transact what. In many ways a centralised, but highly anonymous, digital cash system puts stronger practical limits on the power of a single individual. In such a system the the only way the central authority (bank etc) to doctor history, once money has moved out of their hands, is to drive the entire currency in to the ground and start over. [0] http://www.coindesk.com/ghash-commits-40-hashrate-cap-bitcoin-mining-summit/ http://www.coindesk.com/ghash-commits-40-hashrate-cap-bitcoi...
- drcode 12y ago
  Decentralization is not overrated, just misunderstood. The primary concern should be whether a central entity can change the agreed-upon communication protocol at will (i.e. arbitrarily change the "rules" of the money) With Bitcoin, such a change is very difficult. With stellar, it is easy for the central Corp to change the rules at it's whim.
- joosters 12y ago
  What? Without decentralisation, Bitcoin has nothing to offer over everyone trusting some random entity offering to keep a ledger of account balances. And we already have plenty of those around.

ghshephard 12y ago

I'm somewhat familiar with Bitcoin, and I've heard of Stellar, and my sense is they are related - both a decentralized consensus based ledger system. I'm wondering if the problem that Stellar encountered is something that Bitcoin has some resistance to. Anybody have some insight?

bachback 12y ago
Stellar is not decentralized. If problems like this occur, there is the Stellar organization /corporation, to enforce consenus. Whenever there is a legal entity behind it, it's not de-central. Bitcoin has solved all these problems. Ripple was invented before Bitcoin. Stellar is a Ripple fork.
- soroushjp 12y ago
  Stellar is not currently decentralized. The use of a single validating node is a temporary measure while they rewrite their consensus algorithm. This is obviously far from ideal, but the Stellar foundation has been transparent that this is a temporary measure. In the original design, and in their stated mission that they've never wavered from, this will be a open protocol where anyone can join as a validating node. But they needed a stop gap measure while they fixed this ledger fork issue. Again, far from ideal, but let's not jump to harsh conclusions either.

bachback 12y ago

The Ripple / Stellar protocol is bad, so this is not a surprise. Ripple was invented before Bitcoin. Ripple/Stellar requires explicit trust, where Bitcoin does not. Also the distribution was widely known to be gameable (I know many people involved who managed to get free stellar). The only surprising thing is that Stripe and Ravikant invested in a technology which is obviously inferior to Bitcoin. Prof. Mazieré and JedMcCaleb have no significant background in economic research and it shows.

argonaut 12y ago
I don't agree that Stellar is an "obviously inferior" technology. Stellar just makes different tradeoffs. I particularly enjoy this quote from http://adamierymenko.com/decentralization-i-want-to-believe/ http://adamierymenko.com/decentralization-i-want-to-believe/: A centralized alternative to Bitcoin would be a simple SQL database with a schema representing standard double-entry accounting and some meta-data fields. The entire transaction volume of the Bitcoin network could be handled by a Raspberry Pi in a shoebox. Instead, we have a transaction volume of a few hundred thousand database entries a day being handled by a compute cluster comparable to those used to simulate atomic bomb blasts with physically realistic voxel models or probabilistically describe a complete relationship graph for the human proteome.
- drcode 12y ago
  I agree that stellar-like systems might have a role to play in the future, but surely a coin with an incident of corrupted transaction logs is inferior to one without such an incident, all else being equal. EDIT: To those pointing out the 2013 Bitcoin fork, good point. However, in cryptocurrency land, 2013 is ancient history, and I'd argue standards are higher already.
  - moreati 12y ago
    Yes, but note that Bitcoin had a blockchain fork in 2013 http://bitcoinmagazine.com/3668/bitcoin-network-shaken-by-blockchain-fork/ http://bitcoinmagazine.com/3668/bitcoin-network-shaken-by-bl.... Without knowing the details of Stellar's fork I make no comment on the relative severities/merits.

dsr_ 12y ago

" Any distributed consensus system on the Internet must sacrifice one of these features." is incomplete: a distributed consensus system (which is, at heart, a distributed database) can not have all three features... but there is no guarantee that a distributed database has any of those features. As with everything else, execution matters. If you're going to be recording history for financial transactions, you need to put immutability as your first goal. This is not compatible with unlimited space-time separation of trusted inputs, so the second thing you need is to decide how you're going to resolve inconsistent histories. Doing so always involves a centralized trusted system, even if it is fed from a distributed system: someone needs to decide what transactions really happened. You can claim that you have a distributed algorithm to do so (consensus) but that itself will always fall into the same distribution problem. And that's what seems to have happened here.

drcode 12y ago
> ...how you're going to resolve inconsistent histories. Doing so always involves a centralized trusted system False, the Bitcoin system resolves inconsistent histories in a completely decentralized manner.
- wcoenen 12y ago
  One could argue that a block chain fork could invalidate an arbitrary number of blocks at any time, and therefore the Bitcoin protocol does not by itself have this property of being able to tell which transactions really happened. To limit the number of blocks that could be invalidated in this way, the bitcoin reference implementation contains checkpoint hashes. These are indeed decided centrally. If you accept the core assumption in bitcoin that an attacker will never control a majority of the hashing power, then there probably isn't a problem and the checkpoint s aren't needed.
  - carry_bit 12y ago
    Each block chain fork is in essence a difference currency, and so you can tell which transactions happened for a given currency. Which currency is the favored one depends on the client hence the user. The users then just need to make sure that they are selecting the currency that they want to select. The checkpoint hashes are not imposed on the users, but rather the users must choose to upgrade to the new checkpoint hashes. As such, you can argue that it is decentralized.

marco_salvatori 12y ago

"Prof. Mazières’s research indicated some risk that consensus could fail, though we were nor certain if the required circumstances for such a failure were realistic." I'm surprised to see the above statement in a press release; maybe it was not worded quite right. At the scale of 100s of thousands, or a millions of transactions a day, "some risk" will manifest itself on operation timescales itself. So when one is "not certain" it's always best to assume that problems will show up and it will take less time than expected. "We are still investigating the triggers for this consensus failure, but believe it is caused by the innate weaknesses of the Ripple/Stellar consensus system outlined above compounded by the number of accounts in the network." Also not great wording. Saying the system had "innate weaknesses" and we "believe" kind of implies engineers are still guessing on the trigger. Your building a financial corporation and if you lose you loose everything

yc1010 12y ago

Well it was to be expected, Ripple scammed thousands of users with similar promises They re-branded as "Stellar" + somehow got Stripe to give them a mention and the scam repeats

subnatant 12y ago

Scary! I just pulled my money out of ripple.

rfugger 12y ago

Ripple Labs' response: https://ripple.com/why-the-stellar-forking-issue-does-not-affect-ripple/ https://ripple.com/why-the-stellar-forking-issue-does-not-af... (Ripple Labs develops the software that Stellar modifies for their own use, and the original Ripple network still runs on unmodified Ripple Labs software. Stellar was started by Jed McCaleb, who founded Ripple Labs, but broke with the CEO last year.) The Ripple consensus protocol puts certain requirements on the topology of the network of transaction-validating nodes in order to work properly. They are still investigating, but it's possible Stellar's network fell outside the workable range. Ripple Labs manages their network topology more carefully than Stellar, and this incident may validate their approach. We'll have to see what actually happened.