11 ms·

Lightning Network: “Bitcoin Doesn’t Scale” [pdf]

138 points by bootload 12y ago

DougN7 12y ago
I've always wondered how long until the blockchain would grow before it became unlikely new users/miners would want to hold a copy. At some point, it seems like it would grow faster than typical network download rates meaning a new miner, etc would never be able to get their initial copy.
- desdiv 12y ago
  You don't need the full blockchain to become a miner, you just need a database of all unspent transaction outputs. The last time I checked said database was only a couple hundred MB. As for people who do need the full blockchain in the future scenario you describe, they could always pay someone to sneakernet it to them.
  - TeMPOraL 12y ago
    > As for people who do need the full blockchain in the future scenario you describe, they could always pay someone to sneakernet it to them. I just imagined an armed convoy protecting a truck loaded with hard drives.
    - vanattab 12y ago
      I just imagined and army of Converse wearing hipsters loaded with hard drives.
      - sp332 12y ago
        https://www.youtube.com/watch?feature=player_detailpage&v=jrHXu7M-U68#t=85 https://www.youtube.com/watch?feature=player_detailpage&v=jr...
    - uptown 12y ago
      Which would make no sense. The whole idea of a distributed ledger is that there's not one authoritative copy. Stealing those drives would yield nothing because it wouldn't correlate to the rest of the copies in existence.
      - deleted 12y ago
        [deleted]
    - gwern 12y ago
      Imagine instead one armed courier with a briefcase containing a key, and a bunch of hard drives with signed files mailed via USPS. :)
- sickpig 12y ago
  Users could use SPV clients in case of resource constraints, on the other hand there are incentives in place, that till now, have kept miners from shutting down their nodes. Bitcoin core latest version (0.10.0) have reduced significantly the bootstrap time of a node (e.g. through "header-first" mechanism). Future releases will introduce other improvements that will lead to a reduction in terms of network bandwidth/latency (O(1) block propagation, IBLT) and IO space (e.g. pruning) That said it's true there's no economic incentives in place to "just" run a full node. Person that run full nodes does that for a lot of reasons modulo the economic one. Hence the number of full nodes is steadily declining http://www.bitcoinpulse.com/#/chart/bitnodes/num_nodes http://www.bitcoinpulse.com/#/chart/bitnodes/num_nodes There are proposals (1) floating around to fix the situation through, guess what... economic incentives, but we are for from getting some sort of agreement, worse than that I dare say that in this case we're still struggling with problem awareness. (1) https://bitcoinism.liberty.me/2015/02/09/economic-fallacies-and-the-block-size-limit-part-2-price-discovery/ https://bitcoinism.liberty.me/2015/02/09/economic-fallacies-...
oleganza 12y ago
That's an interesting idea, can't wait for the paper to come out. Compared to my scheme of Ripple-like IOU network with point-to-point joint escrow (https://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg07006.html https://www.mail-archive.com/bitcoin-development@lists.sourc...) here are pros/cons (to the best of my understanding so far): Pros: 1. They already have a paper and I don't :-) 2. No need for extra capital lock up between nodes to cover bidirectional IOUs. Cons: 1. Channels can be "closed" any time forcing everyone in the chain (could be 7-10 nodes) to make a BTC tx whether they really want it or not. 2. Necessary timeouts require channels to be closed after some time, while in theory some of them can be kept open indefinitely. In my scheme, interior pairs of nodes can pay each other accumulated transaction fees in separate transactions whenever they want without affecting anyone.
knocte 12y ago
Interesting! So I guess this means that thanks to the Lightning network, we can have off-chain transactions without the need to rely on centralization (i.e. ChangeTip).
sharpneli 12y ago
Their assumption of 2 channel roll-overs per year per person is quite likely far too low. In their comparison they basically state that a person would be participant in a channel roll over per 3650 transactions made. This seems to be overly optimistic considering that the acceptable credit card chargeback rate is 1%. 36 times per the rate of channel roll overs. In addition even nicely working channel has to be created. So even without anything like chargebacks assuming 3650 transactions per year on a single chain seems to be extremely optimistic. It might reduce spam on the network though. Bringing us from the 240G blocks into 24G blocks (Based on the well known Stetson-Harrison method). A fine improvement, but doesn't really seem to be a silver bullet for the scalability issues.
- moe 12y ago
  This seems to be overly optimistic considering that the acceptable credit card chargeback rate is 1%. 36 times per the rate of channel roll overs. That may be true. But even when multiplied with your proposed factor 36 it still looks very promising: ~4.7G blocks, ~108Mbit/s, 252T/yr for full blockchain archival. In fact, that looks almost too good to be true for a blockchain with 7 billion participants. Did I make a mistake in my math? Keep in mind that (if I understand it correctly) these are only the requirements for miners ("supernodes"). Mere mortals would largely operate off-chain (and SPV) with near-zero bandwidth and storage requirements.
  - sharpneli 12y ago
    Even if it would help that much the original issue with scalability still remains. It's not feasible for any individual to validate the blockchain. Maintaining a full node is going to be expensive. So even in the best case it would be "Instead of scaling ridiculously badly bitcoin scales only horrendously badly". A natural solution for this, if we really really want to get cryptocurrencies to work, would be to basically have a cryptocurrency per nation or so, with different PoW function to avoid shifting asics around. The only reason Bitcoiners want a person in Argentina to keep a Finnish potato sack purchase in their books is just to pump up the value of their 'investment'. Every solution made in relation to Bitcoin is made from the viewpoint that if it would make any current Bitcoin holders potentially miss out on global domination it's ignored. Even if it would actually work better.
late2part 12y ago
Sorry, you lost me at: The SQL Database Model ○ Very scalable, very fast
- rnbrady 12y ago
  lol
- imaginenore 12y ago
  Pretty much every modern DB is capable of thousands of transactions per second on moderate hardware.
bhouston 12y ago
Interesting parallel to the growth and development of decentralized P2P networks around 1999/2000. First there was Gnutella (as Napster had a centralized directory) - but it didn't scale, although it proved popular. Then they added support to Gnutella for spoke/hub models, which had superpeers that reduced the load on most individuals. But in the end Gnutella never really took off. What replaced Gnutella was bittorrent, which actually only did transfer via P2P but the searching and discovery was centralized again PirateBay and Trackers. But for all bittorrent's success, Netflix (and others), which are centralized and highly controlled alternatives, are far more popular and commercially successful for watching online video -- it is just less work to use Netflix, even though it doesn't have the benefits of being free and open. So for all the attraction of P2P decentralization, often centralized, commercial focused easy solutions win out. There are costs to be paid by being decentralized and often general people who are not ideological do not want to pay these costs. But I am mostly an outsider with regards to Bitcoin so who knows.
- flatline 12y ago
  While there are certain advantages to netflix's distribution model, such as my ability to stream directly to a tablet or AppleTV, the main advantage in my mind is that it is a legal means of acquiring content with ease. If BitTorrent allowed me to pay for and legally obtain content, I'd probably use it quite a bit. The inefficiencies in the Bitcoin protocol are pretty big by comparison; every full node has to maintain a copy of the entire blockchain, the proof of work scheme is terribly redundant and wasteful. Some partial solutions (peercoin, ripple/stellar) have arisen in the crypto space but they just don't have the popularity of Bitcoin, it'll be interesting to see where the field stands in even five years. The real world demand for decentralized digital currency is still questionable in my mind.
  - bhouston 12y ago
    > The real world demand for decentralized digital currency is still questionable in my mind. This is a key point. I believe that general people do not care about decentralized currency, what they want are easy payment solutions. If bitcoin doesn't provide easy payment solutions that are cheap, convenient, works everywhere, then it will be beat by something that does provide that. Even if you could with huge engineering effort slowly move Bitcoin to cheap, easy, everywhere, it is likely going to be beat by non-decentralized solutions because those non-decentralized solutions are cheaper to create, deploy and maintain -- also easier to monetize. I think that the future is to companies like Stripe, ApplePay -- technologies which focus on customer capture via ease of use and many options and being agnostic towards the underlying technologies. Bitcoin is neat, just like Gnutella was, but being neat doesn't correlate with commercial success.
_cpancake 12y ago
Bitcoin either needs tons of bandwidth and storage space, or centralization. You can't have your cake and eat it too.
- sp332 12y ago
  It takes a lot of bandwidth to verify the blockchain initially, but you don't have to store it all. Once you see that an input has been spent, you can discard the earlier transactions without losing information about the current balances. So you'd only need a few hundred MB of storage space.
  - towelguy 12y ago
    This is interesting, I thought in order to use a fully independent client you needed to store the whole blockchain. Which clients support this?
    - sp332 12y ago
      Most of what's needed for this has been in the main/Satoshi client since 2012. The client uses the pruned chain for most transactions because it's faster, and just uses the full blockchain for syncing with other clients. But I don't know of any clients that turn off the full-blockchain part and use only the pruned one. https://bitcointalk.org/index.php?topic=119525.0 https://bitcointalk.org/index.php?topic=119525.0 https://bitcointalk.org/index.php?topic=118660.msg1285690#msg1285690 https://bitcointalk.org/index.php?topic=118660.msg1285690#ms...
- IkmoIkmo 12y ago
  Not really. US citizen makes 2 transactions per day. That's 1 kilobyte of data total. That's really not a problem, centralized or decentralized, for bandwidth, storage or cpu to handle. Fact of the matter is that a transaction costs some money, like 5 cents, which is in a way the price to transmit 500 bytes of data, verify it and store it. That's economical at any scale. And guess what, the marginal cost is only decreasing due to technology improvements. (moore's, kryder's and nielsen's laws) The price of 1 gigabyte of storage is about 3 pennies today (retail), for example. And storage is not permanent either, you can prune old transactions (that's an innovation currently in development). And again, that price is only dropping, in 1990 that price was a few thousand dollars by comparison.
inoop 12y ago
While I appreciate the effort, I wonder why people think Bitcoin will gain mainstream adoption in the first place. Besides buying drugs online, what is really the killer app here?
- rplnt 12y ago
  Getting rid of payment processors would benefit just about everyone. Only benefit they have is insurance.
  - TylerE 12y ago
    For consumers that's a MASSIVE benefit.
    - aianus 12y ago
      Not really, the largest single transaction I've ever made on a credit card is maybe $2000 in flight tickets. At ~$30,000 a year in credit card spending I'm throwing that amount away in credit card fees every two and a half years. I'm perfectly capable of and would prefer the option to self-insure, thank you very much.
      - TylerE 12y ago
        Are you really naive enough to think that retailers won't just pocket whatever savings they have?
        aianus 12y ago
        Retail is not a very high-margin industry, I imagine Walmart, Amazon, Airlines et al would pass on the savings.
      - myhf 12y ago
        Fees? Shouldn't you be getting that amount back in loyalty points?
revmoo 12y ago
Bitcoin can scale just fine. Handing dollar bills to everyone manually doesn't scale either, hence payment networks have sprung up to meet the need.
Animats 12y ago
That distributed-trust scheme is so complicated that there's probably a security hole in there somewhere. Ripple tries to do something similar. How's that working out? Are the fancier operations in Ripple actually used? The Bitcoin transaction rate is currently only about 100K/day, or 1.1/sec. That's only twice what it was two years ago. There's no urgency in solving this problem.

138 points by bootload 12y ago

I've always wondered how long until the blockchain would grow before it became unlikely new users/miners would want to hold a copy. At some point, it seems like it would grow faster than typical network download rates meaning a new miner, etc would never be able to get their initial copy.

desdiv 12y ago
You don't need the full blockchain to become a miner, you just need a database of all unspent transaction outputs. The last time I checked said database was only a couple hundred MB. As for people who do need the full blockchain in the future scenario you describe, they could always pay someone to sneakernet it to them.
- TeMPOraL 12y ago
  > As for people who do need the full blockchain in the future scenario you describe, they could always pay someone to sneakernet it to them. I just imagined an armed convoy protecting a truck loaded with hard drives.
  - vanattab 12y ago
    I just imagined and army of Converse wearing hipsters loaded with hard drives.
    - sp332 12y ago
      https://www.youtube.com/watch?feature=player_detailpage&v=jrHXu7M-U68#t=85 https://www.youtube.com/watch?feature=player_detailpage&v=jr...
  - uptown 12y ago
    Which would make no sense. The whole idea of a distributed ledger is that there's not one authoritative copy. Stealing those drives would yield nothing because it wouldn't correlate to the rest of the copies in existence.
    - deleted 12y ago
      [deleted]
  - gwern 12y ago
    Imagine instead one armed courier with a briefcase containing a key, and a bunch of hard drives with signed files mailed via USPS. :)
sickpig 12y ago
Users could use SPV clients in case of resource constraints, on the other hand there are incentives in place, that till now, have kept miners from shutting down their nodes. Bitcoin core latest version (0.10.0) have reduced significantly the bootstrap time of a node (e.g. through "header-first" mechanism). Future releases will introduce other improvements that will lead to a reduction in terms of network bandwidth/latency (O(1) block propagation, IBLT) and IO space (e.g. pruning) That said it's true there's no economic incentives in place to "just" run a full node. Person that run full nodes does that for a lot of reasons modulo the economic one. Hence the number of full nodes is steadily declining http://www.bitcoinpulse.com/#/chart/bitnodes/num_nodes http://www.bitcoinpulse.com/#/chart/bitnodes/num_nodes There are proposals (1) floating around to fix the situation through, guess what... economic incentives, but we are for from getting some sort of agreement, worse than that I dare say that in this case we're still struggling with problem awareness. (1) https://bitcoinism.liberty.me/2015/02/09/economic-fallacies-and-the-block-size-limit-part-2-price-discovery/ https://bitcoinism.liberty.me/2015/02/09/economic-fallacies-...

oleganza 12y ago

That's an interesting idea, can't wait for the paper to come out. Compared to my scheme of Ripple-like IOU network with point-to-point joint escrow (https://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg07006.html https://www.mail-archive.com/bitcoin-development@lists.sourc...) here are pros/cons (to the best of my understanding so far): Pros: 1. They already have a paper and I don't :-) 2. No need for extra capital lock up between nodes to cover bidirectional IOUs. Cons: 1. Channels can be "closed" any time forcing everyone in the chain (could be 7-10 nodes) to make a BTC tx whether they really want it or not. 2. Necessary timeouts require channels to be closed after some time, while in theory some of them can be kept open indefinitely. In my scheme, interior pairs of nodes can pay each other accumulated transaction fees in separate transactions whenever they want without affecting anyone.

knocte 12y ago

Interesting! So I guess this means that thanks to the Lightning network, we can have off-chain transactions without the need to rely on centralization (i.e. ChangeTip).

sharpneli 12y ago

Their assumption of 2 channel roll-overs per year per person is quite likely far too low. In their comparison they basically state that a person would be participant in a channel roll over per 3650 transactions made. This seems to be overly optimistic considering that the acceptable credit card chargeback rate is 1%. 36 times per the rate of channel roll overs. In addition even nicely working channel has to be created. So even without anything like chargebacks assuming 3650 transactions per year on a single chain seems to be extremely optimistic. It might reduce spam on the network though. Bringing us from the 240G blocks into 24G blocks (Based on the well known Stetson-Harrison method). A fine improvement, but doesn't really seem to be a silver bullet for the scalability issues.

moe 12y ago
This seems to be overly optimistic considering that the acceptable credit card chargeback rate is 1%. 36 times per the rate of channel roll overs. That may be true. But even when multiplied with your proposed factor 36 it still looks very promising: ~4.7G blocks, ~108Mbit/s, 252T/yr for full blockchain archival. In fact, that looks almost too good to be true for a blockchain with 7 billion participants. Did I make a mistake in my math? Keep in mind that (if I understand it correctly) these are only the requirements for miners ("supernodes"). Mere mortals would largely operate off-chain (and SPV) with near-zero bandwidth and storage requirements.
- sharpneli 12y ago
  Even if it would help that much the original issue with scalability still remains. It's not feasible for any individual to validate the blockchain. Maintaining a full node is going to be expensive. So even in the best case it would be "Instead of scaling ridiculously badly bitcoin scales only horrendously badly". A natural solution for this, if we really really want to get cryptocurrencies to work, would be to basically have a cryptocurrency per nation or so, with different PoW function to avoid shifting asics around. The only reason Bitcoiners want a person in Argentina to keep a Finnish potato sack purchase in their books is just to pump up the value of their 'investment'. Every solution made in relation to Bitcoin is made from the viewpoint that if it would make any current Bitcoin holders potentially miss out on global domination it's ignored. Even if it would actually work better.

late2part 12y ago

Sorry, you lost me at: The SQL Database Model ○ Very scalable, very fast

rnbrady 12y ago
lol
imaginenore 12y ago
Pretty much every modern DB is capable of thousands of transactions per second on moderate hardware.

bhouston 12y ago

Interesting parallel to the growth and development of decentralized P2P networks around 1999/2000. First there was Gnutella (as Napster had a centralized directory) - but it didn't scale, although it proved popular. Then they added support to Gnutella for spoke/hub models, which had superpeers that reduced the load on most individuals. But in the end Gnutella never really took off. What replaced Gnutella was bittorrent, which actually only did transfer via P2P but the searching and discovery was centralized again PirateBay and Trackers. But for all bittorrent's success, Netflix (and others), which are centralized and highly controlled alternatives, are far more popular and commercially successful for watching online video -- it is just less work to use Netflix, even though it doesn't have the benefits of being free and open. So for all the attraction of P2P decentralization, often centralized, commercial focused easy solutions win out. There are costs to be paid by being decentralized and often general people who are not ideological do not want to pay these costs. But I am mostly an outsider with regards to Bitcoin so who knows.

flatline 12y ago
While there are certain advantages to netflix's distribution model, such as my ability to stream directly to a tablet or AppleTV, the main advantage in my mind is that it is a legal means of acquiring content with ease. If BitTorrent allowed me to pay for and legally obtain content, I'd probably use it quite a bit. The inefficiencies in the Bitcoin protocol are pretty big by comparison; every full node has to maintain a copy of the entire blockchain, the proof of work scheme is terribly redundant and wasteful. Some partial solutions (peercoin, ripple/stellar) have arisen in the crypto space but they just don't have the popularity of Bitcoin, it'll be interesting to see where the field stands in even five years. The real world demand for decentralized digital currency is still questionable in my mind.
- bhouston 12y ago
  > The real world demand for decentralized digital currency is still questionable in my mind. This is a key point. I believe that general people do not care about decentralized currency, what they want are easy payment solutions. If bitcoin doesn't provide easy payment solutions that are cheap, convenient, works everywhere, then it will be beat by something that does provide that. Even if you could with huge engineering effort slowly move Bitcoin to cheap, easy, everywhere, it is likely going to be beat by non-decentralized solutions because those non-decentralized solutions are cheaper to create, deploy and maintain -- also easier to monetize. I think that the future is to companies like Stripe, ApplePay -- technologies which focus on customer capture via ease of use and many options and being agnostic towards the underlying technologies. Bitcoin is neat, just like Gnutella was, but being neat doesn't correlate with commercial success.

_cpancake 12y ago

Bitcoin either needs tons of bandwidth and storage space, or centralization. You can't have your cake and eat it too.

sp332 12y ago
It takes a lot of bandwidth to verify the blockchain initially, but you don't have to store it all. Once you see that an input has been spent, you can discard the earlier transactions without losing information about the current balances. So you'd only need a few hundred MB of storage space.
- towelguy 12y ago
  This is interesting, I thought in order to use a fully independent client you needed to store the whole blockchain. Which clients support this?
  - sp332 12y ago
    Most of what's needed for this has been in the main/Satoshi client since 2012. The client uses the pruned chain for most transactions because it's faster, and just uses the full blockchain for syncing with other clients. But I don't know of any clients that turn off the full-blockchain part and use only the pruned one. https://bitcointalk.org/index.php?topic=119525.0 https://bitcointalk.org/index.php?topic=119525.0 https://bitcointalk.org/index.php?topic=118660.msg1285690#msg1285690 https://bitcointalk.org/index.php?topic=118660.msg1285690#ms...
IkmoIkmo 12y ago
Not really. US citizen makes 2 transactions per day. That's 1 kilobyte of data total. That's really not a problem, centralized or decentralized, for bandwidth, storage or cpu to handle. Fact of the matter is that a transaction costs some money, like 5 cents, which is in a way the price to transmit 500 bytes of data, verify it and store it. That's economical at any scale. And guess what, the marginal cost is only decreasing due to technology improvements. (moore's, kryder's and nielsen's laws) The price of 1 gigabyte of storage is about 3 pennies today (retail), for example. And storage is not permanent either, you can prune old transactions (that's an innovation currently in development). And again, that price is only dropping, in 1990 that price was a few thousand dollars by comparison.

inoop 12y ago

While I appreciate the effort, I wonder why people think Bitcoin will gain mainstream adoption in the first place. Besides buying drugs online, what is really the killer app here?

rplnt 12y ago
Getting rid of payment processors would benefit just about everyone. Only benefit they have is insurance.
- TylerE 12y ago
  For consumers that's a MASSIVE benefit.
  - aianus 12y ago
    Not really, the largest single transaction I've ever made on a credit card is maybe $2000 in flight tickets. At ~$30,000 a year in credit card spending I'm throwing that amount away in credit card fees every two and a half years. I'm perfectly capable of and would prefer the option to self-insure, thank you very much.
    - TylerE 12y ago
      Are you really naive enough to think that retailers won't just pocket whatever savings they have?
      - aianus 12y ago
        Retail is not a very high-margin industry, I imagine Walmart, Amazon, Airlines et al would pass on the savings.
    - myhf 12y ago
      Fees? Shouldn't you be getting that amount back in loyalty points?

revmoo 12y ago

Bitcoin can scale just fine. Handing dollar bills to everyone manually doesn't scale either, hence payment networks have sprung up to meet the need.

Animats 12y ago

That distributed-trust scheme is so complicated that there's probably a security hole in there somewhere. Ripple tries to do something similar. How's that working out? Are the fancier operations in Ripple actually used? The Bitcoin transaction rate is currently only about 100K/day, or 1.1/sec. That's only twice what it was two years ago. There's no urgency in solving this problem.