13 ms·

Bitcoin's ASICBOOST Problem Explained [pdf]

152 points by jlrubin 10y ago

mjevans 10y ago
Wasn't the entire point of picking SHA-256 to both check the integrity of the existing ledger (why it's a hash) AND as a proof of work lottery? The "problem" being that instead of being a proof of actual work, shortcuts in the work were found. It seems then that a good solution would be to force many different types of proof of work. Different basis of hashes are obvious, but maybe some type of actual work every so often as well.
- jlrubin 10y ago
  The problem with the proof of work is that the machinery became dependent on the selection of transactions in the block. A "standard" miner does not have this dependency. It's not a problem (depending on who you ask) that an optimization was found, the amount of work expended is still the same. It's not clear that switching proof of works would eliminate this class of concern. It's also clear that eliminating this class of problem is not in the interest of the miners, hence the opposition to SegWit (which does exactly that).
  - hudon 10y ago
    > It's also clear that eliminating this class of problem is not in the interest of the miners, hence the opposition to SegWit Most miners have been supportive of SegWit for at least a year, see the HK agreement: https://medium.com/@bitcoinroundtable/bitcoin-roundtable-consensus-266d475a61ff https://medium.com/@bitcoinroundtable/bitcoin-roundtable-con... The agreement was that they'd support it if it came with a 2mb hardfork, and it never did. Sergio Lerner released segwit2mb recently, which the miners support, but many Core devs were vocal about their opposition to that.
    - jlrubin 10y ago
      SegWit + HF does not eliminate this class of optimization. Most SegWit+HF proposals would reorganize the bitcoin header structure so as to be compatible with ASICBOOST-like optimizations.
      - hudon 10y ago
        I see what you're saying, but I don't think you read the agreement. > We understand that SegWit continues to be developed actively as a soft-fork and is likely to proceed towards release over the next two months, as originally scheduled. The key part here being that SegWit as a soft-fork is what was agreed to.
        jlrubin 10y ago
        That's one way to read it. Another way is: we are OK with a segwit soft-fork sooner so long as we can later clean it up with a hard-fork and re-enable our optimizations by June 2017. When a hard-fork happens, it can do almost anything, including rewrite a soft fork to be cleaner. Luke Dashjr's hard-fork proposals, fulfilling his end of the HK agreement, did not modify SegWit's commitment, but did modify other annoying parts of the protocol (like version numbers -- breaking compatibility on headers) unrelated to block size. I can't speak for Luke, but if someone had told Luke about ASICBOOST's concern with SegWit, I'm sure he would have made his 3 BIPs address that too.
        hudon 10y ago
        > Luke Dashjr's hard-fork proposals, fulfilling his end of the HK agreement, [...] Which proposal are you talking about? Luke proposed a block size decrease, followed by a small increase in a decade... The HK agreement asks for a max block size increase, not a decrease and Luke never proposed such a hard fork. Again, I recommend reading the link I posted so we can discuss without false statements :)
        jlrubin 10y ago
        https://github.com/luke-jr/bips/blob/bip-blksize/bip-blksize.mediawiki https://github.com/luke-jr/bips/blob/bip-blksize/bip-blksize..., which includes a reference implementation. It does optionally include an immediate reduction, followed by a gradual increase. This was developed, as far as I understand, because Luke felt it fulfilled his personal end of the HK agreement -- to develop a hard-fork proposal. To quote the intro: > The block size limit is reduced to a reasonably safe value (optional), and gradually increased over time, eventually expanding beyond the current limits.
    - midmagico 10y ago
      No, instead of came with a 4MB softfork, which is a strictly superior alternative, since for all the reasons they support extblocks, SegWit is completely superior.
- wyager 10y ago
  > The "problem" being that instead of being a proof of actual work, shortcuts in the work were found. The same "problem" occurred when people invented GPU mining, then FPGA mining, then ASIC mining. As long as SHA256 isn't broken, it doesn't matter what kind of speedup miners manage to get; whenever someone finds a big advantage, everyone else uses it soon enough. > Different basis of hashes are obvious "Use more hash functions" is sort of a reflexive response you get a lot from non-cryptographers, but the intuitive reason this is a bad idea is that if your hash function isn't broken, your cryptosystem shouldn't be either. It's better to fix the cryptosystem than to half-assedly patch it over by obfuscating any problems with more hash functions. > but maybe some type of actual work every so often as well. What do you mean by "actual work"? In all likelihood, whatever you're thinking of has none of the properties that make PoW a viable mechanism for sybil resistance.
  - petertodd 10y ago
    > everyone else uses it soon enough. Unfortunately in this case, ASICBOOST is both patented, and adds a fair bit of complexity to mining hardware. Both these problems have the undesirable effect of increasing the barriers to entry for new asic manufacturers - an undesirable thing from the point of view of Bitcoin users. You're quite correct that shortcuts are irrelevant, but the reality of manufacturing makes the playing field a lot less level than we'd like.
    - shea256 10y ago
      Exactly, I'd go further and say that if a particular technique is patented and cannot be matched by a related technique, giving a permanent unfair advantage, it is up to the community to change the proof of work and work around the advantage imposed by a state monopoly.
      - jcoffland 10y ago
        Then Bitcoin has to hard fork every time someone comes up with a new patentable idea. I don't see why people think mining must be fair. It only has to not become completely centralized.
- jcoffland 10y ago
  > It seems then that a good solution would be to force many different types of proof of work. This would create many more opportunities for finding shortcuts like ASCIBOOST. Some of them might be much worse.
  - MichaelGG 10y ago
    Really? Suppose both a SHA256 and a SHA1 hash were needed. How could it be possible that it's easier to find a simultaneous weakness in both? I get that randomly throwing stuff together isn't favoured (like TLS's PRF SHA1+MD5 construction). But is there any actual work where requiring 2 separate full checks to be worse than just one? That is, needing to verify MD5 plus SHA1 independently.
    - jcoffland 10y ago
      First of all we aren't talking about weaknesses in hash functions. We are talking about ways to compute hashes faster. > But is there any actual work where requiring 2 separate full checks to be worse than just one? That is, needing to verify MD5 plus SHA1 independently. One could then use both SHA1 and MD5 shortcuts.
      - MichaelGG 10y ago
        OK, but having to use both a SHA1 and MD5 shortcut is still going to be more work than using just one of them.
        yorwba 10y ago
        But it's less work than having to compute both SHA1 and MD5 without shortcut, which is the comparison that matters.
    - lmm 10y ago
      > I get that randomly throwing stuff together isn't favoured (like TLS's PRF SHA1+MD5 construction). But is there any actual work where requiring 2 separate full checks to be worse than just one? That is, needing to verify MD5 plus SHA1 independently. Even a crappy 256-bit hash (by using two 128-bit hashes) will probably be more secure than a (single) 128-bit hash, sure. But it won't be anywhere near as secure as a proper 256-bit hash. If you can afford 256 bits of hash, the best way to spend them is on a dedicated 256-bit hash.
- midmagico 10y ago
  > The "problem" being that instead of being a proof of actual work, shortcuts in the work were found. Nope, the problem is that covert ASICBoost mining has created a strong, valuable incentive for miners to oppose protocol upgrades while supporting much more destructive ones like the recent extblocks, which is susceptible to instantaneous, irreversible full transaction invalidation on a reorg. ASICBoost in its other forms is completely compatible with SegWit.
LARBRcom 10y ago
Larbr | Pukhtunkhwa Social Network https://www.larbr.com/ https://www.larbr.com/
jcoffland 10y ago
This is excellent. I've been trying to find a technical explanation of ASCIBOOST for some time. Most of the articles and discussions gloss over the details. It is possible that this is part of the reason some miners are blocking SegWit but there is more to it. Core (the main Bitcoin development team) promised to provide a block size scaling solution and then reneged. This pissed a lot of people off and led to the blocking of SegWit. I don't see ASCIBOOST as a real problem for Bitcoin.
- tromp 10y ago
  The AsicBoost inventors also wrote a good technical explanation at https://arxiv.org/pdf/1604.00575v1 https://arxiv.org/pdf/1604.00575v1
  - jlrubin 10y ago
    Notably, they don't include the optimal root generation algorithm, nor do they discuss segwit incompatibilities. Also, their 20% figure is unclear. The more collisions you can find, the more hash per second you should be able to gain.
    - jcoffland 10y ago
      > The more collisions you can find, the more hash per second you should be able to gain. But the chance of finding three blocks with the necessary 4 byte collision is much more difficult. It's the same problem as finding 3 people with the same birthday. You need 87 people to have a 50/50 chance of three people having the same birthday vs. 23 for two people who share a birthday. That's nearly a 4x increase in required computation for a small increase in effective hash rate.
      - jlrubin 10y ago
        It's not that big of a computation even at 4 collisions, but yes, I'm familiar with the math (I wrote the PDF).
wmf 10y ago
Some context: http://hackingdistributed.com/2017/04/05/bitcoin-drama-response/ http://hackingdistributed.com/2017/04/05/bitcoin-drama-respo...
- AgentME 10y ago
  I find this article very misleading. >2. If ASICBOOST was actually used, we'd see ample evidence on the blockchain. The covert form of ASICBOOST (where they don't use the version field, the form that was only recently publicly discovered) would only show up as a higher than usual number of empty blocks or blocks with reordered or missing transactions (depending on how the attacker implemented it; it's not necessarily both). >3. ... As you can also examine for yourself, the transactions are ordered essentially by fee-per-byte, which is not what we would see if they were shuffled to create collisions. I find it extremely disingenuous that the author only focuses on looking for reordered transactions and ignores the often-noticed unusual number of empty blocks that Antpool mines[0], and the article reaches absurdity when the author claims that the lack of reordered transactions means there's absolutely nothing to worry about. >4. Building more efficient mining chips is what miners are supposed to do, as it secures the blockchain. Framing an optimization as an attack is disingenuous. Going from 28nm to 16nm was not an attack on the network. Better mining algorithms have never been considered an attack. If for example a mining "optimization" encouraged every miner to mine only empty blocks, then calling it an attack doesn't sound out of place. An optimization that only often encourages miners to make empty blocks isn't categorically different. >7. Segwit is being offered as a solution to ASICBOOST. Yet Segwit has at best a tenuous connection as a response to ASICBOOST mining. There are countless other, non-controversial solutions that can disable ASICBOOST, should one choose (wrongly) to do so. Greg Maxwell's very post announcing the issue introduces a fix for the covert form of ASICBOOST that does not involve implementing SegWit! [0] https://bitcoin.stackexchange.com/questions/50184/why-does-antpool-mine-so-many-empty-blocks https://bitcoin.stackexchange.com/questions/50184/why-does-a... http://www.livebitcoinnews.com/antpool-generating-large-amount-empty-bitcoin-blocks/ http://www.livebitcoinnews.com/antpool-generating-large-amou...
  - kylebenzle 10y ago
    Either I am crazy and don't understand bitcoin or some well funded group is spending a lot of money to support SegWit and push Core's agenda. The few sensible posts like yours are downvoted to hell and back while pro-SegWit rocket to the top. Every point you make is spot on and there is no sensible reply to any of them, just a bunch of garbage about no one supports BU and BU will never happen...
needs 10y ago
Just to point out that SegWit is everything but "already adopted in the industry". It turns out that SegWit is the solution promoted by the developers of bitcoin core to allow for bigger blocks, and solve some issues like transactions maleability. It's more than 10 000 lines of code highly controversial because they require... A soft fork, and will change bitcoin in a fundamental ways. Not to add that a company called "Blockstream" have paid almost every developers of bitcoin core to push Segwit without a clear consensus from the community. Worse, this company actually support the huge censorship happening in bitcointalk and r/bitcoin. Blockstream received more than 75 millions from AXA, for obscure reasons. The paper only focus on Segwit, without mentioning that Segwit has nothing to do with ASICBOOST. The community and a growing majority of miners is actually switching to bitcoin unlimited, a fork of bitcoin core. Sorry for the little off topic comment but I love Bitcoin and I hate that a few dishonest people took control of it and try to destroy it.
- efesak 10y ago
  - there are ~5000 LOC of https://github.com/bitcoin/bitcoin/pull/7910/files https://github.com/bitcoin/bitcoin/pull/7910/files and almost 3/4 of these lines are tests. - Segwit actualy block harmful form of ASICBOOST - by "growing majority" you mean handful of chinese miners and 2% of nodes (http://luke.dashjr.org/programs/bitcoin/files/charts/software.html http://luke.dashjr.org/programs/bitcoin/files/charts/softwar...)
  - needs 10y ago
    > - Segwit actualy block harmful form of ASICBOOST It's - of course - not the only solution to this "problem", but somehow this paper doesn't propose anything else other than segwit, and his conclusion is a typical Bitcoin Core propaganda. > by "growing majority" you mean handful of chinese miners and 2% of nodes (http://luke.dashjr.org/programs/bitcoin/files/charts/softwar... http://luke.dashjr.org/programs/bitcoin/files/charts/softwar...) You aren't without knowing that in Bitcoin, miners establish consensus by voting with their hashrate. Nodes count doesn't prove anything because its really easy to host a node. The point is: there is more miners supporting BU than Segwit [1]. [1]: http://xtnodes.com/#bitcoin_classic_blocks http://xtnodes.com/#bitcoin_classic_blocks
fpgaminer 10y ago
Besides the technical issues involved here, there's been a lot of political quarrels as well. Here's my run down of them. Note that I was heavily involved in the Bitcoin community a few years ago, but have been on the side lines recently. As with all things political, take my interpretations with a grain of salt: 1) The Bitcoin network began experiencing congestion due to rise in popularity driving large numbers of transactions. This resulted in slower transaction verification and higher transaction fees. 2) Users started asking for larger blocks, so that the network would have higher transaction bandwidth. The defacto Bitcoin developers at the time pushed back. Increasing block size is simple in terms of code change, but requires a risky hard fork. It's also only a temporary fix. Bitcoin will grow and require another increase, and thus another hard fork. They wanted a better, long term solution. 3) After a lot of discussion the developers came back with Segregated Witness. It's a soft modification to Bitcoin which meant no hard fork. It doesn't explicitly allow larger blocks. It does, however, upgrade Bitcoin to allow side chains. The argument is that faster, more abundant transactions can be implemented in a side chain system. A sort of second layer on top of Bitcoin. Much like how HTTP is implemented on top of the underlying TCP/IP protocol. It should be noted that, while this enablement of side chains is the biggest feature of SegWit, there were a few smaller improvements. It _does_ increase effective block size slightly; ~2x. And it fixes a few minor annoyances in the Bitcoin protocol (e.g. malleability). 4) The community was disappointed by the lack of larger blocks. While developers were busy putting together their idea and specification for SegWit, the congestion issues on the network continued to grow worse. 5) Eventually the debate became "on-chain" versus "off-chain". Some of the community argued that they wanted to keep all transaction on the Bitcoin blockchain. In other words, they still wanted bigger and bigger blocks. I believe the root of this argument is that Bitcoin has served us well for many years, so why try to build another, potentially weaker system on top of it? The other parts of the community were in favor of SegWit and off-chain transactions. Side chains would allow for faster transactions and significantly more of them, and they would consume far less disk space (a growing concern in the community is the disk space required to run a Bitcoin node). Side chains achieve this by compromising on some of the strengths of Bitcoin, either by reducing security or requiring more centralization. 6) Eventually the debate in the community shifted from just on-chain versus off-chain to SegWit versus bigger blocks. Now, to be clear, SegWit doesn't preclude the possibility of larger blocks. Obviously larger blocks can be added regardless of SegWit. But, for whatever reason, the dialog shifted. 7) When SegWit was finally released as part of the Bitcoin Core client (the defacto standard for Bitcoin clients/wallets) a sort of battle amongst the community began. Mining operations that ran the SegWit enabled clients started getting attacked; DDOS, etc. There was a lot of strife, hate, and fear. The way SegWit is implemented in the code, it requires a large majority of miners to support it before it officially activates and can be used. This is by design, to ensure the community agrees with the feature and to ensure almost all miners are able to understand the new blocks and not get left behind after activation. Because of this majority requirement, the battle in the community and between miners was very important. If half the miners didn't agree with SegWit, it would never activate. 8) This "battle" went on for a long time and is, currently, still on-going. Bitcoin continues to remain congested. Transaction fees have continued to rise. SegWit is still not activated. During this time a couple factions of the anti-SegWit community split off in attempts to fork Bitcoin. They modified versions of the Bitcoin client to enable larger blocks. So far, none of those forks have been successful, in terms of taking away any majority of users from the main Bitcoin network. 9) Recently it became public that a large mining operation may have been using a technology called ASICBOOST during this time. It allows mining chips to be more efficient, which means those chips make more Bitcoins for less money. Obviously an advantage. This technology has been known about for awhile now, but as far as the community knew no one was using it. It was covered by patents, and there's a sort of gentleman's agreement in the community not to use it as a result of the patent and potential ill effects on the network. (The patent means not all miners would be able to use it, so it presents an unfair playing field). ASICBOOST, the way this mining operation has supposedly been using it, is not compatible with SegWit. That's explained in the OP. The accused mining operation was also involved in a lot of the anti-SegWit activity in the community; promoting the alternative forks of Bitcoin. They had been accused in the past of using sock puppets to drive anti-SegWit agendas, drive character assassinations, etc. They are in control of a large percentage of the total Bitcoin mining power. The public revelation of them using ASICBOOST painted a dark, but enlightening picture. As I mentioned before, SegWit ultimately has nothing to do with the on-chain versus off-chain debate. It merely _enables_ off-chain possibilities. It doesn't force them. So it was odd to see the discussion shift from on-chain versus off-chain to SegWit versus Bigger Blocks. The theory now, given the accusations against this large mining operation, is that they are responsible for the majority of the anti-SegWit movement. Because the activation of SegWit would force them to stop using ASICBOOST the way they've been using it, it was in their financial interest to prevent SegWit from activating. That's basically my summary of events. I think that theory, that this mining operation has been basically using anti-SegWit propaganda and other attacks to prevent SegWit from activating, so they can keep using their secret version of ASICBOOST and profit (to the tune of $100 million a year), makes a lot of sense. Why else would anyone oppose SegWit? I've read through the SegWit specs. It enables a lot of really cool tech for Bitcoin. Side chains are not just about trying to get more, faster transactions into the Bitcoin network. They're also about allowing other technologies like smart contracts to tie themselves in Bitcoin. You know all those cool features that various alt coins are testing out? SegWit enables those features to become side chains so that you can gain all the security that Bitcoin offers. Side chain versus altcoin is like browser addon versus a whole new browser. At the end of the day, people in favor of bigger blocks could still campaign for bigger blocks. SegWit doesn't prevent that. A mining operation being financially incentivized to block SegWit is the simplest explanation I've heard yet for why there would be significant opposition to it.
RichardHeart 10y ago
Anything that makes mining empty blocks more profitable is bad. Empty blocks are an attack on the network. Someone is going to say "Empty blocks give people a reason to raise fees." DDOS attacks give people reasons to buy more expensive routers too.
tlrobinson 10y ago
This might not be the place for this question, but can someone explain why we can't / shouldn't have both unlimited block sizes and SegWit? Block sizes will supposedly be constrained by bandwidth / propagation times. SegWit will allow more transactions to occur off chain, reducing the need for larger blocks. They seem complementary to me.
- Kinnard 10y ago
  You are correct and such proposals have been put forward: http://moneyandstate.com/thoughts-on-segwit2mb/ http://moneyandstate.com/thoughts-on-segwit2mb/
- stale2002 10y ago
  The Core team does not want blocksize increases. They explicitly want prices to be high/number of transactions to be limited (think, supply and demand) . This is because transaction "price" is equivalent to "amount of money spent on securing the blockchain".
  - deleted 10y ago
    [deleted]
- shea256 10y ago
  SegWit does in fact represent a block size increase. It results in 1.7x the # of single-signature transactions per block and 4x the # of multi-signature transactions per block. Unlimited block size? There are many reasons that's a very bad idea but here are a few: 1. It would put immense pressure on the network in terms of latency between miners, leading to less stable mining, a higher rate of reorganizations, and a massive advantage for the larger miners, resulting in increased mining centralization. 2. It would result in a substantial increase in the amount of bandwidth that each node has to handle. Modest increases are OK but unlimited blocks means the vast majority of nodes will die off. Remember that nodes have to check blocks as they come in, so this would be an insane DDoS vector. Imagine putting a funnel in someone's mouth instead of a straw and being able to force whatever you want in there. 3. It would result in a substantial increase in the amount of data that each node has to store. Remember that in Bitcoin 100% of data must be stored by 100% of nodes. Imagine "unlimited emails" with Gmail meant that you had to store all of the emails in the world for EVERYONE. Not a good idea.
swedrupe 10y ago
9 months ago people already claimed on Reddit that Antminer H/W was using AsicBoost. This story seems just being dug up now by GMax for one or more of several reasons: * discredit Jihan / Bitmain / AntPool who support BU * distract from SegWit problems, buy time * solidify the "Chinese miners bad, Core good" narrative to support UASF, SegWit's final death throw * maybe give BitFury and other ASIC manufacturers some more business by scaring customers away from Bitmain In truth, there's no emergency or attack from ASIC manufacturers on Bitcoin, mining is more decentralized than before through many pools. This is about the politics of a small clique in the incumbent dev team who don't want to let go of power, at all costs, nor do they seem to be very worried about what happens to Bitcoin because of their unwillingness to compromise and even to deliver on their agreements.
cashmonkey85 10y ago
I genuinely believe most of the comments on this page are paid trolls. Which I didn't think was a thing on HN People should read up of the Chinese miners disinformation campaign. Really crazy stuff. https://medium.com/@WhalePanda/the-extended-extension-block-story-5bc3d888bdde https://medium.com/@WhalePanda/the-extended-extension-block-...

152 points by jlrubin 10y ago

Wasn't the entire point of picking SHA-256 to both check the integrity of the existing ledger (why it's a hash) AND as a proof of work lottery? The "problem" being that instead of being a proof of actual work, shortcuts in the work were found. It seems then that a good solution would be to force many different types of proof of work. Different basis of hashes are obvious, but maybe some type of actual work every so often as well.

jlrubin 10y ago
The problem with the proof of work is that the machinery became dependent on the selection of transactions in the block. A "standard" miner does not have this dependency. It's not a problem (depending on who you ask) that an optimization was found, the amount of work expended is still the same. It's not clear that switching proof of works would eliminate this class of concern. It's also clear that eliminating this class of problem is not in the interest of the miners, hence the opposition to SegWit (which does exactly that).
- hudon 10y ago
  > It's also clear that eliminating this class of problem is not in the interest of the miners, hence the opposition to SegWit Most miners have been supportive of SegWit for at least a year, see the HK agreement: https://medium.com/@bitcoinroundtable/bitcoin-roundtable-consensus-266d475a61ff https://medium.com/@bitcoinroundtable/bitcoin-roundtable-con... The agreement was that they'd support it if it came with a 2mb hardfork, and it never did. Sergio Lerner released segwit2mb recently, which the miners support, but many Core devs were vocal about their opposition to that.
  - jlrubin 10y ago
    SegWit + HF does not eliminate this class of optimization. Most SegWit+HF proposals would reorganize the bitcoin header structure so as to be compatible with ASICBOOST-like optimizations.
    - hudon 10y ago
      I see what you're saying, but I don't think you read the agreement. > We understand that SegWit continues to be developed actively as a soft-fork and is likely to proceed towards release over the next two months, as originally scheduled. The key part here being that SegWit as a soft-fork is what was agreed to.
      - jlrubin 10y ago
        That's one way to read it. Another way is: we are OK with a segwit soft-fork sooner so long as we can later clean it up with a hard-fork and re-enable our optimizations by June 2017. When a hard-fork happens, it can do almost anything, including rewrite a soft fork to be cleaner. Luke Dashjr's hard-fork proposals, fulfilling his end of the HK agreement, did not modify SegWit's commitment, but did modify other annoying parts of the protocol (like version numbers -- breaking compatibility on headers) unrelated to block size. I can't speak for Luke, but if someone had told Luke about ASICBOOST's concern with SegWit, I'm sure he would have made his 3 BIPs address that too.
        hudon 10y ago
        > Luke Dashjr's hard-fork proposals, fulfilling his end of the HK agreement, [...] Which proposal are you talking about? Luke proposed a block size decrease, followed by a small increase in a decade... The HK agreement asks for a max block size increase, not a decrease and Luke never proposed such a hard fork. Again, I recommend reading the link I posted so we can discuss without false statements :)
        jlrubin 10y ago
        https://github.com/luke-jr/bips/blob/bip-blksize/bip-blksize.mediawiki https://github.com/luke-jr/bips/blob/bip-blksize/bip-blksize..., which includes a reference implementation. It does optionally include an immediate reduction, followed by a gradual increase. This was developed, as far as I understand, because Luke felt it fulfilled his personal end of the HK agreement -- to develop a hard-fork proposal. To quote the intro: > The block size limit is reduced to a reasonably safe value (optional), and gradually increased over time, eventually expanding beyond the current limits.
  - midmagico 10y ago
    No, instead of came with a 4MB softfork, which is a strictly superior alternative, since for all the reasons they support extblocks, SegWit is completely superior.
wyager 10y ago
> The "problem" being that instead of being a proof of actual work, shortcuts in the work were found. The same "problem" occurred when people invented GPU mining, then FPGA mining, then ASIC mining. As long as SHA256 isn't broken, it doesn't matter what kind of speedup miners manage to get; whenever someone finds a big advantage, everyone else uses it soon enough. > Different basis of hashes are obvious "Use more hash functions" is sort of a reflexive response you get a lot from non-cryptographers, but the intuitive reason this is a bad idea is that if your hash function isn't broken, your cryptosystem shouldn't be either. It's better to fix the cryptosystem than to half-assedly patch it over by obfuscating any problems with more hash functions. > but maybe some type of actual work every so often as well. What do you mean by "actual work"? In all likelihood, whatever you're thinking of has none of the properties that make PoW a viable mechanism for sybil resistance.
- petertodd 10y ago
  > everyone else uses it soon enough. Unfortunately in this case, ASICBOOST is both patented, and adds a fair bit of complexity to mining hardware. Both these problems have the undesirable effect of increasing the barriers to entry for new asic manufacturers - an undesirable thing from the point of view of Bitcoin users. You're quite correct that shortcuts are irrelevant, but the reality of manufacturing makes the playing field a lot less level than we'd like.
  - shea256 10y ago
    Exactly, I'd go further and say that if a particular technique is patented and cannot be matched by a related technique, giving a permanent unfair advantage, it is up to the community to change the proof of work and work around the advantage imposed by a state monopoly.
    - jcoffland 10y ago
      Then Bitcoin has to hard fork every time someone comes up with a new patentable idea. I don't see why people think mining must be fair. It only has to not become completely centralized.
jcoffland 10y ago
> It seems then that a good solution would be to force many different types of proof of work. This would create many more opportunities for finding shortcuts like ASCIBOOST. Some of them might be much worse.
- MichaelGG 10y ago
  Really? Suppose both a SHA256 and a SHA1 hash were needed. How could it be possible that it's easier to find a simultaneous weakness in both? I get that randomly throwing stuff together isn't favoured (like TLS's PRF SHA1+MD5 construction). But is there any actual work where requiring 2 separate full checks to be worse than just one? That is, needing to verify MD5 plus SHA1 independently.
  - jcoffland 10y ago
    First of all we aren't talking about weaknesses in hash functions. We are talking about ways to compute hashes faster. > But is there any actual work where requiring 2 separate full checks to be worse than just one? That is, needing to verify MD5 plus SHA1 independently. One could then use both SHA1 and MD5 shortcuts.
    - MichaelGG 10y ago
      OK, but having to use both a SHA1 and MD5 shortcut is still going to be more work than using just one of them.
      - yorwba 10y ago
        But it's less work than having to compute both SHA1 and MD5 without shortcut, which is the comparison that matters.
  - lmm 10y ago
    > I get that randomly throwing stuff together isn't favoured (like TLS's PRF SHA1+MD5 construction). But is there any actual work where requiring 2 separate full checks to be worse than just one? That is, needing to verify MD5 plus SHA1 independently. Even a crappy 256-bit hash (by using two 128-bit hashes) will probably be more secure than a (single) 128-bit hash, sure. But it won't be anywhere near as secure as a proper 256-bit hash. If you can afford 256 bits of hash, the best way to spend them is on a dedicated 256-bit hash.
midmagico 10y ago
> The "problem" being that instead of being a proof of actual work, shortcuts in the work were found. Nope, the problem is that covert ASICBoost mining has created a strong, valuable incentive for miners to oppose protocol upgrades while supporting much more destructive ones like the recent extblocks, which is susceptible to instantaneous, irreversible full transaction invalidation on a reorg. ASICBoost in its other forms is completely compatible with SegWit.

LARBRcom 10y ago

Larbr | Pukhtunkhwa Social Network https://www.larbr.com/ https://www.larbr.com/

jcoffland 10y ago

This is excellent. I've been trying to find a technical explanation of ASCIBOOST for some time. Most of the articles and discussions gloss over the details. It is possible that this is part of the reason some miners are blocking SegWit but there is more to it. Core (the main Bitcoin development team) promised to provide a block size scaling solution and then reneged. This pissed a lot of people off and led to the blocking of SegWit. I don't see ASCIBOOST as a real problem for Bitcoin.

tromp 10y ago
The AsicBoost inventors also wrote a good technical explanation at https://arxiv.org/pdf/1604.00575v1 https://arxiv.org/pdf/1604.00575v1
- jlrubin 10y ago
  Notably, they don't include the optimal root generation algorithm, nor do they discuss segwit incompatibilities. Also, their 20% figure is unclear. The more collisions you can find, the more hash per second you should be able to gain.
  - jcoffland 10y ago
    > The more collisions you can find, the more hash per second you should be able to gain. But the chance of finding three blocks with the necessary 4 byte collision is much more difficult. It's the same problem as finding 3 people with the same birthday. You need 87 people to have a 50/50 chance of three people having the same birthday vs. 23 for two people who share a birthday. That's nearly a 4x increase in required computation for a small increase in effective hash rate.
    - jlrubin 10y ago
      It's not that big of a computation even at 4 collisions, but yes, I'm familiar with the math (I wrote the PDF).

wmf 10y ago

Some context: http://hackingdistributed.com/2017/04/05/bitcoin-drama-response/ http://hackingdistributed.com/2017/04/05/bitcoin-drama-respo...

AgentME 10y ago
I find this article very misleading. >2. If ASICBOOST was actually used, we'd see ample evidence on the blockchain. The covert form of ASICBOOST (where they don't use the version field, the form that was only recently publicly discovered) would only show up as a higher than usual number of empty blocks or blocks with reordered or missing transactions (depending on how the attacker implemented it; it's not necessarily both). >3. ... As you can also examine for yourself, the transactions are ordered essentially by fee-per-byte, which is not what we would see if they were shuffled to create collisions. I find it extremely disingenuous that the author only focuses on looking for reordered transactions and ignores the often-noticed unusual number of empty blocks that Antpool mines[0], and the article reaches absurdity when the author claims that the lack of reordered transactions means there's absolutely nothing to worry about. >4. Building more efficient mining chips is what miners are supposed to do, as it secures the blockchain. Framing an optimization as an attack is disingenuous. Going from 28nm to 16nm was not an attack on the network. Better mining algorithms have never been considered an attack. If for example a mining "optimization" encouraged every miner to mine only empty blocks, then calling it an attack doesn't sound out of place. An optimization that only often encourages miners to make empty blocks isn't categorically different. >7. Segwit is being offered as a solution to ASICBOOST. Yet Segwit has at best a tenuous connection as a response to ASICBOOST mining. There are countless other, non-controversial solutions that can disable ASICBOOST, should one choose (wrongly) to do so. Greg Maxwell's very post announcing the issue introduces a fix for the covert form of ASICBOOST that does not involve implementing SegWit! [0] https://bitcoin.stackexchange.com/questions/50184/why-does-antpool-mine-so-many-empty-blocks https://bitcoin.stackexchange.com/questions/50184/why-does-a... http://www.livebitcoinnews.com/antpool-generating-large-amount-empty-bitcoin-blocks/ http://www.livebitcoinnews.com/antpool-generating-large-amou...
- kylebenzle 10y ago
  Either I am crazy and don't understand bitcoin or some well funded group is spending a lot of money to support SegWit and push Core's agenda. The few sensible posts like yours are downvoted to hell and back while pro-SegWit rocket to the top. Every point you make is spot on and there is no sensible reply to any of them, just a bunch of garbage about no one supports BU and BU will never happen...

needs 10y ago

Just to point out that SegWit is everything but "already adopted in the industry". It turns out that SegWit is the solution promoted by the developers of bitcoin core to allow for bigger blocks, and solve some issues like transactions maleability. It's more than 10 000 lines of code highly controversial because they require... A soft fork, and will change bitcoin in a fundamental ways. Not to add that a company called "Blockstream" have paid almost every developers of bitcoin core to push Segwit without a clear consensus from the community. Worse, this company actually support the huge censorship happening in bitcointalk and r/bitcoin. Blockstream received more than 75 millions from AXA, for obscure reasons. The paper only focus on Segwit, without mentioning that Segwit has nothing to do with ASICBOOST. The community and a growing majority of miners is actually switching to bitcoin unlimited, a fork of bitcoin core. Sorry for the little off topic comment but I love Bitcoin and I hate that a few dishonest people took control of it and try to destroy it.

efesak 10y ago
- there are ~5000 LOC of https://github.com/bitcoin/bitcoin/pull/7910/files https://github.com/bitcoin/bitcoin/pull/7910/files and almost 3/4 of these lines are tests. - Segwit actualy block harmful form of ASICBOOST - by "growing majority" you mean handful of chinese miners and 2% of nodes (http://luke.dashjr.org/programs/bitcoin/files/charts/software.html http://luke.dashjr.org/programs/bitcoin/files/charts/softwar...)
- needs 10y ago
  > - Segwit actualy block harmful form of ASICBOOST It's - of course - not the only solution to this "problem", but somehow this paper doesn't propose anything else other than segwit, and his conclusion is a typical Bitcoin Core propaganda. > by "growing majority" you mean handful of chinese miners and 2% of nodes (http://luke.dashjr.org/programs/bitcoin/files/charts/softwar... http://luke.dashjr.org/programs/bitcoin/files/charts/softwar...) You aren't without knowing that in Bitcoin, miners establish consensus by voting with their hashrate. Nodes count doesn't prove anything because its really easy to host a node. The point is: there is more miners supporting BU than Segwit [1]. [1]: http://xtnodes.com/#bitcoin_classic_blocks http://xtnodes.com/#bitcoin_classic_blocks

fpgaminer 10y ago

Besides the technical issues involved here, there's been a lot of political quarrels as well. Here's my run down of them. Note that I was heavily involved in the Bitcoin community a few years ago, but have been on the side lines recently. As with all things political, take my interpretations with a grain of salt: 1) The Bitcoin network began experiencing congestion due to rise in popularity driving large numbers of transactions. This resulted in slower transaction verification and higher transaction fees. 2) Users started asking for larger blocks, so that the network would have higher transaction bandwidth. The defacto Bitcoin developers at the time pushed back. Increasing block size is simple in terms of code change, but requires a risky hard fork. It's also only a temporary fix. Bitcoin will grow and require another increase, and thus another hard fork. They wanted a better, long term solution. 3) After a lot of discussion the developers came back with Segregated Witness. It's a soft modification to Bitcoin which meant no hard fork. It doesn't explicitly allow larger blocks. It does, however, upgrade Bitcoin to allow side chains. The argument is that faster, more abundant transactions can be implemented in a side chain system. A sort of second layer on top of Bitcoin. Much like how HTTP is implemented on top of the underlying TCP/IP protocol. It should be noted that, while this enablement of side chains is the biggest feature of SegWit, there were a few smaller improvements. It _does_ increase effective block size slightly; ~2x. And it fixes a few minor annoyances in the Bitcoin protocol (e.g. malleability). 4) The community was disappointed by the lack of larger blocks. While developers were busy putting together their idea and specification for SegWit, the congestion issues on the network continued to grow worse. 5) Eventually the debate became "on-chain" versus "off-chain". Some of the community argued that they wanted to keep all transaction on the Bitcoin blockchain. In other words, they still wanted bigger and bigger blocks. I believe the root of this argument is that Bitcoin has served us well for many years, so why try to build another, potentially weaker system on top of it? The other parts of the community were in favor of SegWit and off-chain transactions. Side chains would allow for faster transactions and significantly more of them, and they would consume far less disk space (a growing concern in the community is the disk space required to run a Bitcoin node). Side chains achieve this by compromising on some of the strengths of Bitcoin, either by reducing security or requiring more centralization. 6) Eventually the debate in the community shifted from just on-chain versus off-chain to SegWit versus bigger blocks. Now, to be clear, SegWit doesn't preclude the possibility of larger blocks. Obviously larger blocks can be added regardless of SegWit. But, for whatever reason, the dialog shifted. 7) When SegWit was finally released as part of the Bitcoin Core client (the defacto standard for Bitcoin clients/wallets) a sort of battle amongst the community began. Mining operations that ran the SegWit enabled clients started getting attacked; DDOS, etc. There was a lot of strife, hate, and fear. The way SegWit is implemented in the code, it requires a large majority of miners to support it before it officially activates and can be used. This is by design, to ensure the community agrees with the feature and to ensure almost all miners are able to understand the new blocks and not get left behind after activation. Because of this majority requirement, the battle in the community and between miners was very important. If half the miners didn't agree with SegWit, it would never activate. 8) This "battle" went on for a long time and is, currently, still on-going. Bitcoin continues to remain congested. Transaction fees have continued to rise. SegWit is still not activated. During this time a couple factions of the anti-SegWit community split off in attempts to fork Bitcoin. They modified versions of the Bitcoin client to enable larger blocks. So far, none of those forks have been successful, in terms of taking away any majority of users from the main Bitcoin network. 9) Recently it became public that a large mining operation may have been using a technology called ASICBOOST during this time. It allows mining chips to be more efficient, which means those chips make more Bitcoins for less money. Obviously an advantage. This technology has been known about for awhile now, but as far as the community knew no one was using it. It was covered by patents, and there's a sort of gentleman's agreement in the community not to use it as a result of the patent and potential ill effects on the network. (The patent means not all miners would be able to use it, so it presents an unfair playing field). ASICBOOST, the way this mining operation has supposedly been using it, is not compatible with SegWit. That's explained in the OP. The accused mining operation was also involved in a lot of the anti-SegWit activity in the community; promoting the alternative forks of Bitcoin. They had been accused in the past of using sock puppets to drive anti-SegWit agendas, drive character assassinations, etc. They are in control of a large percentage of the total Bitcoin mining power. The public revelation of them using ASICBOOST painted a dark, but enlightening picture. As I mentioned before, SegWit ultimately has nothing to do with the on-chain versus off-chain debate. It merely _enables_ off-chain possibilities. It doesn't force them. So it was odd to see the discussion shift from on-chain versus off-chain to SegWit versus Bigger Blocks. The theory now, given the accusations against this large mining operation, is that they are responsible for the majority of the anti-SegWit movement. Because the activation of SegWit would force them to stop using ASICBOOST the way they've been using it, it was in their financial interest to prevent SegWit from activating. That's basically my summary of events. I think that theory, that this mining operation has been basically using anti-SegWit propaganda and other attacks to prevent SegWit from activating, so they can keep using their secret version of ASICBOOST and profit (to the tune of $100 million a year), makes a lot of sense. Why else would anyone oppose SegWit? I've read through the SegWit specs. It enables a lot of really cool tech for Bitcoin. Side chains are not just about trying to get more, faster transactions into the Bitcoin network. They're also about allowing other technologies like smart contracts to tie themselves in Bitcoin. You know all those cool features that various alt coins are testing out? SegWit enables those features to become side chains so that you can gain all the security that Bitcoin offers. Side chain versus altcoin is like browser addon versus a whole new browser. At the end of the day, people in favor of bigger blocks could still campaign for bigger blocks. SegWit doesn't prevent that. A mining operation being financially incentivized to block SegWit is the simplest explanation I've heard yet for why there would be significant opposition to it.

RichardHeart 10y ago

Anything that makes mining empty blocks more profitable is bad. Empty blocks are an attack on the network. Someone is going to say "Empty blocks give people a reason to raise fees." DDOS attacks give people reasons to buy more expensive routers too.

tlrobinson 10y ago

This might not be the place for this question, but can someone explain why we can't / shouldn't have both unlimited block sizes and SegWit? Block sizes will supposedly be constrained by bandwidth / propagation times. SegWit will allow more transactions to occur off chain, reducing the need for larger blocks. They seem complementary to me.

Kinnard 10y ago
You are correct and such proposals have been put forward: http://moneyandstate.com/thoughts-on-segwit2mb/ http://moneyandstate.com/thoughts-on-segwit2mb/
stale2002 10y ago
The Core team does not want blocksize increases. They explicitly want prices to be high/number of transactions to be limited (think, supply and demand) . This is because transaction "price" is equivalent to "amount of money spent on securing the blockchain".
- deleted 10y ago
  [deleted]
shea256 10y ago
SegWit does in fact represent a block size increase. It results in 1.7x the # of single-signature transactions per block and 4x the # of multi-signature transactions per block. Unlimited block size? There are many reasons that's a very bad idea but here are a few: 1. It would put immense pressure on the network in terms of latency between miners, leading to less stable mining, a higher rate of reorganizations, and a massive advantage for the larger miners, resulting in increased mining centralization. 2. It would result in a substantial increase in the amount of bandwidth that each node has to handle. Modest increases are OK but unlimited blocks means the vast majority of nodes will die off. Remember that nodes have to check blocks as they come in, so this would be an insane DDoS vector. Imagine putting a funnel in someone's mouth instead of a straw and being able to force whatever you want in there. 3. It would result in a substantial increase in the amount of data that each node has to store. Remember that in Bitcoin 100% of data must be stored by 100% of nodes. Imagine "unlimited emails" with Gmail meant that you had to store all of the emails in the world for EVERYONE. Not a good idea.

swedrupe 10y ago

9 months ago people already claimed on Reddit that Antminer H/W was using AsicBoost. This story seems just being dug up now by GMax for one or more of several reasons: * discredit Jihan / Bitmain / AntPool who support BU * distract from SegWit problems, buy time * solidify the "Chinese miners bad, Core good" narrative to support UASF, SegWit's final death throw * maybe give BitFury and other ASIC manufacturers some more business by scaring customers away from Bitmain In truth, there's no emergency or attack from ASIC manufacturers on Bitcoin, mining is more decentralized than before through many pools. This is about the politics of a small clique in the incumbent dev team who don't want to let go of power, at all costs, nor do they seem to be very worried about what happens to Bitcoin because of their unwillingness to compromise and even to deliver on their agreements.

cashmonkey85 10y ago

I genuinely believe most of the comments on this page are paid trolls. Which I didn't think was a thing on HN People should read up of the Chinese miners disinformation campaign. Really crazy stuff. https://medium.com/@WhalePanda/the-extended-extension-block-story-5bc3d888bdde https://medium.com/@WhalePanda/the-extended-extension-block-...