10 ms·
Bitcoin From Scratch – Part 1
- nemo1618 6y ago> The PoW is suffice if the hash begins with a certain number of 0s This is a common explanation of PoW, but is actually incorrect. If you think about it, this would mean that the PoW difficulty could only increase (or decrease) by a factor of two. In reality, the block hash is simply interpreted as a (very large) number, and this number must be less than some other very large number (the "target"). So you do end up with a lot of leading zeros -- but these are just a side effect, not the thing being measured.
- sodascripts 6y agoTIL. I wonder why that myth is so commonly perpetuated.
- Jasper_ 6y agoIt used to work this way, but got changed to a less-than check for performance reasons. The Bitcoin whitepaper [0] even has the original approach: > The proof-of-work involves scanning for a value that when hashed, such as with SHA-256, the hash begins with a number of zero bits. [0] https://bitcoin.org/bitcoin.pdf https://bitcoin.org/bitcoin.pdf
- sodascripts 6y agoGotcha, thanks for the explanation.
- nemo1618 6y ago> but got changed to a less-than check for performance reasons. Performance as in "it takes less time to verify the PoW," or something else? Because the "leading zeros" and "less than target" approaches would both take a negligible amount of time.
- Jasper_ 6y agoThe story I was told from Bitcoin developers was that it was done for code performance reasons. However, since the less-than check goes back all the way to the code's "Initial commit" [0], I'm not sure if that's fully accurate -- it might have been based on correspondence from the original author. At the very least, the misconception comes from Bitcoin's own whitepaper. [0] https://github.com/bitcoin/bitcoin/blob/e071a3f6c06f41068ad17134189a4ac3073ef76b/main.cpp#L1179-L1181 https://github.com/bitcoin/bitcoin/blob/e071a3f6c06f41068ad1...
- nemo1618 6y agoWeird. That doesn't make any sense to me. btw, the whitepaper probably used "number of leading zeros" because that's what Hashcash does.
- monokh 6y agoThe article mentions the hash is required to begin with a certain number of 0s in hex digits (half byte) whereas the Bitcoin whitepaper refers to it as number of bits. I'm unsure about the comparison (== or >) but the whitepaper seems accurate with regards to the target.
- nullc 6y ago> It used to work this way, but got changed to a less-than check for performance reasons. I don't know who told you that, but they were weirdly confused. It not unlikely the case that sometime early in development (long before publication) that it was bits-based, not only is this how the standard hashcash code works-- but the Bitcoin code calls the relevant field that encods the difficulty "bits". But Bitcoin itself never worked that way, not from the first instant. Changing it would have been a consensus incompatible change, and Bitcoin is still more or less consensus compatible all the way back. (There are outright bugs that make the old software get stuck, but if you fix those, it follows along).
- saurik 6y agoI did not read the sentence you are responding to as implying it was changed after final network launch, only that it was changed.
- nullc 6y agoWell there is no evidence of that other than "bits" and that's how hashcash worked. Sounds like an invitation on the meaning of the word "changed". No bitcoin software was ever released that worked another way.
- joosters 6y agoI don't know who told you that, but they were weirdly confused. The bitcoin whitepaper told them that, in more than one place too, so it's not a simple typo, e.g. ...we implement the proof-of-work by incrementing a nonce in the block until a value is found that gives the block's hash the required zero bits.
- garmaine 6y agoBitcoin never worked that way. However the original hashcash proposal was specified as leading zeroes. I presume that’s what you are mixing up with.
- jbverschoor 6y agoThere’s a difference between mining bitcoins, and mining for network fees.
- trickstra 6y agoWhat difference would that be? Both the fees and the block reward go to the miner who first published a valid block with hash lower than the current difficulty. The only difference is where those bitcoins came from, but that has no impact on how easy it was to mine.
- exdsq 6y agoYeah as a miner you add the fee and the earned coins to the single coinbase transaction, so it’s a single mining action to get both.
- runeks 6y agoIt’s not a myth. It’s a useful simplification.
- tromp 6y agoIt's also accurate if you say that hash value H has -log_2 (H / 2^256) = 256 - log_2 H leading 0s and allow for fractional number of leading 0s. The nBits header field closely resembles a 32-bit floating point representation of this number (I say resembles because it's really a floating point representation of the threshold H value).
- monokh 6y ago(author here) Absolutely. Thanks for pointing this out. If difficulty adjusted in factors of two, the ability of the network to accurately maintain its 10 mins blocktime become impaired. I try to keep these concepts rather simplified to ease the reading and learning process. Perhaps there should be a note for these situations.
- sowbug 6y agoIn defense of the explanation, sometimes it's asking for an unnecessary mental leap of the reader to think of a bit array as a number, especially if the reader is not accustomed to thinking in base-2 or base-16 number systems.
- nullc 6y agoThe hash is just a number regardless-- surely no one has any problem understanding any other number in the protocol as a number! :)
- bouncycastle 6y agoEven this sentence is just a number.
- deleted 6y ago[deleted]
- yaktubi 6y agoReally it’s not because it’s a sequence of glyphs first and foremost. It can be mapped to a number though.
- bouncycastle 6y agoNope, no mapping necessary. 36 octets, so a single 288 bit number in base 2. (Big-endian, unsigned, I guess) Then convert to any base you want. (I should have probably used 32 octets so that it fits neatly in 256 bits)
- SilasX 6y agoThat doesn't seem like a simplification that misrepresents a relevant, core dynamic of Bitcoin. The exact details of how you ramp up difficulty are a pretty low-level technical issue.
- martinko 6y agoHow is network consensus achieved on the difficulty adjustment?
- johnpaulkiser 6y agoEach node independently comes to the new difficulty on their own in an entirely deterministic way. After 2 weeks there should be 2016 blocks mined, 1 block every 10 minutes. After each 2 week period every node recalculates the new difficulty based on the actual number of blocks mined in the last 2 weeks. If the actual number of blocks produced was 25% higher than the target of 2016, then simply increase the difficulty by 25%.
- martinko 6y agoThanks. Given that this relies on subjective timestamps, is this really entirely deterministic? I can imagine that one node counted 2014 blocks in the past two weeks, another 2015. What happens in such cases?
- tromp 6y agoDifficulty only changes at heights that are a multiple of 2016.
- deleted 6y ago[deleted]
- tromp 6y agoIt is computed using only information embedded in the block chain, specifically the timestamps of blocks 2016 * i and 2016 * (i+1).
- martinko 6y agoBut aren't blockchain timestamps unreliable? It was my understanding that difficulty was based on subjective node timestamps..
- WhoIsSatoshi 6y agoDo you have a source I can look at or do I have to go look at at or can you point to code by chance? I'm quite surprised I wasn't aware of this if this is actually how it is implemented, the leading zeroes seemed right (and the difficulty by factor of 2 also!) Thanks!
- nemo1618 6y agoSomeone else linked it below: https://github.com/bitcoin/bitcoin/blob/e071a3f6c06f41068ad17134189a4ac3073ef76b/main.cpp#L1179-L1181 https://github.com/bitcoin/bitcoin/blob/e071a3f6c06f41068ad1... Equivalent check in btcd, an alternative Bitcoin implementation: https://github.com/btcsuite/btcd/blob/991d32e72fe84d5fbf9c47cd604d793a0cd3a072/blockchain/validate.go#L329 https://github.com/btcsuite/btcd/blob/991d32e72fe84d5fbf9c47...
- ozankabak 6y agoClear, concise and to the point. Looking forward to reading the rest. If you finish the whole series and fix/improve the posts in this sequence using the feedback from here, I think it might serve as one of the go-to pages for technically inclined people who are interested to learn about Bitcoin and blockchains.
- person_of_color 6y agoExcept they’re polarising as hell. Expect someone to chime in and state how they are a complete waste of time and energy.
- gjs278 6y agothose people are just mad that they are poor, disregard
- nurettin 6y agoOf course people who try to profit from alternative currencies are pure cancer to this planet, but that doesn't diminish the value or interestingness of applied cryptography or consensus algorithms.
- dang 6y agoThat's generally true. The trouble is that your post more or less does to the thread what you're predicting that people will do to the thread—the polarity of the high-order bit doesn't matter much. It's best to abstain. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- obilgic 6y agoHere is a simple bitcoin implementation in ruby including the networking for interested, that I did some time ago: https://github.com/oguzbilgic/zincir https://github.com/oguzbilgic/zincir
- cchance 6y agoPlease finish this series! Especially the networking and consensus models it's an interesting read, really like that it's in Rust as I'm also learning Rust so it's nice to see the concepts in that language :)
- aciswhat 6y agohttps://github.com/lhartikk/naivechain https://github.com/lhartikk/naivechain is a rather decent introductory implementation as well.
- ggrrhh_ta 6y agoFor people that need or want to know (really understand) in depth about Bitcoin without any previous background in Computer Science (for example, in Finance, investors, journalists, historians, curious people in general) I have to recommend Ted Nelson's "How Bitcoin actually works" (2014) from his series "Computers for Cynics" [0]. I enjoyed it. (+) [0] https://youtu.be/3CMucDjJQ4E https://youtu.be/3CMucDjJQ4E (+) For the people with background in CS: it might be that the video at some point talks about the number of zeros instead of the comparison, but really, it does not detract a 'bit' from its value. edit: formatting
- leorio 6y agothat was entertaining
- tlrobinson 6y agoI wrote a toy (but working) implementation of the Bitcoin algorithm here: https://github.com/tlrobinson/tomcoin https://github.com/tlrobinson/tomcoin The main logic is a few hundred lines of code: https://github.com/tlrobinson/tomcoin/blob/master/src/node.js https://github.com/tlrobinson/tomcoin/blob/master/src/node.j... A somewhat unique aspect is since it's written in JavaScript a node can run on the server or the browser, which makes for easy demos: 1. Open https://tomcoin.herokuapp.com/ https://tomcoin.herokuapp.com/ and https://tomcoin1.herokuapp.com/ https://tomcoin1.herokuapp.com/ in two browser windows (the private key is stored in localStorage so using different domains ensures each gets a different wallet) 2. Click "Start Mining" on one or both nodes 3. Once you've mined some TomCoin (should just take a few seconds unless more people start mining) copy the public key from one node to the other's "publicKey" field, enter an amount, and click "Transfer" 4. Wait for another block to be mined and you should see the balances transfer (There's no persistence so the chain will reset when all the Heroku nodes idle out and other nodes close) (It also just uses WebSockets to connect to the instances on Heroku, or in theory other server instances people are running. I meant to implement WebRTC for P2P between browser nodes but never got around to it)
- TekMol 6y agoconst { mine, sign, verify, hash, validateProofOfWork, stripTxSignatures, generatePrivateKey, getPublicKey } = require("./util"); What does this do? And will this really run in a browser? I think brosers don't have require()?
- robjan 6y agoYou can see what your functions do by opening the file called util.js in the same directory. And, yes you can run this in the browser by either polyfilling "require" or transpiling it
- TekMol 6y agoBy that logic, everything runs in a browser. The principle of computational equivalence tells us that one turing complete language can do what any other can. So yes, everything can be polyfilled or transpiled. But that does not really meet my understanding of "Runs in the browser".
- mmahemoff 6y agoThanks for this info Monokh. Just want to raise a minor typo, which is the first line of code in "Signed transactions". I think it's missing two of the map keys.
- shuringai 6y agoI really like the intention and good will behind the post but almost every part is incorrect (even on an abstract level not restricting to bitcoin). I feel like OP is yet another someone who needed the keyword "blockchain expert" on his/her linkedin profile and trying to justify it now with misleading and incomplete information.
- martinko 6y agoCould you give some examples of what is incorrect?
- tluyben2 6y agoThis is HN; you are supposed to point out (like some people have in this thread about the leading 000 count) the incorrect parts, providing some links or explanation why they are incorrect. Many people probably clicked here to learn something and so it's helpful to point out what they should be learning instead of this if it is 'incorrect'.
- monokh 6y ago(author here) I'd love to know the inaccuracies so I can address it. Generally I tried to simplify things to make it easy to follow in code. Anything worth noting can be added and I've done this with some things brought up in this thread. I'd love to do the same with your suggestions. I already qualify for the "blockchain expert" tag by working in the space for 2 years, though I don't find it that desirable. Maybe a few years ago.
- deleted 6y ago[deleted]
- deevolution 6y agoBookmarking this one.
- rammy1234 6y agoMining part is confusing for me. Is POW is essentially to achieve difficulty level ? You do in this while loop until you have achieved this difficulty level ? Is this the primary factor that differentiates having various protocol in crypto ?
- jstanley 6y agoYes. Proof of Work is required so that there is a global limit on the rate at which new coins are created.
- monokh 6y agoIt also: - Acts as the primary mechanism to make the data in the chain immutable, as the cumulation of the work (chain of valid blocks) becomes harder and harder to reproduce. - Anti spam, i.e. you need to put something at stake (electricity and hardware cost) to participate in the issuance and transaction ordering
- Obi_Juan_Kenobi 6y agoThere are different ways to think about it, but I find this the most illustrative. Mining is about authorship of the blockchain. The system is simple enough; people broadcast transactions, and they get collected and entered into a shared ledger. But the details are tricky. Order is one major issue; does A or B come first? A and B may be mutually exclusive, so whichever is first is critical. Because the network is imperfect, and network users may be malicious, it is very important that we come up with an extremely robust way of deciding how transactions are entered into the blockchain. Authorship - that is, mining a block - is everything. The simple solution is to make it random. If it's random, then no individual miner can reliably control authorship, and thus cannot engage in attacks like double spends (unless they control a majority of the mining power, i.e. a 51% attack). POW is a way to achieve randomness in a robust way. The only way to mine a block is to solve a problem that can only be solved via brute-force hashing. This is costly to do, so incentives are offered to get people to do it via coinbase rewards and transaction fees. This adds to the security model because miners now have strong economic ties to the POW algorithm and blockchain with capital and marginal costs. Of course, there are other ways to achieve randomness, but can you think of one that will force a decentralized network to achieve robust consensus (i.e. to not fork)? Much of the 'magic' of Bitcoin comes when you realize the incentives under which miners operate. Once a valid block is broadcast on the network, miners have extremely powerful economic incentives to accept this block and begin mining on top of it. Consider what happens if, instead, the miner continues to mine at the 'old' block height. They may come up with a valid block fairly soon (it's random, so they have no way of knowing), but each moment that passes, the other miner's valid block is being transmitted over the network. Any subsequently transmitted valid solution is at a disadvantage in becoming the consensus, and that grows as time passes. Basically, any time spent mining on the old block once a valid one has been seen is pure waste. Since the entire network operates from these same incentives, consensus grows very strongly as no one wants to mine on a minority chain. _____ Difficulty is a different issue; this is just a way to keep block times fairly consistent, even as the network grows and more mining power comes online. Because it is calculated directly from the block chain, difficulty rules follow the same strong incentives that build consensus on the majority chain.
- fogetti 6y ago> the transaction needs to be "Signed" by the `from` public key Is that so? The code actually signs with the private key. Although I might have misunderstood what that sentence refer to.
- monokh 6y agoThanks, that was a mistake. I've corrected it.
- ur-whale 6y agoLove this way of learning about tech. : build one yourself (the truly noble way would be to try and build without having seen the specs, just from a description of what it does, but that'd take ages). However: when it comes to implementations, the real hard part of the whole Bitcoin affair is where the gnarly details hide, namely consensu, and it is missing at this point.
- Koshkin 6y agoIt would be nice to be told what programming language this page is using.