9 ms·
I've spent a lot of time reviewing the original Bitcoin codebase. It's brilliant code. It's production-grade C++. There's nothing in it that hints at academic
by sillysaurus3 9y ago
I've spent a lot of time reviewing the original Bitcoin codebase.
It's brilliant code. It's production-grade C++. There's nothing in it that hints at academic origins. Most people are either academics or professional coders -- to be both is a rare exception.
The codebase seemed to materialize out of nowhere. One of the earliest commits in the SVN repo contains 36 thousand lines of code. "Satoshi" (or this group of people) must have worked months or a year on this before putting it up on source control.
The code also uses irc to find seed nodes, which is amusing. It just connects to #bitcoin and assumes that some of the people in the channel are running bitcoin nodes. That's a cool way around the "What if all the hardcoded seed nodes fail?" problem. I know it's probably a standard tactic, but bitcoin integrates so many standard tactics so well in addition to its academic work.
Here it is as one gigantic file: https://gist.github.com/anonymous/b4d5d1ab333c5d6e238fdc2242a428b5 https://gist.github.com/anonymous/b4d5d1ab333c5d6e238fdc2242...
- ChrisClark 9y agoJust clicked to check it out and was reminded about how much I like the base-58 scheme they used for addresses.
- sillysaurus3 9y agoThere's so much like that. I'm going to try to write a blog post highlighting all the gems. My favorite is: IMPLEMENT_SERIALIZE ( READWRITE(prevout); READWRITE(scriptSig); READWRITE(nSequence); ) It's a C++ macro that implements reading and writing member variables to/from disk/network. It uses C++ templates to figure out the sizes of everything, so it winds up packing items efficiently. We used something similar in the gamedev industry, so this was a delightful surprise. Ohh, no, I take it back. This is my favorite: // // Compact size // size < 253 -- 1 byte // size <= USHRT_MAX -- 3 bytes (253 + 2 bytes) // size <= UINT_MAX -- 5 bytes (254 + 4 bytes) // size > UINT_MAX -- 9 bytes (255 + 8 bytes) // inline unsigned int GetSizeOfCompactSize(uint64 nSize) { if (nSize < UCHAR_MAX-2) return sizeof(unsigned char); else if (nSize <= USHRT_MAX) return sizeof(unsigned char) + sizeof(unsigned short); else if (nSize <= UINT_MAX) return sizeof(unsigned char) + sizeof(unsigned int); else return sizeof(unsigned char) + sizeof(uint64); } When encoding an arbitrary value, if it's less than 253, it only uses 1 byte of space. When you read a byte off the network, if it's 253, you know there are two more bytes to read. 254 = 4 bytes, 255 = 8 bytes. Again, it's a small thing -- very standard tactic. But there are dozens of tiny, effective decisions exactly like this all throughout the codebase. There's a secure allocator for wiping your private key so that it doesn't hang out in memory, with hacks to work around MSVC8 problems. Which academics bother with MSVC8 hacks? // // Allocator that clears its contents before deletion // template<typename T> struct secure_allocator : public std::allocator<T> { // MSVC8 default copy constructor is broken typedef std::allocator<T> base; typedef typename base::size_type size_type; typedef typename base::difference_type difference_type; typedef typename base::pointer pointer; typedef typename base::const_pointer const_pointer; typedef typename base::reference reference; typedef typename base::const_reference const_reference; typedef typename base::value_type value_type; secure_allocator() throw() {} secure_allocator(const secure_allocator& a) throw() : base(a) {} ~secure_allocator() throw() {} template<typename _Other> struct rebind { typedef secure_allocator<_Other> other; }; void deallocate(T* p, std::size_t n) { if (p != NULL) memset(p, 0, sizeof(T) * n); allocator<T>::deallocate(p, n); } }; Another gem (note the comments): case OP_EQUAL: case OP_EQUALVERIFY: //case OP_NOTEQUAL: // use OP_NUMNOTEQUAL { // (x1 x2 - bool) if (stack.size() < 2) return false; valtype& vch1 = stacktop(-2); valtype& vch2 = stacktop(-1); bool fEqual = (vch1 == vch2); // OP_NOTEQUAL is disabled because it would be too easy to say // something like n != 1 and have some wiseguy pass in 1 with extra // zero bytes after it (numerically, 0x01 == 0x0001 == 0x000001) //if (opcode == OP_NOTEQUAL) // fEqual = !fEqual; stack.pop_back(); stack.pop_back(); stack.push_back(fEqual ? vchTrue : vchFalse); if (opcode == OP_EQUALVERIFY) { if (fEqual) stack.pop_back(); else pc = pend; } } break; Bitcoin is worth N billion dollars right now, and this codebase really is a billion dollar codebase. Is this really the work of a single person?
- userbinator 9y agoThat was probably inspired by ASN.1 or other serialisation protocols. The formatting of the code in that function also puzzles you for an instant, but then you realise just how much clearer it makes it over the "lame" but "consistently formatted" alternative of if ( ) { .... } else if ( ) { ... } ....
- virtuexru 9y agoI really hope you make that post highlighting your favorite parts of the C++ Bitcoin codebase; it is super interesting! Keep up the great work.
- kobeya 9y agoNo those weren't the work of a single person. You're quoting versions of the code base that post date the public introduction of bitcoin. Some of the things you call out were written by others, and others were written in response to public review.
- kobeya 9y agoEven people working on bitcoin aren't fans of base58. Variable length serialization? Bignums in the serializer? Poor error detection? No error correction? Confused characters in alphabet? Case sensitivity? Lots of reasons to dislike.
- gruez 9y ago>Poor error detection? No error correction? It's implemented at a higher level. For bitcoin, a 32 bit check field is appended to the data to serve as a checksum. If you want parity, I'd imagine that could be done as well. >Confused characters in alphabet? Case sensitivity? Some look alike letters are excluded, hence why base58 rather than base62.
- ChrisClark 9y ago> Confused characters in alphabet? That's really the only reason I was saying I liked base58. Because it removes those confusing characters compared to base64 or others.
- kobeya 9y agoAh, but it doesn't do a very good job of that. It only removed a few pairings, and made some odd choices at that. Hence why I list it as one of the issues.
- mrb 9y agoI recovered and archived version 0.1.0 (directly from Hal Finney): http://www.zorinaq.com/pub/bitcoin-0.1.0.tgz http://www.zorinaq.com/pub/bitcoin-0.1.0.tgz It has 19k lines of code. That's certainly a few months of work minimum. Fun fact: I was contacted by the Computer History Museum in Mountain View since this tarball (and the .rar) hosted on my site is the earliest known public copy of the source code.
- sillysaurus3 9y agoNotice how all the files use \r\n for newlines. Satoshi was a Windows programmer! They used Windows. Anyone who has tried to do cross-platform development knows it's extremely difficult in C++. You can do it, but it's not a trivial thing. I guess it's no surprise that Satoshi used Visual Studio. If you start out writing code in not-Visual-Studio, it's far harder to go back and make it run on VS. I vividly remember how maligned Windows programmers were in 2008. I felt like a black sheep getting my start in the gamedev industry, because if you wanted to do Big League Programming in gamedev, that meant one thing: Windows. Microsoft has finally made some inroads in hacker culture. It's far less common now to be dismissed as less talented just because you used Windows. But in 2008 this feeling was very much alive.
- nadapkoe 9y agoMost CIA and NSA code that has been leaked has been developed on Windows using Visual Studio IIRC. The bitcoin code looks and feels a lot like of their projects but could easily be some NSA programmers side project. Any hacker can tell this was a one person project from looking at the code.
- mike_hearn 9y agoBitcoin was originally a Windows only program. We used Wine to run it on Mac/Linux in the early days: https://sourceforge.net/p/bitcoin/mailman/message/23827020/ https://sourceforge.net/p/bitcoin/mailman/message/23827020/ I used to be a Wine developer so it didn't put me off when I first found the project, but I'm sure the fact that it was Windows only hurt the projects uptake in the first year. It wasn't until Bitcoin 0.2 that a Linux version was available. If it helps, Satoshi was also an Outlook user ;)
- cs702 9y agoWhich is why I wouldn't be surprised if "Satoshi Nakamoto" turns out to be, not an individual, but a tightly-knit team of people, including BOTH professional coders who write software for a living AND computer scientists with strong academic backgrounds. Bitcoin is in my view too impressive an achievement to be the work of a single person working in isolation.
- busterarm 9y ago> too impressive an achievement to be the work of a single person working in isolation. It's been done before... for example, djb
- ghostbrainalpha 9y agoCould you explain what his accomplishment was in terms a non-cryptographer could understand?
- btilly 9y agoFirst of all djb is https://en.wikipedia.org/wiki/Daniel_J._Bernstein https://en.wikipedia.org/wiki/Daniel_J._Bernstein and the accomplishment was releasing a secure email server (qmail), a secure webserver (publicfile), and a secure BIND server (djbdns). Both qmail and djbdns became widely used, and the main alternatives were projects supported by large teams. It took 8 years before the first bug was found in these projects. I highly recommend reading through the guidelines in https://cr.yp.to/qmail/guarantee.html https://cr.yp.to/qmail/guarantee.html for how to write secure software. In particular point 5, Don't Parse, is broadly applicable and under appreciated. A well-known class of bugs arising from this error are SQL injection attacks.
- jameskegel 9y agoWow, this was a path I did not expect this conversation to take. Thank you for mentioning this.
- kobeya 9y agoOr Linus and Linux?
- pash 9y ago> "Satoshi" (or this group of people) must have worked months or a year on this before putting it up on source control. In one [0] of his emails to Mike Hearn [1] Satoshi Nakomoto wrote that he worked on Bitcoin for two years before releasing the paper and code: > I must admit, this project was 2 years of development before release, and I could only spend so much time on each of the many issues. His emails to Hearn and others make clear that his creative process consisted mainly of writing code until he was convinced things worked. In my opinion, Satoshi Nakomoto was a lone hacker. 0. https://pastebin.com/wA9Jn100 https://pastebin.com/wA9Jn100 1. https://bitcointalk.org/index.php?topic=2080206.0 https://bitcointalk.org/index.php?topic=2080206.0
- FooHentai 9y agoAlso, an unreliable narrator.
- FooHentai 9y agoHmm, I think this comment was taken little more negatively than I had intended. My fault of course for not expanding on the point, which is this: Satoshi maintains anonymity, by choice. Therefore, any information about Satoshi's identity, directly stated or inferred from things said by them, logically have to be treated as unreliable. With the goal of remaining anonymous, Satoshi has nothing to gain and a lot to lose from providing true information about themselves. It is in their interest to not provide any information about themselves, and for any information that is provided them be inaccurate. None of this casts any aspersions on either Bitcoin or Satoshi. Ah heck, I sound like Vizzini. Maybe I should just drink the damn wine ;)
- keypusher 9y ago> It'd be worth implementing some kind of more robust auto update mechanism, or a schedule for the phase in of this [the block size], if only because when people evaluate "is BitCoin worth my time and effort" a solid plan for scaling up is good to have written down. Prescient words from Mike Hearn.
- 9y ago
- ringaroundthetx 9y agoMonero's codebase uses an array of subdomains which return a list of other seed nodes in a GET response, and then the codebase appends the appropriate port number onto those All the cryptocurrencies built on top of techniques that worked on other cryptocurrencies. I really like the iteration speed as well as how non-finite any limitation today is.
- arca_vorago 9y agoSince you seem familiar with Monero, I would like to ask what you think of it? I ask from the perspective of one who thinks the anonymous function of cash is a key factor bitcoin doesn't provide, and I've heard monero is one of a few options that does.
- jameskegel 9y agoMonero is the -only- option that offers fungible currency. Monero Research Lab is actively employing full time PHD researchers to asses and improve where needed the strength of the underlying crypto and the implementation thereof. Zcash is not an acceptable substitute, as is the common opinion- I mention this only because you say "monero is on of a few options".
- Ar-Curunir 9y agoZerocash was developed by (and is still being improved by) full time academics.
- ringaroundthetx 9y agoWhich is probably why they have no idea that the user experience is irreparably broken to ensure privacy. The same criticisms exist as since Zerocash was just a white paper. I will say that there is the possibility that they can make the optional opaque transactions the more ready option, but you can look at the blockchain and see that nobody uses those addresses. Zerocash is just a bitcoin clone if you don't use the opaque addresses.
- feelix 9y agoI'm curious as to why you consider brilliant, production ready code. I also read through the original codebase and I came to a different conclusion. To me it looks like something that was hacked at for a year or so, and was all just glued together. It looked like the first iteration of a codebase, the thing that you usually don't release, but then re-work so it is presentable to the outside world and then release. To me that says a couple of things. One is that Satoshi is one, normal, human. Another is that he is very human. For example, a lot of the OP codes have had to be disabled because they were totally insecure. In early versions of bitcoin there were bugs that allowed anyone to spend anyone's bitcoins, and so forth. Not that I mean to attack Satoshi, he was a good coder who started what could become a revolution. But he was a human like the rest of us, and it shows. What is it about his code that makes you consider it brilliant and production ready?
- keymone 9y agoin a codebase of tens of thousands of non trivial C++ after 8 years there was found one critical protocol bug. one. if this doesn't blow your mind - i don't know what will.
- deleted 9y ago[deleted]
- mike_hearn 9y agoThere was more than one critical protocol bug. But yes, it was an impressive effort. It used the STL well, was tightly written, and of the security bugs that did exist none were buffer overflows, stack smashes or double frees from what I recall. There were some issues in the interpreter opcodes but I think they may have been crashes (DoS). The code was clearly the work of one person though. There were not enough comments or other forms of code documentation for it to have been a team effort, even if the Satoshi personality was a composite. The idiosyncratic code is another hint: Windows only and Hungarian notation in 2008 is not something commonly seen. The early Bitcoin code gave me the feeling of an experienced developer who learned their craft in the 1990s and had probably been around for quite a long time, but who probably hadn't worked in large professional development teams in recent years. For example, there were no unit tests. Common in the 1990s, not so much in 2008.
- nullnilvoid 9y agoInteresting code. Some first impressions. 1. This gigantic file can be split into multiple smaller files. It takes even a while for my browser to render the whole source file. 2. There is inline assembly in the code. Is this for performance or some other purposes?
- csomar 9y agoThe code is in smaller files but that link was for demonstration purposes I guess. Here is the most updated bitcoin repo: https://github.com/bitcoin/bitcoin https://github.com/bitcoin/bitcoin
- csomar 9y agoMy impression being in the bitcoin community and talking to some core dev is the opposite. The code was rushed and rather amateurish that there were bugs professional developers should not make.
- xorcist 9y agoIt's worth repeating: This is a C++ codebase. It listens to open ports on the public Internet. One single remote exploit and you lose all your money. The author basically threw code over the wall and the open source community where contributors come and go all the time took over. And one single remote exploit is all it takes. (This causation is perhaps less true today when it is more common to use encrypted or even hardware wallets, but before that everyone just used the standard wallet.) Yet none of this has happened. The odds of this seems vanishingly unlikely. Then there's the risk of consensus problems that would enable double spending, which is very difficult to test for. At the same time original Bitcoin was far from perfect. Someone wrote up a summary of important changes Hal Finney did which I can't seem to find. He pointed out a lot of problems which would have made Bitcoin not work at all which resulted in some early redesigns and the removal of many opcodes. Parts of Bitcoin also went nowhere, notably the marketplace, pay-to-IP and payment channels. The ideas live on as Openbazaar and Lightning but completely redesigned from the Satoshi origins. In so many ways it is an enigma.
- seanwilson 9y agoI'm super curious why C++ was chosen when the risks are so high. Performance?
- Normal_gaussian 9y agoLikely experience. Most would write something so large in a language they are comfortable in.
- seanwilson 9y agoThat's a really bad reason for something you'll be working on for a few years where a single bad bug could ruin it when it goes live. You'd think picking a language where buffer overflows and memory management bugs were impossible would be a minimum requirement. If you're good at C++ most other languages are a breeze in comparison as well.
- wslh 9y ago