15 ms·

Nobody's just reading your code

352 points by chauhankiran 9y ago

bsimpson 9y ago
One of the things that sets a good programmer apart is the willingness to fearlessly dig into someone else's code. I've heard it said before that you're responsible for every line of code you ship to your users. It follows that you shouldn't treat your dependencies as black boxes. Getting a stack trace that's 3 or 4 levels deep in Django/React/<insert library here>? Dig under the covers and understand what's happening. Include that information in a bug report, and better yet, open a PR offering to fix it. You'll learn a whole lot and be more effective if you let your curiosity expand beyond code you/your team has written.
- jotto 9y ago
  No mention of time in the article or in your comment. With infinite time we could fix a lot of things.
  - ef4 9y ago
    Sure, but I think many programmers underestimate the investment value of that time. Diving into codebases is a skill that gets stronger with use, such that you can eventually do it radically faster. That makes a much larger set of problems economically practical to fix.
  - bsimpson 9y ago
    Of course you can expand this to be unreasonable - you could also make a strawman argument about how you need to understand all the compilers, OS APIs, and chip architectures that support your product. Obviously, that's not what I'm saying. I'm making a few assumptions: the code you're depending on was written by people like you (e.g. other open source volunteers with expertise in the same language as you); the limitation you're encountering is problematic for your users, product, or business; and that you'd rather have it work correctly than design around it (e.g. remove the affected feature). If those are all true, yeah, it's probably worth a week or two to figure out what's wrong and fix it (especially since you'll gain comfort, familiarity, and efficiency in that codebase over time - it might even inspire other enhancements). I realize that nobody has infinite time and I trust you to make reasonable decisions about what to spend where. Still, I implore you to not let yourselves be afraid of Other People's Code. It's probably not as scary as you think it is; after all, it was written by other people like you.
    - copperx 9y ago
      > It's probably not as scary as you think it is; after all, it was written by other people like you. Most interesting code has been written by people smarter than me (the kernel, compositing window manager, network stack, web rendering engine, virtual machine, etc.), and while reading that code probable isn't intractable, it would take me decades of effort to understand. At the same time, I think I can be an effective developer by building on top of those technologies (i.e., being a great plumber). And that's OK.
      - deleted 9y ago
        [deleted]
    - astura 9y ago
      >it's probably worth a week or two to figure out what's wrong and fix it Ha! I'm convinced that people who say stuff like this haven't worked where their paycheck comes from actual paying customers, as opposed to VC dollars or subsidies from other parts of the business's revenue. > Still, I implore you to not let yourselves be afraid of Other People's Code. Wtf? Nobody said they were and what is with the Condescending Proper Noun? >It's probably not as scary as you think it is Nobody said it was.
- panic 9y ago
  That's one thing that bothers me about modern Electron-style apps which depend on an entire web rendering engine -- actually digging through all the dependencies to track down a bug can be almost impossible. For example, I was hitting a bug in an Electron app a while back and wanted to track it down. I got the app building with a locally-built version of Electron, but it became clear the real bug involved Chromium's loader code. My poor 4-year-old laptop ran out of disk space before I could even download the whole libchromiumcontent repository, let alone build it!
  - hinkley 9y ago
    I don’t think Library designers have adapted yet to a library heavy world. We still write them like we are using dozens but we have hundreds. Every library demands a fraction of our attention greater than its relative fraction of the code. Complex calling conventions, deep call stacks. Why am I even using a single purpose library with multiple levels of abstraction in it? At this point a library should BE the abstraction, not contain them. I have to trace code through my code into yours into your dependencies. Just stop already. Keep It Simple Stupid.
    - gaius 9y ago
      I don’t think Library designers have adapted yet to a library heavy world. We still write them like we are using dozens but we have hundreds. The other day I was debugging an app that - no joke - pulled in 600M of libraries that were all so it could use one file parsing function that could have been re-implemented in <100 lines. Crazy!
- curtis 9y ago
  > ... willingness to fearlessly dig into someone else's code. I've done this many times. The problem is that it takes a lot of time, and there's no way you can dig through more than a fraction of a large codebase. You gotta pick your battles.
  - nathanaldensr 9y ago
    Pragmatism wins the day, for sure. I'll happily read someone else's code (e.g., an open-source GitHub repository I rely on) if it's necessary to troubleshoot issues or implement a feature. There needs to be a reason (i.e., motivation) to read code.
    - andrea_sdl 9y ago
      This is the only reason (at least for now) that makes me dig into a library code. And I always got some benefit from it. I think we often treat libraries as voodoo blackboxes, but they're more often thatn not created by normal people with great skills. Sometimes it's a great way to learn a way to code, or even better, to understand the behaviour of a library. I found myself doing it more often when the IDE itself downloaded and linked the sources. (Lazy, I know).
      - TeMPOraL 9y ago
        > I found myself doing it more often when the IDE itself downloaded and linked the sources. (Lazy, I know). Same here. It's about trivial inconveniences. I sometimes can be bothered to download and read through the codebase of a dependency, but most of the times I end up in third-party code happen when it's just a single jump-to-definition keypress away - the same keypress (M-.) which I use all the time for my code. That seamless browsing really blurs the difference.
      - WillReplyfFood 9y ago
        The blackbox vodoo view- sometimes is just a excuse for lazy evaluation. One can profile a black box- but many fear the moment they discover that there beloved "I know all the api calls from memory alone" library shows bad perfRomance.
- tobr 9y ago
  I always get an uneasy feeling when this is not (practically) possible. Whenever I add a dependency to a project I will at least skim through the source to get an idea of what it is I’m adding, and it pleases me a lot when a library is well written and self contained enough that this does not mostly leave me confused or overwhelmed. There’s a similar reason why I have such mixed feelings about some build tools and compilers, especially in the JS world - the output is often hideous and so unlike what a human might write. I know you can say that what the output looks like doesn’t really matter, as long as it works as it should. But it just makes it so much harder to feel confident that it actually does.
  - icebraining 9y ago
    The output of gcc and clang probably don't look much like hand-written machine code either, though. Or, for that matter, the output of V8 and SpiderMonkey. You can't avoid the compilers :)
    - tobr 9y ago
      Of course! I’m not saying it’s rational. If I had experience hand-writing machine code maybe I would feel the same about that...
      - khedoros1 9y ago
        I've done bits of assembly, but I've also studied compilers. They felt less magical to me when I learned some of how they work. Then again, I'm primarily a C++ developer, so I'm used to the output of my build tools being non-trivial to understand.
    - deleted 9y ago
      [deleted]
  - akvadrako 9y ago
    That's one reason rollup is better than webpack; the output looks almost hand rolled.
  - 9y ago
- taeric 9y ago
  Don't let survivorship and confirmation bias cloud your judgment. What sets many successful people apart is not any one thing, but that they were successful. Many are willing to dive into other people's code in methods that are far deeper than I can comprehend, and yet they have failed at whatever task they were going about. So, yes. By and large, you shouldn't stop at your dependency boundaries. However, your job is to get results. For most of us, even if the dependency is bad, you are better off changing your code. Why? Because you can more quickly get a change in your codebase than you can in the dependency. And heaven help you if you think it is worth forking. This is not to say don't update the base place. By all means, please do. But don't wait for that change to land before you do something on your end, too. Even if that means not using your actual fix.
  - icebraining 9y ago
    Forking is not that bad, unless upstream has a lot of activity yet takes a long time to accept patches. Many projects are mostly dormant, so keeping a patchset and occasionally rebasing is easy enough. We use git-aggregator[1] for easily applying pending PRs/MRs over the upstream branch, and for the most part, conflicts are rare. [1] https://pypi.python.org/pypi/git-aggregator https://pypi.python.org/pypi/git-aggregator
    - heavenlyblue 9y ago
      The thing I find with python/ruby ecosystems - is that the versions are generally pinned, so if you do fork a change for your specific need; you'll end up using it for the project. If the change isn't merged - then at least you may use it yourself.
      - taeric 9y ago
        These are both valid choices. I had not meant it to be that it is not possible. However, in both cases you now have a new line of work. It is common in my experience to find folks are behind in their first job, so adding a new one is dangerous.
  - 9y ago
- deleted 9y ago
  [deleted]
- natecavanaugh 9y ago
  I can only say Amen to this. If you ship it, you either accept responsibility as far as you can, or you're "that" person who redirects you another department because it's not technically their job. Plus, as you mentioned the opportunity for learning is immense.
- segphault 9y ago
  > I've heard it said before that you're responsible for every line of code you ship to your users. This is a great way of thinking about it. I wish that more people took it to heart when choosing third-party dependencies to adopt, especially in the Node.js ecosystem where people are tragically cavalier about adding literally hundreds of under-scrutinized transitive dependencies to their projects.
- geezerjay 9y ago
  I agree with the spirit but in practice more often than not that's a complete waste of time and energy, unless you're trying to figure out the cause of a specific bug when you've already excluded your code as its source.
- lawn 9y ago
  To be sure. There's also a trade off to doing it too much as the time it can take to really go deep can effectively paralyze the development. You need to be able to properly use abstraction layers and be able to treat code as black boxes when needed, you only need sufficient knowledge to work with a system. Of course we shouldn't stop until we reach that level and too often developers give up. But it's also common, especially for very skilled or perfectionist developers, to go too far.
- watwut 9y ago
  Really good programmer is imo set apart by ability to decide when to dig into others code and when not.
- chiefalchemist 9y ago
  "...open a PR offering to fix it..." Perhaps I've been asking to wrong people (at the wrong time) - forgive me - but (as dar as I can tell) git has no easy / reasonable way to repo'tize a (vendor/) dependency. That means you have to bend over backwards (read: make too much extra effort) to patch such a library local and then eventually PR that change. This feels like a fairly significant flaw in the current OSS "ecosystem." It encourages the wrong behaviour, and discourages the right one. Or is there something I'm just not understanding?
  - andrewaylett 9y ago
    It's not really so much that _git_ is lacking the functionality -- it's actually much more possible to do reasonable vendoring with git than with (say) subversion, as you can use submodules to point at your fork. To my mind the issue is more with tooling: you don't typically want to vendor your code, most of the time you want to use a published module. And the process for publishing your fork is too complex to be worthwhile. Also, the work to keep the fork itself up-to-date when you realise it's necessary is minimal, but by switching away from the original you lose tooling support for telling you when you need to make that update.
    - chiefalchemist 9y ago
      I understand there are workarounds, but they are just that. I mean, for example, I'm working away but hit a bug in a dependency. Now i have to fork, submodule, etc. I'm not expecting frictionless per se. But it just feels like there's more overhead than there needs to be. Also, it's possible (in rare cases) the vendor folder is not .gitignore'd. Now what? Submodules is out. Again, given the nature of most modern dev (i.e., dependencies) vit feels underprepared. Re: "...but by switching away from the original you lose tooling support for telling you when you need to make that update..." Yes. But doesn't this actually support the idea (mentioned above) not to treat your dependencies like black boxes? That is, no pain no gain? Yeah, perhaps some sort of simple notification your fork needs an update. Just the same, ALL code should, in theory, be treated as your own. You wouldn't blindly merge/commit a colleague's work, so why shouldn't your dependencies be semi thoroughly reviewed as well? Are we too busy to do the right thing(s)?
- giancarlostoro 9y ago
  I've only found time to do this with CherryPy really, mostly cause every time someone asks for help they say if something is missing in the documentation you could always read the source which is also commented and readable. It really is approachable and not too big. I never have time to go through the source of every single library but you raise a good time to do so: stack traces. Although when doing front-end JS code it's a bit less approachable if your files are already minified (client hands source like this) and the error becomes a lot less useful.
- Daycrawler 9y ago
  I just write that the library's function at hand is unreliable and can't be used this way.
- Cthulhu_ 9y ago
  It's why (in theory) I love Go; as opposed to the JS hell as mentioned by another commenter, it has a lot more emphasis on consistency and, in a way, predictability, so there's less surprises and distractions (e.g. code style) when you browse other people's code. Also no "black box" dependencies (usually), you pull every lib from source and have it on your machine.
- noway421 9y ago
  Isn't it the purpose of blackboxes, to abstract me from doing this? I'm here to get the work done and hopefully having to debug only code of my colleagues, why would I want to waste time on debugging something that should have been battle tested by myriads of people before me? Argument like this almost justifies leaky abstractions as something to be praised.
  - amazingman 9y ago
    Isn’t the purpose of open source also to empower you, the developer using the library, to fix the bug you find so that everyone may benefit?
- bitL 9y ago
  > willingness to fearlessly dig into someone else's code. That's kinda illusion. If you start digging into some really complex code, like distributed processing core, meta-programming framework, low-level optimizations, it can take you years to understand all that is going on. Often it's more economical just to rewrite it, even if the previous system was far better than what you'll produce. Whether companies like it or not, projects live with original team members and once they move on, it's getting complicated to maintain or extend them.
  - nevir 9y ago
    The inherent danger in rewriting dependencies always seems to be greatly underestimated, though. Especially when this is your first time tackling the problem space in earnest. * Getting the solution 80% of the way there is often pretty fast (which ends up deceiving everyone depending on it "it's almost done!") * You are nearly guaranteed to encounter a bunch of bugs that were already solved in the original dependency. * Due to the previous points, there's a big chance that your initial design had a few fatal flaws, and requires significant refactoring to address (making the project take longer, and if not careful making it much harder to understand & maintain) Rewriting dependencies can definitely be worth it from a code ownership/maintenance point of view; but more often than not, you'll end up creating a cycle where the new version of the dependency is just as obtuse as the previous—and once you leave, the same problem all over again. If a dependency is well documented, has a clear architecture, and follows consistent coding practices, it will stand the test of time (including loss of authors/maintainers). Most people/companies aren't willing to put the time/effort into that level of polish.
- shortoncash 9y ago
  Take all my upvotes and offer a newsletter for subscription please. The lack of digging is a frustration I have with a lot of new guys who join the team. For example, they might create something new and tack on some new, untested piece of code without reading whether something already exists within the same codebase. Even in a startup environment where documentation is scarce, it's worthwhile to ask the question of, "Did someone else here think of this problem here before me? What did they do?"
  - mberger 9y ago
    It comes down to unknown unknowns. Should I poll my supervisor/co-worker about every problem to see if there is already a solution? As the new guy, I have no idea who or what I should poll people about. The second problem is that people will keep stuff hidden. Have that sweet script that creates 1000 users for testing? You might share it with the team, you might also think its too trivial to bother sharing. That being said, I can see how it can be frustrating. We have packages upon packages of existing functionality, languishing somewhere in source control, used maybe only once. In the meantime, someone is going through the cycle, not only of ignoring what someone put on gitub, but what has already been tested and qa'd by their own company.
    - clintonb 9y ago
      > Should I poll my supervisor/co-worker about every problem to see if there is already a solution? As the new guy, I have no idea who or what I should poll people about. It helps to have a company culture that encourages openness and asking questions. My current company goes so far as to have a Slack channel, #askanything, where folks can ask anything. We'd much rather you ask early in the development process than pursue a path that will lead to duplication.
- chapill 9y ago
  >Include that information in a bug report, and better yet, open a PR offering to fix it. In my experience, never happens on permissively licensed projects. "Contribute time and code to open source on company time? Are you mad? Fork it and keep that as our competitive advantage!" Nevermind how dumb this approach actually is, it's what management thinks. "GPL? Oh, we have to? Oh, well, I guess if we have to."
- yorby 9y ago
  Do you consider the OS a "library"? Who checks the OS's code before shipping a program? What about the programming language's code? Do you code in assembly to reduce the amount of dependencies?
supermatt 9y ago
"But you can’t expect a map to tell you what questions to ask, and it makes no sense to read a map linearly from top to bottom, left to right." But you can read/view a map as an illustration to get an overall view of the layout of things and how they are connected. I read code. It surprises me that people deem themselves an expert in some 3rd party lib without actually understanding the internals.
- deleted 9y ago
  [deleted]
encoderer 9y ago
I’d like to spend more time reading great code. I think I’d get something out of it. Possibly more important, though, I’ve always had a fearlessness to dive into a codebase. I’ve built a little thing called Cronitor and recently I was giving a talk on building our first server agent in Go. Long story short, there were a few people surprised that I studied a couple crond implementations to figure out some important details for proper monitoring and I was surprised they were surprised. I don’t write much c or any c++ but as long as you approach it with some fearlessness and a simple plan you can learn from unfamiliar codebases. Spend up to an hour figuring out how a project is structure (hopefully a lot less) and then just skim/grep to find a spot probably close to what you’re looking for. Spread out from there. The best way to learn a new language is to build something useful and learn what you need along the way. I think studying code can be done the same way.
- bsimpson 9y ago
  Another way to approach this: review other people's code at work (e.g. code in another language or for another product). Even if you don't know the syntax of a language, you can probably follow enough of it to learn something interesting. A commit is going to be a much more easily-digestible unit than a whole codebase, and you can go ask the author to explain anything that you don't understand. Plus, over time you might understand enough to help out on that project, or help integrate it with yours.
- heavenlyblue 9y ago
  I agree with you - there are many details that are either missing from the documentation or the books on the topic. Actually reading the code usually provides the necessary _practical_ structures used in the real projects. One of the counter-examples had been the implementation of egalitarian paxos for me: when I had first read the paper I've thought that I didn't understand a lot. By reading throught the code written by the actual author I had realised that the paper by itself described something quite far away from a practically working prototype; So it wasn't my understanding of the paper in the first place that was the problem.
dizzystar 9y ago
Sometimes reading code saves your butt. I've had to make very minor changes to a code base to make it compatible with Python 3, and I've had to add things to HTML libs to prevent execution to JavaScript. Taking an open source library at face value can sometimes put you in a precarious situation. For me, it's an exercise of following the errors or trying to break the code. I'm not really sure if that counts as reading. Reading a large code base, in my experience, calls for a pencil and paper. It's a lot more work than passively reading and trying to hold all the information in my head. Reading code isn't the same as reading a novel. Perhaps better guidance on what actively reading code means is in due order?
kazinator 9y ago
We read code for all kinds of purposes other than to make an intended change. Like How the heck does this behavior (good or bad, expected or not) come about? Is this correct behavior assured in every case? What are all the possible values of these poorly documented configuration parameters or other inputs? Will this apparently working operation break under concurrency? Why is this so slow for these inputs, or under these conditions? / Why is this so unexpectedly fast? Is this doing nasty covert things I don't want, in addition to the overt functionality that I want? Is this using a particular library or OS feature to do something obvious, or did they roll their own? How will this scale in time, memory use or whatever for certain large inputs that are too impractical to actually supply just to answer this question? ...
- stenecdote 9y ago
  Hi kazinator, thanks for your comment (I'm the post's author). When I originally wrote the post, I actually had a whole section about "reading for questions", which discussed how the best way to read without modifying a program is to read to answer focused questions you have about the codebase. It kind of got lost when I shifted focus to active interaction versus passive exploration, but I agree that reading with questions in mind is far superior to just reading the code to understand it generally.
peterburkimsher 9y ago
"we usually read when we want to edit... That hacking produces better comprehension than passive, linear reading fits with what we know about learning... solid understanding emerges from active exploration, critical examination, repetition, and synthesis. Hacking beats passive reading on three out of four of these criteria." This doesn't just apply to learning programming languages - also learning Chinese. https://pingtype.github.io/docs/failed.html https://pingtype.github.io/docs/failed.html I've had much more success since writing my own tool to translate the text I'm interested in, rather than being spoon-fed some patronising phrases to recite from a textbook.
mjevans 9y ago
One of the reasons I love golang* is how the entire source code for the standard library is just a few easy clicks away from the documentation on how to use it. As an example: When I want to do something and the interfaces I'm seeing provide friction (XML parsing not /quite/ handling normal, slightly incorrect, HTML) I can get a better idea of what the library is doing behind the scenes. I got an example of how to use other exposed interfaces of the library as a tool kit for my own iteration (which added some state tracking and cleanup of that mess) instead of relying on it to decode everything and dump it back for me. Other parts of the library are similar, maybe it does 90% of what your use case needs, and you can extend those interfaces with your own library to add the corner cases that are required in your specific use case. *(even if it can be tedious at times because it's sort of a mid-level language; it abstracts some REALLY bread and butter things that probably can't be lived without, but it doesn't shield you from true complexity pitfalls)
- pjmlp 9y ago
  Smalltalk did it first and I have always been able to do it with C++, Java and Python as well.
- henesy 9y ago
  Being able to browse the Go source so easily and clearly is a fantastic way to both learn and grow in understanding of the language. It's definitely helped me as I've learned and written in Go over the years.
- majewsky 9y ago
  > One of the reasons I love golang* is how the entire source code for the standard library is just a few easy clicks away from the documentation on how to use it. Not just a few clicks. In my editor, I just navigate into /usr/lib/go/src/<package-name> which is where my distribution (Arch) puts the stdlib sources. For example, /usr/lib/go/src/io/pipe.go for the implementation of io.Pipe() which I was looking at the other day.
- Zpalmtree 9y ago
  Haskell does this well on hackage, which I personally use for all the documentation - there's a source link next to every function. Also great if you only want one function from a library rather than taking the whole dependency.
SilentCrossing 9y ago
Interesting read, but I find comparing code to a map to simplistic. I would at least say that code is more like a puzzle of a map. You may or may not understand each puzzle piece and where it fits it. That is my experience with reading code. Code is like Ogers and Ogers like Onions. They have layers of complexity :P
W0lf 9y ago
I wouldn't have thought to be in the minority when stating that I actually read a lot of foreign code out of curiosity. Especially when dealing with architectural decisions, I like to dig into codebases I consider to be good and professional (a rarity in the professional/proprietary development world unfortunately). For instance, reading the chromium source code or mozilla for that matter, give you a lot of good input since both codebases are sufficiently complex and deal with a lot of layers and stacks.
- albertgoeswoof 9y ago
  How do you know a codebase is good before you invest time digging into it?
  - W0lf 9y ago
    I don't. There are a few indicators though like the overall structure of the source files, how they are laid out on the file system. Minor details like inconsistent formatting. Typically if one or more of those indications is messed up, I'm pretty sure the rest of the codebase is also not in a good state. I still do skim through some files to confirm my expectation though.
quickthrower2 9y ago
I love reading code written in Elm. I always dive in and have a look. The reason I love it is that Elm is quite restrictive, there is a way of doing things and you can't deviate too much. Therefore reading someone else's code is usually pretty easy. In addition in Elm you can clone and run an Elm package and see the examples, do some "time travelling debugging" all within 30 seconds :-), again because there is one way of building and debugging things. No "Oh no browserify! I normally use webpack" or "WTF all that global npm shit I need to install" moment. Just once per computer: npm install elm Then do this git clone ... elm-reactor And open your browser to localhost:8080 I'd probably say Elm is more Zen than Python!
abcd_f 9y ago
How about some good _annotated_ code? :) http://www.kohala.com/start/tcpipiv2.html http://www.kohala.com/start/tcpipiv2.html - 15K line implementation of TCP/IP stack from BSD. Fantastic book, reads like a novel. http://www.pbrt.org/ http://www.pbrt.org/ - fairly detailed write up on building a pretty decent 3D renderer
poindontcare 9y ago
The only way to be sure you understand everything is to learn quantum mechanics perhaps string theory too just to be safe. But in practice modesty dictates we limit to learning as much as we can to get the job done.
markpapadakis 9y ago
Studying codebases is one of my rewarding hobbies. I ‘ve learned more from that practice( https://medium.com/@markpapadakis/interesting-codebases-159fec5a8cc https://medium.com/@markpapadakis/interesting-codebases-159f... ), than from most technical books I ‘ve read (by the way, most technical books aren’t particularly great).
svat 9y ago
Peter Seibel, the author of the Coders at Work book mentioned in the post, actually discussed this in a blog post: http://www.gigamonkeys.com/code-reading/ http://www.gigamonkeys.com/code-reading/ It's not quite true that none of the programmers interviewed in the book routinely read code for fun. In his blog post, Seibel mentions the exceptions: > First, when I did my book of interviews with programmers, Coders at Work, I asked pretty much everyone about code reading. And while most of them said it was important and that programmers should do it more, when I asked them about what code they had read recently, very few of them had any great answers. Some of them had done some serious code reading as young hackers but almost no one seemed to have a regular habit of reading code. Knuth, the great synthesizer of computer science, does seem to read a lot of code and Brad Fitzpatrick was able to talk about several pieces of open source code that he had read just for the heck of it. But they were the exceptions. The Knuth exception is notable: he does routinely read code for fun. In the book he mentions things like > I’ve got lots of collections of source code. I have compilers, the Digitek compilers from the 1960s were written in a very interesting way. They had their own language and they used identifiers that were 30 characters long but very descriptive, and their compilers ran circles around the competition at the time—this company made the state-of-the-art compilers of 1963 or ’64. And I’ve got Dijkstra’s source code for the THE operating system. […] I collected it because I’m sure it would be interesting to read if I had time. One time I broke my arm—fell off a bike—and I had a month where I couldn’t do anything much, so I read source code that I had heard had some clever ideas in it that hadn’t been documented. I think those were all extremely important experiences for me. And then a passage quoted in the post about how he reads code, with a Fortran compiler as example (not repeating here it's in Seibel's blog post: http://www.gigamonkeys.com/code-reading/ http://www.gigamonkeys.com/code-reading/ ), after which the interview (and the book) ends with Knuth saying: > don’t only read the people who code like you. (The examples in the book of people who last seriously read code years ago also interesting, e.g. Douglas Crockford mentions some programs he read, and there are multiple pages in the book about Guy L. Steele reading the TeX program.) Anyway, back to Knuth: I think that, the way he reads and digests code (or even papers: http://blog.computationalcomplexity.org/2011/10/john-mccarthy-1927-2011.html?showComment=1319546990817#c6154784930906980717 http://blog.computationalcomplexity.org/2011/10/john-mccarth...), it addresses a lot of the points the linked post makes. Even when “just reading” code, or anything for that matter, you are supposed to be doing the things the post mentions in “hacking versus passive reading”: active exploration, critical examination, synthesis. For example, Knuth read the code of ADVENT (aka Colossal Cave Adventure), and loved the code (not just the game itself) so much that he rewrote it to share with others to read, in his own preferred literate-programming (CWEB) style that matches the way he thinks (http://www.literateprogramming.com/adventure.pdf http://www.literateprogramming.com/adventure.pdf). This definitely doesn't sound like “passive reading” to me. Nevertheless, great post.
mattbierner 9y ago
On this spectrum, there's also "read by refactoring" (https://www.jamasoftware.com/blog/read-by-refactoring/ https://www.jamasoftware.com/blog/read-by-refactoring/). You'll need good refactoring tooling and good versioning (as you'll almost certainly want to discard many of your trial refactorings) but I've found that the approach helps me evaluate how my mental model matches up with reality. It also generally leads to a more readable codebase too
henesy 9y ago
One of the best parts of the Plan 9 system is that all of the kernel/command/library source is on hand and readily available to consult at any given time. Jointly, an important detail is that the complexity of said source is kept to a sane minimum and general style trends mean that most, if not all, of the source is formatted similarly and legibly. The system is so compact that you can keep most of the system source in your head at one time if you really need to.
- henesy 9y ago
  A fun tidbit from the Plan 9 compilers only really able to be found by reading the source: https://groups.google.com/forum/#!msg/comp.os.plan9/uMF7A4gkJsw/kNWBQ8U93LcJ https://groups.google.com/forum/#!msg/comp.os.plan9/uMF7A4gk...
jmadsen 9y ago
I had to use a js library for something today. npm install downloaded 113 new packages. I'm not even a javascript programmer; I'm saddled with being "full-stack" I'm supposed to read all that every time I need a new library? Or write it myself, when I have absolutely no idea how to build something that complicated, if I had time, negating the whole point of OSS in the first place?
- aepiepaey 9y ago
  The point made is to read code to become a better programmer. That doesn't mean you have to read all code you ever use.
- akkartik 9y ago
  OP makes no prescription about reading code. It merely points out what successful attempts to understand strange codebases have in common, and discusses what writers can do to help readers succeed. (I proof-read drafts of OP. And I have made prescriptions about reading code. But that's a separate story.)
superasn 9y ago
By corollary why it's also so important and terrifying to release open source projects too. You're oftening wondering who is gonna dig into this code of yours and think about your coding, esp. when you're not following the best practices and TDD.
Moodles 9y ago
In a similar vein, I often wonder just how many people actually check cryptographic algorithms. We'd all probably agree that it is good not to roll your own crypto algorithms, or even in implementation of known crypto such as AES. But how many experts in the world are there for AES and secure hashing? A lot of the major MD5 and SHA1 work all came from Marc Stevens' team. How many experts are there in this field, really? "Provable security" is another issue: Shoup, Menezes and Koblitz have all expressed concern about the current state of the art, I think.
- simias 9y ago
  I'm far from an expert in cryptography but I've had to dig into OpenSSL's codebase on more that one occasion and every time it left me with a deep sense of uneasiness. It's really not what I would deem good code: too many macros, too many potentially confusing API conventions (inconsistent return values in case of error, unclear resource lifetimes, ...), not enough comments and documentation... It's not the worst C codebase I've ever seen (far from it) but for something as critical I'd put the bar very high it terms of code quality and OpenSSL doesn't even come close.
antirez 9y ago
This is a bit western centric maybe, because I see Chinese developers reading tons of code, to the point that I receive an incredible amount of Redis PRs about conceptual bugs that can never happen in practice, since some Chinese developer is reading the code and doing the math in her/his head.
- ralusek 9y ago
  That's interesting, but also not so surprising. China has had decades of directing primary engineering/manufacturing efforts towards reverse engineering western products, or executing specific designs specified from western clients. The attitude of looking to someone else's engineering to learn from it is probably something the rest of us could do a bit more of.
- heavenlyblue 9y ago
  Any case you could share a link to one of these, if possible? A matter of curiosity.
  - __s 9y ago
    First PR I saw on redis GH: https://github.com/antirez/redis/pull/4714 https://github.com/antirez/redis/pull/4714 That same user also made https://github.com/antirez/redis/pull/4685 https://github.com/antirez/redis/pull/4685 This may be? https://github.com/antirez/redis/pull/4622 https://github.com/antirez/redis/pull/4622 This was apparently critical, but also kind of an edgecase: https://github.com/antirez/redis/pull/4568 https://github.com/antirez/redis/pull/4568 & one more from someone else: https://github.com/antirez/redis/pull/4568 https://github.com/antirez/redis/pull/4568
  - antirez 9y ago
    Probably soloestoy recently (but he also does a lot of Real world Redis) and Sun He in the past hold the records, but there are several...
- stenecdote 9y ago
  I wish one of these pull requesters would comment on this post! I'd love to hear how they're going about reading your code and whether they're actually using Redis or just reading it for enjoyment.
mar77i 9y ago
If you want to exercise reading concise and easily understandable standard C code, grab a toy project from suckless.org. Even though I write my code more defensively in some ways, that stuff is really straightforward to hack on.
alister 9y ago
Reading code just for fun was a thing in the early and mid years of UNIX (eg., 7th edition or System V). People eagerly passed around faint 10th generation photocopies of Lion's printout of and commentary[1] on the UNIX source code. The annual USENIX conference had a popular short course in which they went through the entire UNIX kernel line by line. Why was that a thing back then and not now? Certainly UNIX was an amazing piece of work, but there are amazing works today -- the modern browser for example. I think that the difference is that the totality of the UNIX kernel was comprehensible for a single mind. The UNIX kernel in 1983 was less than 20,000 lines of code, almost all in C, and more than 75% was not machine-dependent.[2] The amazing software of today, something worthy of reading for fun, is literally millions of lines of code, written in multiple languages you don't know, and evolving so fast that whatever you read might be completely different a few months later. There's no joy in knowing a tiny part of that. It would be like reading 10 pages from the middle of a novel, a novel so big that you know you'll never finish it. [1] https://en.wikipedia.org/wiki/Lions%27_Commentary_on_UNIX_6th_Edition,_with_Source_Code https://en.wikipedia.org/wiki/Lions%27_Commentary_on_UNIX_6t... [2] https://en.wikipedia.org/wiki/History_of_Unix#1980s https://en.wikipedia.org/wiki/History_of_Unix#1980s
- pm215 9y ago
  I suspect in a lot of cases reading the Lions commentaries was not purely for fun. The original audience (Lions' students) were doing it as part of their university course. Many of the contemporary readers would have been reading it because they needed to understand, fix bugs or add features to their unix systems (I think one of the introductions in the reprint book is from somebody who was in this position). But you're right I'm sure that the the key was the combination of (a) real world and genuinely useful code that the readers were using (b) size such that you could realistically understand all of it (c) excellent accompanying code commentary. I read through the Lions book in the late 1990s when the reprint came out, essentially "just for fun" -- it remains probably the only sizeable piece of code I've read for fun.
- Klathmon 9y ago
  This is still done today, just with other software. I've read blog posts that go through the entire redux javascript library line by line, or review an entire chunk of React. Even just yesterday here on HN was a submission that just contained shaders from Wolfenstein.
z3t4 9y ago
I read the source code for the libs all the time, mostly to see what methods and properties they expose. I learn something new every time. This is JavaScript/NodeJS. I don't miss compiled .dll's
INTPenis 9y ago
I love to dig through peoples code. Unfortunately I know myself enough to know that it can be a giant time sink if I let myself go. In swedish there's a phrase called "snöa in", to get snowed in literally, and that's what happens to me. So after enough years in the biz I try to hold myself back from this habit.
chukye 9y ago
Once I propose for a team to read the code of Redux before adding it to the codebase (if you don't know, redux is a very small libraries, very easy to read). But no one seemed to care, they was just saying things like "Well, I'll read this blog post that teach us how to use it", and I was like: "BUT, if you read the code You will know exactly how to use it! Why not go to the source instead of read other people opinions?!"
mannykannot 9y ago
I must disagree with this sentiment, though I understand where it is coming from: "Clean, solidified abstractions are like well-marked, easy-to-follow paths through a forest — very useful if they lead in the direction we need to go, but less useful when we want to blaze arbitrary new paths through the forest." Using the same analogy, if you want to blaze a new trail through the woods, you need a map of the territory, and a map for this territory takes the form of higher-level abstractions with clear, explicit semantics and minimal but effective, and unambiguous, interfaces. One of the biggest problems in modifying or reusing existing code is the risk of breaking higher-level consistency, for example by violating an implicit constraint that is necessary for correctness (the biggest Ethereum errors have been of this form, where the implicit assumptions included when and how often initialization is performed, and that a certain library will be present.) If the clean code movement has been delivering code that works as step-by-step instructions for getting from A to B, but not as a map, it may be because breaking code down into small functions and classes is easier, more visible and more measurable than making coherent abstractions with consistent and minimal interfaces at a higher level.
- stenecdote 9y ago
  To push back on this point a little bit, I think we agree that a map is one of the most useful things you can have but disagree on what the right type of map is. While I'm not anti-abstraction, I think people often impose poorly chosen high-level abstractions on top of messy lower-level components (akkartik calls this Authoritarian High Modernism in a comment (https://lobste.rs/s/gtxi5y/nobody_s_just_reading_your_code_how_we#c_0joqn2 https://lobste.rs/s/gtxi5y/nobody_s_just_reading_your_code_h...) on Lobsters). This approach leaves readers having to understand a bad abstraction and its internals, a map, its territory, and inconsistencies between the two. As I understand it (please correct me if I'm wrong), you want people to do the work to find better high-level abstractions. I agree that it would be good if we could do this more often. I just also think we can find other ways to give people maps that work with our existing, messy codebases that don't have good, minimal abstractions already.
  - mannykannot 9y ago
fghtr 9y ago
https://www.fsf.org/blogs/community/who-actually-reads-the-code https://www.fsf.org/blogs/community/who-actually-reads-the-c... "Two-and-a-half months later I received an email from someone who not only managed to find the comment, but also managed to guess the code had to be rot13'ed."
jesperlang 9y ago
Reading code always felt awkward, slow and ineffective. The cognitive load is huge, you have to compute some things while memorizing others and at the same time keep track of flows. Thats why I always have pen and paper while diving into a new big code base, you simply can't keep it in your head. But something we forget often is... the goal of reading code is almost always to understand a system or some part of it. And reading code for me is a terrible way to do it. Code is just one level of abstraction and one way of seeing things. We should have multiple ways of "reading" a system depending on what level we want to understand it. No, I don't think that rushed, out of date "system documentation" you wrote will do. I still think this is not solved in a satisfying way. We could find inspiration from things that works well, just imagine how google maps lets you look at the world at many different levels, from continents down to street level. Also it provides different views of each level, street maps are useful for tourists, terrain maps for hikers and satellite images for someone who want to study vegetation. How you will "read" a system depends on what you are trying to understand.
- stenecdote 9y ago
  As the author of the post, just wanted to reply and say I'm totally open to the notion that the solutions I proposed will not solve the problem. I'm much more sure about the problem than I am about the solution. A google maps zoom system for code sounds awesome, although I have no idea how that might work.
  - lugg 9y ago
    Brackets inline editor comes to mind. Just replace CSS definitions with function definitions (I think it might already do this for ?) The zooming functionality you're thinking about probably wouldn't be very practical. (Zooming in down to opcode or out to high level framework calls.) But simply being able to inspect definition inline would do.
- douglaswlance 9y ago
  I find it useful to write comments about what's happening in the code as I am trying to figure it out.
mark_l_watson 9y ago
One thing that helps me make reading code in code reviews less passive is to add occasional temporary print statements and then run the unit tests again that cover the code I am reading. Seeing variable values satisfies some curiousity. Sometimes I will use a debugger to step through code I am reviewing but for some reason I prefer print statement to satisfy any curiousity I have about how the code works.
djhworld 9y ago
The thing about code reading for me is you have to have decent tooling. It's pointless reading code in Github, especially for a codebase you are not familiar with, you need to get the codebase open in an IDE to be able to dive in quickly and back again to build the map out in your head. I don't think you can read a portion of code from top to bottom without having tooling....unless the piece of code is really short and just one or two files
- wkz 9y ago
  The one online tool that I use for this kind of work is Woboq Code Browser. It's only C/C++ projects and not many are freely available. The Linux kernel is though and I spend a lot of time in that code base. glibc, gcc and llvm are also available. https://code.woboq.org/ https://code.woboq.org/
  - guruz 9y ago
    Which projects would you like to see on code.woboq.org? Maybe we can add them.
- karmakaze 9y ago
  Absolutely this. You can always tell a PR reviewed via web. Unless it's very localized, I always open up the branch in an IDE and follow the flow through the abstractions and see if the changes make sense, and if there are additional opportunities or less clear gray areas of operation. It will be even better when web tools catch up. Could be even better if they can navigate across repos in different languages even without even checking out a single file.
your-nanny 9y ago
Do I sit back in my comfy chair on an evening and just read through some code repo? Well, actually, yeah, I do... Though not terribly often. Do I sit back and read through programming bloggers' explaining their code? Yes, pretty often, and based on HN, that's pretty true of a lot of programmers. Still I'm more likely to find myself browsing wikipedia for math and cs
gwbas1c 9y ago
Reading code is difficult and time-consuming. To really understand a function you need to try it with different inputs, either with interactive testing or in a debugger; AND you need to know what the functions it calls does. The second part, knowing what the code the function is calling does, is nearly impossible when they are other functions in the program itself.
kruhft 9y ago
Somebody read my code once and commented on my improper use of 'mod' or something like that. It then turned into a long discussion on how they might be incorrect, and so on and so forth on Reddit, back and forth on Lisp. I love Lisp. It's a great language with great programmers.
OutThisLife 9y ago
Best way to figure out someone else's codebase, IME, is to just dive in heavy and start breaking things. Figuring out what's connected to what and why by removing chunks of code that you need to understand.
Stwerner 9y ago
I've been thinking about this idea for a while and love the reimplementation idea. I really want to start live-streaming myself coding (or pair programming) and have been trying to come up with some ideas that would end up being interesting to watch. One idea was taking a popular open source tool with a good test suite and trying to get the tests to pass, but I think checking code out at a point before a new feature is implemented and reimplementing it could end up being really interesting. Could even make it more interactive by sharing the checkout point so viewers can do an implementation themselves and doing a stream walking through different interesting implementations.
pjungwir 9y ago
I agree most programmers are too quick to write before they read, but I'm not sure why it has to be "for fun". Reading code takes effort, just like understanding any new complex thing. The primary virtue here is patience. It's a lot like debugging actually: the goal is to understand. Setting aside the "for fun", of course I read my co-workers' code, but also I read my dependencies' code quite often. In particular when you have something that is meant to be extended, the docs never completely cover all the use cases, and the quickest way can be to start reading. Some examples from my own recent work are reading code from the Ruby gems devise, devise_token_auth, and spree, and also reading Django class-based views (all things meant to be extended). One trick I've learned in Ruby is `cd $(bundle show spree_core)`. From there you can grep, find, read, even add your own `puts` or whatever. Actually I have been reading some code for fun: Postgres! I have an extension that adds temporal foreign keys, and I'm slowing porting it from plpgsql to C. Reading the code for standard foreign keys is very helpful. Same thing for a lot of other extensions I've written in the last few years. It has helped me a ton to read others' extensions and the core code.
- Kagerjay 9y ago
  This is what I've been working on for the past few months. I've been writing a lot of spaghetti code in the past and come back to find how poorly I organized it. I'm suppressing the urge to immediately write code, but rather, look at the big picture first, digesting the information, even if it means prolonging writing anything for a day or several hours. I usually write pseudocode first before doing anything now. Either on paper, as code comments, or as a list. Sometimes I find myself writing tests on paper as well, it helps out a lot too. A programming language is just a means to an end, what they all have in common is a set of logic to follow. Then I debug. I just fork the code, comment things out, add a little bit of functionality, etc. When I have a grasp on it write some TDD I find the easiest codebase that I refer a lot is the todomvc, or any similar ones that have different implementations (differnet languages) for the same general solution. All the core logic between languages is mostly the same, so its easy to dig through and understand a different programming language / how its generally organized. One thing I remember learning about from experience polyglot programmers. Its extremely helpful to have an existing codebase you've written (such as todoMVC), and using that same example / porting it over in a different language to learn that languages nuances.
dmurthy 9y ago
A few years ago in my previous company there was a major push to convert a lot of the codebase to Python. A byproduct of this conversion was an easier way to build UI using predefined widgets. Yet no one was using this functionality due to the mindset of years of TCL. I honestly hated TCL. So I took it upon myself to explore the entire new code base and go about building UI panels. Initially I just couldn't understand how the code mapped out until one day I just printed out some of the code and read it repeatedly till it made sense. The whole exercise was an eye opener.
TheMagicHorsey 9y ago
I read code to learn all the time. And I don’t get paid to program.

352 points by chauhankiran 9y ago

One of the things that sets a good programmer apart is the willingness to fearlessly dig into someone else's code. I've heard it said before that you're responsible for every line of code you ship to your users. It follows that you shouldn't treat your dependencies as black boxes. Getting a stack trace that's 3 or 4 levels deep in Django/React/<insert library here>? Dig under the covers and understand what's happening. Include that information in a bug report, and better yet, open a PR offering to fix it. You'll learn a whole lot and be more effective if you let your curiosity expand beyond code you/your team has written.

jotto 9y ago
No mention of time in the article or in your comment. With infinite time we could fix a lot of things.
- ef4 9y ago
  Sure, but I think many programmers underestimate the investment value of that time. Diving into codebases is a skill that gets stronger with use, such that you can eventually do it radically faster. That makes a much larger set of problems economically practical to fix.
- bsimpson 9y ago
  Of course you can expand this to be unreasonable - you could also make a strawman argument about how you need to understand all the compilers, OS APIs, and chip architectures that support your product. Obviously, that's not what I'm saying. I'm making a few assumptions: the code you're depending on was written by people like you (e.g. other open source volunteers with expertise in the same language as you); the limitation you're encountering is problematic for your users, product, or business; and that you'd rather have it work correctly than design around it (e.g. remove the affected feature). If those are all true, yeah, it's probably worth a week or two to figure out what's wrong and fix it (especially since you'll gain comfort, familiarity, and efficiency in that codebase over time - it might even inspire other enhancements). I realize that nobody has infinite time and I trust you to make reasonable decisions about what to spend where. Still, I implore you to not let yourselves be afraid of Other People's Code. It's probably not as scary as you think it is; after all, it was written by other people like you.
  - copperx 9y ago
    > It's probably not as scary as you think it is; after all, it was written by other people like you. Most interesting code has been written by people smarter than me (the kernel, compositing window manager, network stack, web rendering engine, virtual machine, etc.), and while reading that code probable isn't intractable, it would take me decades of effort to understand. At the same time, I think I can be an effective developer by building on top of those technologies (i.e., being a great plumber). And that's OK.
    - deleted 9y ago
      [deleted]
  - astura 9y ago
    >it's probably worth a week or two to figure out what's wrong and fix it Ha! I'm convinced that people who say stuff like this haven't worked where their paycheck comes from actual paying customers, as opposed to VC dollars or subsidies from other parts of the business's revenue. > Still, I implore you to not let yourselves be afraid of Other People's Code. Wtf? Nobody said they were and what is with the Condescending Proper Noun? >It's probably not as scary as you think it is Nobody said it was.
panic 9y ago
That's one thing that bothers me about modern Electron-style apps which depend on an entire web rendering engine -- actually digging through all the dependencies to track down a bug can be almost impossible. For example, I was hitting a bug in an Electron app a while back and wanted to track it down. I got the app building with a locally-built version of Electron, but it became clear the real bug involved Chromium's loader code. My poor 4-year-old laptop ran out of disk space before I could even download the whole libchromiumcontent repository, let alone build it!
- hinkley 9y ago
  I don’t think Library designers have adapted yet to a library heavy world. We still write them like we are using dozens but we have hundreds. Every library demands a fraction of our attention greater than its relative fraction of the code. Complex calling conventions, deep call stacks. Why am I even using a single purpose library with multiple levels of abstraction in it? At this point a library should BE the abstraction, not contain them. I have to trace code through my code into yours into your dependencies. Just stop already. Keep It Simple Stupid.
  - gaius 9y ago
    I don’t think Library designers have adapted yet to a library heavy world. We still write them like we are using dozens but we have hundreds. The other day I was debugging an app that - no joke - pulled in 600M of libraries that were all so it could use one file parsing function that could have been re-implemented in <100 lines. Crazy!
curtis 9y ago
> ... willingness to fearlessly dig into someone else's code. I've done this many times. The problem is that it takes a lot of time, and there's no way you can dig through more than a fraction of a large codebase. You gotta pick your battles.
- nathanaldensr 9y ago
  Pragmatism wins the day, for sure. I'll happily read someone else's code (e.g., an open-source GitHub repository I rely on) if it's necessary to troubleshoot issues or implement a feature. There needs to be a reason (i.e., motivation) to read code.
  - andrea_sdl 9y ago
    This is the only reason (at least for now) that makes me dig into a library code. And I always got some benefit from it. I think we often treat libraries as voodoo blackboxes, but they're more often thatn not created by normal people with great skills. Sometimes it's a great way to learn a way to code, or even better, to understand the behaviour of a library. I found myself doing it more often when the IDE itself downloaded and linked the sources. (Lazy, I know).
    - TeMPOraL 9y ago
      > I found myself doing it more often when the IDE itself downloaded and linked the sources. (Lazy, I know). Same here. It's about trivial inconveniences. I sometimes can be bothered to download and read through the codebase of a dependency, but most of the times I end up in third-party code happen when it's just a single jump-to-definition keypress away - the same keypress (M-.) which I use all the time for my code. That seamless browsing really blurs the difference.
    - WillReplyfFood 9y ago
      The blackbox vodoo view- sometimes is just a excuse for lazy evaluation. One can profile a black box- but many fear the moment they discover that there beloved "I know all the api calls from memory alone" library shows bad perfRomance.
tobr 9y ago
I always get an uneasy feeling when this is not (practically) possible. Whenever I add a dependency to a project I will at least skim through the source to get an idea of what it is I’m adding, and it pleases me a lot when a library is well written and self contained enough that this does not mostly leave me confused or overwhelmed. There’s a similar reason why I have such mixed feelings about some build tools and compilers, especially in the JS world - the output is often hideous and so unlike what a human might write. I know you can say that what the output looks like doesn’t really matter, as long as it works as it should. But it just makes it so much harder to feel confident that it actually does.
- icebraining 9y ago
  The output of gcc and clang probably don't look much like hand-written machine code either, though. Or, for that matter, the output of V8 and SpiderMonkey. You can't avoid the compilers :)
  - tobr 9y ago
    Of course! I’m not saying it’s rational. If I had experience hand-writing machine code maybe I would feel the same about that...
    - khedoros1 9y ago
      I've done bits of assembly, but I've also studied compilers. They felt less magical to me when I learned some of how they work. Then again, I'm primarily a C++ developer, so I'm used to the output of my build tools being non-trivial to understand.
  - deleted 9y ago
    [deleted]
- akvadrako 9y ago
  That's one reason rollup is better than webpack; the output looks almost hand rolled.
- 9y ago
taeric 9y ago
Don't let survivorship and confirmation bias cloud your judgment. What sets many successful people apart is not any one thing, but that they were successful. Many are willing to dive into other people's code in methods that are far deeper than I can comprehend, and yet they have failed at whatever task they were going about. So, yes. By and large, you shouldn't stop at your dependency boundaries. However, your job is to get results. For most of us, even if the dependency is bad, you are better off changing your code. Why? Because you can more quickly get a change in your codebase than you can in the dependency. And heaven help you if you think it is worth forking. This is not to say don't update the base place. By all means, please do. But don't wait for that change to land before you do something on your end, too. Even if that means not using your actual fix.
- icebraining 9y ago
  Forking is not that bad, unless upstream has a lot of activity yet takes a long time to accept patches. Many projects are mostly dormant, so keeping a patchset and occasionally rebasing is easy enough. We use git-aggregator[1] for easily applying pending PRs/MRs over the upstream branch, and for the most part, conflicts are rare. [1] https://pypi.python.org/pypi/git-aggregator https://pypi.python.org/pypi/git-aggregator
  - heavenlyblue 9y ago
    The thing I find with python/ruby ecosystems - is that the versions are generally pinned, so if you do fork a change for your specific need; you'll end up using it for the project. If the change isn't merged - then at least you may use it yourself.
    - taeric 9y ago
      These are both valid choices. I had not meant it to be that it is not possible. However, in both cases you now have a new line of work. It is common in my experience to find folks are behind in their first job, so adding a new one is dangerous.
- 9y ago
deleted 9y ago
[deleted]
natecavanaugh 9y ago
I can only say Amen to this. If you ship it, you either accept responsibility as far as you can, or you're "that" person who redirects you another department because it's not technically their job. Plus, as you mentioned the opportunity for learning is immense.
segphault 9y ago
> I've heard it said before that you're responsible for every line of code you ship to your users. This is a great way of thinking about it. I wish that more people took it to heart when choosing third-party dependencies to adopt, especially in the Node.js ecosystem where people are tragically cavalier about adding literally hundreds of under-scrutinized transitive dependencies to their projects.
geezerjay 9y ago
I agree with the spirit but in practice more often than not that's a complete waste of time and energy, unless you're trying to figure out the cause of a specific bug when you've already excluded your code as its source.
lawn 9y ago
To be sure. There's also a trade off to doing it too much as the time it can take to really go deep can effectively paralyze the development. You need to be able to properly use abstraction layers and be able to treat code as black boxes when needed, you only need sufficient knowledge to work with a system. Of course we shouldn't stop until we reach that level and too often developers give up. But it's also common, especially for very skilled or perfectionist developers, to go too far.
watwut 9y ago
Really good programmer is imo set apart by ability to decide when to dig into others code and when not.
chiefalchemist 9y ago
"...open a PR offering to fix it..." Perhaps I've been asking to wrong people (at the wrong time) - forgive me - but (as dar as I can tell) git has no easy / reasonable way to repo'tize a (vendor/) dependency. That means you have to bend over backwards (read: make too much extra effort) to patch such a library local and then eventually PR that change. This feels like a fairly significant flaw in the current OSS "ecosystem." It encourages the wrong behaviour, and discourages the right one. Or is there something I'm just not understanding?
- andrewaylett 9y ago
  It's not really so much that _git_ is lacking the functionality -- it's actually much more possible to do reasonable vendoring with git than with (say) subversion, as you can use submodules to point at your fork. To my mind the issue is more with tooling: you don't typically want to vendor your code, most of the time you want to use a published module. And the process for publishing your fork is too complex to be worthwhile. Also, the work to keep the fork itself up-to-date when you realise it's necessary is minimal, but by switching away from the original you lose tooling support for telling you when you need to make that update.
  - chiefalchemist 9y ago
    I understand there are workarounds, but they are just that. I mean, for example, I'm working away but hit a bug in a dependency. Now i have to fork, submodule, etc. I'm not expecting frictionless per se. But it just feels like there's more overhead than there needs to be. Also, it's possible (in rare cases) the vendor folder is not .gitignore'd. Now what? Submodules is out. Again, given the nature of most modern dev (i.e., dependencies) vit feels underprepared. Re: "...but by switching away from the original you lose tooling support for telling you when you need to make that update..." Yes. But doesn't this actually support the idea (mentioned above) not to treat your dependencies like black boxes? That is, no pain no gain? Yeah, perhaps some sort of simple notification your fork needs an update. Just the same, ALL code should, in theory, be treated as your own. You wouldn't blindly merge/commit a colleague's work, so why shouldn't your dependencies be semi thoroughly reviewed as well? Are we too busy to do the right thing(s)?
giancarlostoro 9y ago
I've only found time to do this with CherryPy really, mostly cause every time someone asks for help they say if something is missing in the documentation you could always read the source which is also commented and readable. It really is approachable and not too big. I never have time to go through the source of every single library but you raise a good time to do so: stack traces. Although when doing front-end JS code it's a bit less approachable if your files are already minified (client hands source like this) and the error becomes a lot less useful.
Daycrawler 9y ago
I just write that the library's function at hand is unreliable and can't be used this way.
Cthulhu_ 9y ago
It's why (in theory) I love Go; as opposed to the JS hell as mentioned by another commenter, it has a lot more emphasis on consistency and, in a way, predictability, so there's less surprises and distractions (e.g. code style) when you browse other people's code. Also no "black box" dependencies (usually), you pull every lib from source and have it on your machine.
noway421 9y ago
Isn't it the purpose of blackboxes, to abstract me from doing this? I'm here to get the work done and hopefully having to debug only code of my colleagues, why would I want to waste time on debugging something that should have been battle tested by myriads of people before me? Argument like this almost justifies leaky abstractions as something to be praised.
- amazingman 9y ago
  Isn’t the purpose of open source also to empower you, the developer using the library, to fix the bug you find so that everyone may benefit?
bitL 9y ago
> willingness to fearlessly dig into someone else's code. That's kinda illusion. If you start digging into some really complex code, like distributed processing core, meta-programming framework, low-level optimizations, it can take you years to understand all that is going on. Often it's more economical just to rewrite it, even if the previous system was far better than what you'll produce. Whether companies like it or not, projects live with original team members and once they move on, it's getting complicated to maintain or extend them.
- nevir 9y ago
  The inherent danger in rewriting dependencies always seems to be greatly underestimated, though. Especially when this is your first time tackling the problem space in earnest. * Getting the solution 80% of the way there is often pretty fast (which ends up deceiving everyone depending on it "it's almost done!") * You are nearly guaranteed to encounter a bunch of bugs that were already solved in the original dependency. * Due to the previous points, there's a big chance that your initial design had a few fatal flaws, and requires significant refactoring to address (making the project take longer, and if not careful making it much harder to understand & maintain) Rewriting dependencies can definitely be worth it from a code ownership/maintenance point of view; but more often than not, you'll end up creating a cycle where the new version of the dependency is just as obtuse as the previous—and once you leave, the same problem all over again. If a dependency is well documented, has a clear architecture, and follows consistent coding practices, it will stand the test of time (including loss of authors/maintainers). Most people/companies aren't willing to put the time/effort into that level of polish.
shortoncash 9y ago
Take all my upvotes and offer a newsletter for subscription please. The lack of digging is a frustration I have with a lot of new guys who join the team. For example, they might create something new and tack on some new, untested piece of code without reading whether something already exists within the same codebase. Even in a startup environment where documentation is scarce, it's worthwhile to ask the question of, "Did someone else here think of this problem here before me? What did they do?"
- mberger 9y ago
  It comes down to unknown unknowns. Should I poll my supervisor/co-worker about every problem to see if there is already a solution? As the new guy, I have no idea who or what I should poll people about. The second problem is that people will keep stuff hidden. Have that sweet script that creates 1000 users for testing? You might share it with the team, you might also think its too trivial to bother sharing. That being said, I can see how it can be frustrating. We have packages upon packages of existing functionality, languishing somewhere in source control, used maybe only once. In the meantime, someone is going through the cycle, not only of ignoring what someone put on gitub, but what has already been tested and qa'd by their own company.
  - clintonb 9y ago
    > Should I poll my supervisor/co-worker about every problem to see if there is already a solution? As the new guy, I have no idea who or what I should poll people about. It helps to have a company culture that encourages openness and asking questions. My current company goes so far as to have a Slack channel, #askanything, where folks can ask anything. We'd much rather you ask early in the development process than pursue a path that will lead to duplication.
chapill 9y ago
>Include that information in a bug report, and better yet, open a PR offering to fix it. In my experience, never happens on permissively licensed projects. "Contribute time and code to open source on company time? Are you mad? Fork it and keep that as our competitive advantage!" Nevermind how dumb this approach actually is, it's what management thinks. "GPL? Oh, we have to? Oh, well, I guess if we have to."
yorby 9y ago
Do you consider the OS a "library"? Who checks the OS's code before shipping a program? What about the programming language's code? Do you code in assembly to reduce the amount of dependencies?

supermatt 9y ago

"But you can’t expect a map to tell you what questions to ask, and it makes no sense to read a map linearly from top to bottom, left to right." But you can read/view a map as an illustration to get an overall view of the layout of things and how they are connected. I read code. It surprises me that people deem themselves an expert in some 3rd party lib without actually understanding the internals.

deleted 9y ago
[deleted]

encoderer 9y ago

I’d like to spend more time reading great code. I think I’d get something out of it. Possibly more important, though, I’ve always had a fearlessness to dive into a codebase. I’ve built a little thing called Cronitor and recently I was giving a talk on building our first server agent in Go. Long story short, there were a few people surprised that I studied a couple crond implementations to figure out some important details for proper monitoring and I was surprised they were surprised. I don’t write much c or any c++ but as long as you approach it with some fearlessness and a simple plan you can learn from unfamiliar codebases. Spend up to an hour figuring out how a project is structure (hopefully a lot less) and then just skim/grep to find a spot probably close to what you’re looking for. Spread out from there. The best way to learn a new language is to build something useful and learn what you need along the way. I think studying code can be done the same way.

bsimpson 9y ago
Another way to approach this: review other people's code at work (e.g. code in another language or for another product). Even if you don't know the syntax of a language, you can probably follow enough of it to learn something interesting. A commit is going to be a much more easily-digestible unit than a whole codebase, and you can go ask the author to explain anything that you don't understand. Plus, over time you might understand enough to help out on that project, or help integrate it with yours.
heavenlyblue 9y ago
I agree with you - there are many details that are either missing from the documentation or the books on the topic. Actually reading the code usually provides the necessary _practical_ structures used in the real projects. One of the counter-examples had been the implementation of egalitarian paxos for me: when I had first read the paper I've thought that I didn't understand a lot. By reading throught the code written by the actual author I had realised that the paper by itself described something quite far away from a practically working prototype; So it wasn't my understanding of the paper in the first place that was the problem.

dizzystar 9y ago

Sometimes reading code saves your butt. I've had to make very minor changes to a code base to make it compatible with Python 3, and I've had to add things to HTML libs to prevent execution to JavaScript. Taking an open source library at face value can sometimes put you in a precarious situation. For me, it's an exercise of following the errors or trying to break the code. I'm not really sure if that counts as reading. Reading a large code base, in my experience, calls for a pencil and paper. It's a lot more work than passively reading and trying to hold all the information in my head. Reading code isn't the same as reading a novel. Perhaps better guidance on what actively reading code means is in due order?

kazinator 9y ago

We read code for all kinds of purposes other than to make an intended change. Like How the heck does this behavior (good or bad, expected or not) come about? Is this correct behavior assured in every case? What are all the possible values of these poorly documented configuration parameters or other inputs? Will this apparently working operation break under concurrency? Why is this so slow for these inputs, or under these conditions? / Why is this so unexpectedly fast? Is this doing nasty covert things I don't want, in addition to the overt functionality that I want? Is this using a particular library or OS feature to do something obvious, or did they roll their own? How will this scale in time, memory use or whatever for certain large inputs that are too impractical to actually supply just to answer this question? ...

stenecdote 9y ago
Hi kazinator, thanks for your comment (I'm the post's author). When I originally wrote the post, I actually had a whole section about "reading for questions", which discussed how the best way to read without modifying a program is to read to answer focused questions you have about the codebase. It kind of got lost when I shifted focus to active interaction versus passive exploration, but I agree that reading with questions in mind is far superior to just reading the code to understand it generally.

peterburkimsher 9y ago

"we usually read when we want to edit... That hacking produces better comprehension than passive, linear reading fits with what we know about learning... solid understanding emerges from active exploration, critical examination, repetition, and synthesis. Hacking beats passive reading on three out of four of these criteria." This doesn't just apply to learning programming languages - also learning Chinese. https://pingtype.github.io/docs/failed.html https://pingtype.github.io/docs/failed.html I've had much more success since writing my own tool to translate the text I'm interested in, rather than being spoon-fed some patronising phrases to recite from a textbook.

mjevans 9y ago

One of the reasons I love golang* is how the entire source code for the standard library is just a few easy clicks away from the documentation on how to use it. As an example: When I want to do something and the interfaces I'm seeing provide friction (XML parsing not /quite/ handling normal, slightly incorrect, HTML) I can get a better idea of what the library is doing behind the scenes. I got an example of how to use other exposed interfaces of the library as a tool kit for my own iteration (which added some state tracking and cleanup of that mess) instead of relying on it to decode everything and dump it back for me. Other parts of the library are similar, maybe it does 90% of what your use case needs, and you can extend those interfaces with your own library to add the corner cases that are required in your specific use case. *(even if it can be tedious at times because it's sort of a mid-level language; it abstracts some REALLY bread and butter things that probably can't be lived without, but it doesn't shield you from true complexity pitfalls)

pjmlp 9y ago
Smalltalk did it first and I have always been able to do it with C++, Java and Python as well.
henesy 9y ago
Being able to browse the Go source so easily and clearly is a fantastic way to both learn and grow in understanding of the language. It's definitely helped me as I've learned and written in Go over the years.
majewsky 9y ago
> One of the reasons I love golang* is how the entire source code for the standard library is just a few easy clicks away from the documentation on how to use it. Not just a few clicks. In my editor, I just navigate into /usr/lib/go/src/<package-name> which is where my distribution (Arch) puts the stdlib sources. For example, /usr/lib/go/src/io/pipe.go for the implementation of io.Pipe() which I was looking at the other day.
Zpalmtree 9y ago
Haskell does this well on hackage, which I personally use for all the documentation - there's a source link next to every function. Also great if you only want one function from a library rather than taking the whole dependency.

SilentCrossing 9y ago

Interesting read, but I find comparing code to a map to simplistic. I would at least say that code is more like a puzzle of a map. You may or may not understand each puzzle piece and where it fits it. That is my experience with reading code. Code is like Ogers and Ogers like Onions. They have layers of complexity :P

W0lf 9y ago

I wouldn't have thought to be in the minority when stating that I actually read a lot of foreign code out of curiosity. Especially when dealing with architectural decisions, I like to dig into codebases I consider to be good and professional (a rarity in the professional/proprietary development world unfortunately). For instance, reading the chromium source code or mozilla for that matter, give you a lot of good input since both codebases are sufficiently complex and deal with a lot of layers and stacks.

albertgoeswoof 9y ago
How do you know a codebase is good before you invest time digging into it?
- W0lf 9y ago
  I don't. There are a few indicators though like the overall structure of the source files, how they are laid out on the file system. Minor details like inconsistent formatting. Typically if one or more of those indications is messed up, I'm pretty sure the rest of the codebase is also not in a good state. I still do skim through some files to confirm my expectation though.

quickthrower2 9y ago

I love reading code written in Elm. I always dive in and have a look. The reason I love it is that Elm is quite restrictive, there is a way of doing things and you can't deviate too much. Therefore reading someone else's code is usually pretty easy. In addition in Elm you can clone and run an Elm package and see the examples, do some "time travelling debugging" all within 30 seconds :-), again because there is one way of building and debugging things. No "Oh no browserify! I normally use webpack" or "WTF all that global npm shit I need to install" moment. Just once per computer: npm install elm Then do this git clone ... elm-reactor And open your browser to localhost:8080 I'd probably say Elm is more Zen than Python!

abcd_f 9y ago

How about some good _annotated_ code? :) http://www.kohala.com/start/tcpipiv2.html http://www.kohala.com/start/tcpipiv2.html - 15K line implementation of TCP/IP stack from BSD. Fantastic book, reads like a novel. http://www.pbrt.org/ http://www.pbrt.org/ - fairly detailed write up on building a pretty decent 3D renderer

poindontcare 9y ago

The only way to be sure you understand everything is to learn quantum mechanics perhaps string theory too just to be safe. But in practice modesty dictates we limit to learning as much as we can to get the job done.

markpapadakis 9y ago

Studying codebases is one of my rewarding hobbies. I ‘ve learned more from that practice( https://medium.com/@markpapadakis/interesting-codebases-159fec5a8cc https://medium.com/@markpapadakis/interesting-codebases-159f... ), than from most technical books I ‘ve read (by the way, most technical books aren’t particularly great).

svat 9y ago

Peter Seibel, the author of the Coders at Work book mentioned in the post, actually discussed this in a blog post: http://www.gigamonkeys.com/code-reading/ http://www.gigamonkeys.com/code-reading/ It's not quite true that none of the programmers interviewed in the book routinely read code for fun. In his blog post, Seibel mentions the exceptions: > First, when I did my book of interviews with programmers, Coders at Work, I asked pretty much everyone about code reading. And while most of them said it was important and that programmers should do it more, when I asked them about what code they had read recently, very few of them had any great answers. Some of them had done some serious code reading as young hackers but almost no one seemed to have a regular habit of reading code. Knuth, the great synthesizer of computer science, does seem to read a lot of code and Brad Fitzpatrick was able to talk about several pieces of open source code that he had read just for the heck of it. But they were the exceptions. The Knuth exception is notable: he does routinely read code for fun. In the book he mentions things like > I’ve got lots of collections of source code. I have compilers, the Digitek compilers from the 1960s were written in a very interesting way. They had their own language and they used identifiers that were 30 characters long but very descriptive, and their compilers ran circles around the competition at the time—this company made the state-of-the-art compilers of 1963 or ’64. And I’ve got Dijkstra’s source code for the THE operating system. […] I collected it because I’m sure it would be interesting to read if I had time. One time I broke my arm—fell off a bike—and I had a month where I couldn’t do anything much, so I read source code that I had heard had some clever ideas in it that hadn’t been documented. I think those were all extremely important experiences for me. And then a passage quoted in the post about how he reads code, with a Fortran compiler as example (not repeating here it's in Seibel's blog post: http://www.gigamonkeys.com/code-reading/ http://www.gigamonkeys.com/code-reading/ ), after which the interview (and the book) ends with Knuth saying: > don’t only read the people who code like you. (The examples in the book of people who last seriously read code years ago also interesting, e.g. Douglas Crockford mentions some programs he read, and there are multiple pages in the book about Guy L. Steele reading the TeX program.) Anyway, back to Knuth: I think that, the way he reads and digests code (or even papers: http://blog.computationalcomplexity.org/2011/10/john-mccarthy-1927-2011.html?showComment=1319546990817#c6154784930906980717 http://blog.computationalcomplexity.org/2011/10/john-mccarth...), it addresses a lot of the points the linked post makes. Even when “just reading” code, or anything for that matter, you are supposed to be doing the things the post mentions in “hacking versus passive reading”: active exploration, critical examination, synthesis. For example, Knuth read the code of ADVENT (aka Colossal Cave Adventure), and loved the code (not just the game itself) so much that he rewrote it to share with others to read, in his own preferred literate-programming (CWEB) style that matches the way he thinks (http://www.literateprogramming.com/adventure.pdf http://www.literateprogramming.com/adventure.pdf). This definitely doesn't sound like “passive reading” to me. Nevertheless, great post.

mattbierner 9y ago

On this spectrum, there's also "read by refactoring" (https://www.jamasoftware.com/blog/read-by-refactoring/ https://www.jamasoftware.com/blog/read-by-refactoring/). You'll need good refactoring tooling and good versioning (as you'll almost certainly want to discard many of your trial refactorings) but I've found that the approach helps me evaluate how my mental model matches up with reality. It also generally leads to a more readable codebase too

henesy 9y ago

One of the best parts of the Plan 9 system is that all of the kernel/command/library source is on hand and readily available to consult at any given time. Jointly, an important detail is that the complexity of said source is kept to a sane minimum and general style trends mean that most, if not all, of the source is formatted similarly and legibly. The system is so compact that you can keep most of the system source in your head at one time if you really need to.

henesy 9y ago
A fun tidbit from the Plan 9 compilers only really able to be found by reading the source: https://groups.google.com/forum/#!msg/comp.os.plan9/uMF7A4gkJsw/kNWBQ8U93LcJ https://groups.google.com/forum/#!msg/comp.os.plan9/uMF7A4gk...

jmadsen 9y ago

I had to use a js library for something today. npm install downloaded 113 new packages. I'm not even a javascript programmer; I'm saddled with being "full-stack" I'm supposed to read all that every time I need a new library? Or write it myself, when I have absolutely no idea how to build something that complicated, if I had time, negating the whole point of OSS in the first place?

aepiepaey 9y ago
The point made is to read code to become a better programmer. That doesn't mean you have to read all code you ever use.
akkartik 9y ago
OP makes no prescription about reading code. It merely points out what successful attempts to understand strange codebases have in common, and discusses what writers can do to help readers succeed. (I proof-read drafts of OP. And I have made prescriptions about reading code. But that's a separate story.)

superasn 9y ago

By corollary why it's also so important and terrifying to release open source projects too. You're oftening wondering who is gonna dig into this code of yours and think about your coding, esp. when you're not following the best practices and TDD.

Moodles 9y ago

In a similar vein, I often wonder just how many people actually check cryptographic algorithms. We'd all probably agree that it is good not to roll your own crypto algorithms, or even in implementation of known crypto such as AES. But how many experts in the world are there for AES and secure hashing? A lot of the major MD5 and SHA1 work all came from Marc Stevens' team. How many experts are there in this field, really? "Provable security" is another issue: Shoup, Menezes and Koblitz have all expressed concern about the current state of the art, I think.

simias 9y ago
I'm far from an expert in cryptography but I've had to dig into OpenSSL's codebase on more that one occasion and every time it left me with a deep sense of uneasiness. It's really not what I would deem good code: too many macros, too many potentially confusing API conventions (inconsistent return values in case of error, unclear resource lifetimes, ...), not enough comments and documentation... It's not the worst C codebase I've ever seen (far from it) but for something as critical I'd put the bar very high it terms of code quality and OpenSSL doesn't even come close.

antirez 9y ago

This is a bit western centric maybe, because I see Chinese developers reading tons of code, to the point that I receive an incredible amount of Redis PRs about conceptual bugs that can never happen in practice, since some Chinese developer is reading the code and doing the math in her/his head.

ralusek 9y ago
That's interesting, but also not so surprising. China has had decades of directing primary engineering/manufacturing efforts towards reverse engineering western products, or executing specific designs specified from western clients. The attitude of looking to someone else's engineering to learn from it is probably something the rest of us could do a bit more of.
heavenlyblue 9y ago
Any case you could share a link to one of these, if possible? A matter of curiosity.
- __s 9y ago
  First PR I saw on redis GH: https://github.com/antirez/redis/pull/4714 https://github.com/antirez/redis/pull/4714 That same user also made https://github.com/antirez/redis/pull/4685 https://github.com/antirez/redis/pull/4685 This may be? https://github.com/antirez/redis/pull/4622 https://github.com/antirez/redis/pull/4622 This was apparently critical, but also kind of an edgecase: https://github.com/antirez/redis/pull/4568 https://github.com/antirez/redis/pull/4568 & one more from someone else: https://github.com/antirez/redis/pull/4568 https://github.com/antirez/redis/pull/4568
- antirez 9y ago
  Probably soloestoy recently (but he also does a lot of Real world Redis) and Sun He in the past hold the records, but there are several...
stenecdote 9y ago
I wish one of these pull requesters would comment on this post! I'd love to hear how they're going about reading your code and whether they're actually using Redis or just reading it for enjoyment.

mar77i 9y ago

If you want to exercise reading concise and easily understandable standard C code, grab a toy project from suckless.org. Even though I write my code more defensively in some ways, that stuff is really straightforward to hack on.

alister 9y ago

Reading code just for fun was a thing in the early and mid years of UNIX (eg., 7th edition or System V). People eagerly passed around faint 10th generation photocopies of Lion's printout of and commentary[1] on the UNIX source code. The annual USENIX conference had a popular short course in which they went through the entire UNIX kernel line by line. Why was that a thing back then and not now? Certainly UNIX was an amazing piece of work, but there are amazing works today -- the modern browser for example. I think that the difference is that the totality of the UNIX kernel was comprehensible for a single mind. The UNIX kernel in 1983 was less than 20,000 lines of code, almost all in C, and more than 75% was not machine-dependent.[2] The amazing software of today, something worthy of reading for fun, is literally millions of lines of code, written in multiple languages you don't know, and evolving so fast that whatever you read might be completely different a few months later. There's no joy in knowing a tiny part of that. It would be like reading 10 pages from the middle of a novel, a novel so big that you know you'll never finish it. [1] https://en.wikipedia.org/wiki/Lions%27_Commentary_on_UNIX_6th_Edition,_with_Source_Code https://en.wikipedia.org/wiki/Lions%27_Commentary_on_UNIX_6t... [2] https://en.wikipedia.org/wiki/History_of_Unix#1980s https://en.wikipedia.org/wiki/History_of_Unix#1980s

pm215 9y ago
I suspect in a lot of cases reading the Lions commentaries was not purely for fun. The original audience (Lions' students) were doing it as part of their university course. Many of the contemporary readers would have been reading it because they needed to understand, fix bugs or add features to their unix systems (I think one of the introductions in the reprint book is from somebody who was in this position). But you're right I'm sure that the the key was the combination of (a) real world and genuinely useful code that the readers were using (b) size such that you could realistically understand all of it (c) excellent accompanying code commentary. I read through the Lions book in the late 1990s when the reprint came out, essentially "just for fun" -- it remains probably the only sizeable piece of code I've read for fun.
Klathmon 9y ago
This is still done today, just with other software. I've read blog posts that go through the entire redux javascript library line by line, or review an entire chunk of React. Even just yesterday here on HN was a submission that just contained shaders from Wolfenstein.

z3t4 9y ago

I read the source code for the libs all the time, mostly to see what methods and properties they expose. I learn something new every time. This is JavaScript/NodeJS. I don't miss compiled .dll's

INTPenis 9y ago

I love to dig through peoples code. Unfortunately I know myself enough to know that it can be a giant time sink if I let myself go. In swedish there's a phrase called "snöa in", to get snowed in literally, and that's what happens to me. So after enough years in the biz I try to hold myself back from this habit.

chukye 9y ago

Once I propose for a team to read the code of Redux before adding it to the codebase (if you don't know, redux is a very small libraries, very easy to read). But no one seemed to care, they was just saying things like "Well, I'll read this blog post that teach us how to use it", and I was like: "BUT, if you read the code You will know exactly how to use it! Why not go to the source instead of read other people opinions?!"

mannykannot 9y ago

I must disagree with this sentiment, though I understand where it is coming from: "Clean, solidified abstractions are like well-marked, easy-to-follow paths through a forest — very useful if they lead in the direction we need to go, but less useful when we want to blaze arbitrary new paths through the forest." Using the same analogy, if you want to blaze a new trail through the woods, you need a map of the territory, and a map for this territory takes the form of higher-level abstractions with clear, explicit semantics and minimal but effective, and unambiguous, interfaces. One of the biggest problems in modifying or reusing existing code is the risk of breaking higher-level consistency, for example by violating an implicit constraint that is necessary for correctness (the biggest Ethereum errors have been of this form, where the implicit assumptions included when and how often initialization is performed, and that a certain library will be present.) If the clean code movement has been delivering code that works as step-by-step instructions for getting from A to B, but not as a map, it may be because breaking code down into small functions and classes is easier, more visible and more measurable than making coherent abstractions with consistent and minimal interfaces at a higher level.

stenecdote 9y ago
To push back on this point a little bit, I think we agree that a map is one of the most useful things you can have but disagree on what the right type of map is. While I'm not anti-abstraction, I think people often impose poorly chosen high-level abstractions on top of messy lower-level components (akkartik calls this Authoritarian High Modernism in a comment (https://lobste.rs/s/gtxi5y/nobody_s_just_reading_your_code_how_we#c_0joqn2 https://lobste.rs/s/gtxi5y/nobody_s_just_reading_your_code_h...) on Lobsters). This approach leaves readers having to understand a bad abstraction and its internals, a map, its territory, and inconsistencies between the two. As I understand it (please correct me if I'm wrong), you want people to do the work to find better high-level abstractions. I agree that it would be good if we could do this more often. I just also think we can find other ways to give people maps that work with our existing, messy codebases that don't have good, minimal abstractions already.
- mannykannot 9y ago

fghtr 9y ago

https://www.fsf.org/blogs/community/who-actually-reads-the-code https://www.fsf.org/blogs/community/who-actually-reads-the-c... "Two-and-a-half months later I received an email from someone who not only managed to find the comment, but also managed to guess the code had to be rot13'ed."

jesperlang 9y ago

Reading code always felt awkward, slow and ineffective. The cognitive load is huge, you have to compute some things while memorizing others and at the same time keep track of flows. Thats why I always have pen and paper while diving into a new big code base, you simply can't keep it in your head. But something we forget often is... the goal of reading code is almost always to understand a system or some part of it. And reading code for me is a terrible way to do it. Code is just one level of abstraction and one way of seeing things. We should have multiple ways of "reading" a system depending on what level we want to understand it. No, I don't think that rushed, out of date "system documentation" you wrote will do. I still think this is not solved in a satisfying way. We could find inspiration from things that works well, just imagine how google maps lets you look at the world at many different levels, from continents down to street level. Also it provides different views of each level, street maps are useful for tourists, terrain maps for hikers and satellite images for someone who want to study vegetation. How you will "read" a system depends on what you are trying to understand.

stenecdote 9y ago
As the author of the post, just wanted to reply and say I'm totally open to the notion that the solutions I proposed will not solve the problem. I'm much more sure about the problem than I am about the solution. A google maps zoom system for code sounds awesome, although I have no idea how that might work.
- lugg 9y ago
  Brackets inline editor comes to mind. Just replace CSS definitions with function definitions (I think it might already do this for ?) The zooming functionality you're thinking about probably wouldn't be very practical. (Zooming in down to opcode or out to high level framework calls.) But simply being able to inspect definition inline would do.
douglaswlance 9y ago
I find it useful to write comments about what's happening in the code as I am trying to figure it out.

mark_l_watson 9y ago

One thing that helps me make reading code in code reviews less passive is to add occasional temporary print statements and then run the unit tests again that cover the code I am reading. Seeing variable values satisfies some curiousity. Sometimes I will use a debugger to step through code I am reviewing but for some reason I prefer print statement to satisfy any curiousity I have about how the code works.

djhworld 9y ago

The thing about code reading for me is you have to have decent tooling. It's pointless reading code in Github, especially for a codebase you are not familiar with, you need to get the codebase open in an IDE to be able to dive in quickly and back again to build the map out in your head. I don't think you can read a portion of code from top to bottom without having tooling....unless the piece of code is really short and just one or two files

wkz 9y ago
The one online tool that I use for this kind of work is Woboq Code Browser. It's only C/C++ projects and not many are freely available. The Linux kernel is though and I spend a lot of time in that code base. glibc, gcc and llvm are also available. https://code.woboq.org/ https://code.woboq.org/
- guruz 9y ago
  Which projects would you like to see on code.woboq.org? Maybe we can add them.
karmakaze 9y ago
Absolutely this. You can always tell a PR reviewed via web. Unless it's very localized, I always open up the branch in an IDE and follow the flow through the abstractions and see if the changes make sense, and if there are additional opportunities or less clear gray areas of operation. It will be even better when web tools catch up. Could be even better if they can navigate across repos in different languages even without even checking out a single file.

your-nanny 9y ago

Do I sit back in my comfy chair on an evening and just read through some code repo? Well, actually, yeah, I do... Though not terribly often. Do I sit back and read through programming bloggers' explaining their code? Yes, pretty often, and based on HN, that's pretty true of a lot of programmers. Still I'm more likely to find myself browsing wikipedia for math and cs

gwbas1c 9y ago

Reading code is difficult and time-consuming. To really understand a function you need to try it with different inputs, either with interactive testing or in a debugger; AND you need to know what the functions it calls does. The second part, knowing what the code the function is calling does, is nearly impossible when they are other functions in the program itself.

kruhft 9y ago

Somebody read my code once and commented on my improper use of 'mod' or something like that. It then turned into a long discussion on how they might be incorrect, and so on and so forth on Reddit, back and forth on Lisp. I love Lisp. It's a great language with great programmers.

OutThisLife 9y ago

Best way to figure out someone else's codebase, IME, is to just dive in heavy and start breaking things. Figuring out what's connected to what and why by removing chunks of code that you need to understand.

Stwerner 9y ago

I've been thinking about this idea for a while and love the reimplementation idea. I really want to start live-streaming myself coding (or pair programming) and have been trying to come up with some ideas that would end up being interesting to watch. One idea was taking a popular open source tool with a good test suite and trying to get the tests to pass, but I think checking code out at a point before a new feature is implemented and reimplementing it could end up being really interesting. Could even make it more interactive by sharing the checkout point so viewers can do an implementation themselves and doing a stream walking through different interesting implementations.

pjungwir 9y ago

I agree most programmers are too quick to write before they read, but I'm not sure why it has to be "for fun". Reading code takes effort, just like understanding any new complex thing. The primary virtue here is patience. It's a lot like debugging actually: the goal is to understand. Setting aside the "for fun", of course I read my co-workers' code, but also I read my dependencies' code quite often. In particular when you have something that is meant to be extended, the docs never completely cover all the use cases, and the quickest way can be to start reading. Some examples from my own recent work are reading code from the Ruby gems devise, devise_token_auth, and spree, and also reading Django class-based views (all things meant to be extended). One trick I've learned in Ruby is `cd $(bundle show spree_core)`. From there you can grep, find, read, even add your own `puts` or whatever. Actually I have been reading some code for fun: Postgres! I have an extension that adds temporal foreign keys, and I'm slowing porting it from plpgsql to C. Reading the code for standard foreign keys is very helpful. Same thing for a lot of other extensions I've written in the last few years. It has helped me a ton to read others' extensions and the core code.

Kagerjay 9y ago
This is what I've been working on for the past few months. I've been writing a lot of spaghetti code in the past and come back to find how poorly I organized it. I'm suppressing the urge to immediately write code, but rather, look at the big picture first, digesting the information, even if it means prolonging writing anything for a day or several hours. I usually write pseudocode first before doing anything now. Either on paper, as code comments, or as a list. Sometimes I find myself writing tests on paper as well, it helps out a lot too. A programming language is just a means to an end, what they all have in common is a set of logic to follow. Then I debug. I just fork the code, comment things out, add a little bit of functionality, etc. When I have a grasp on it write some TDD I find the easiest codebase that I refer a lot is the todomvc, or any similar ones that have different implementations (differnet languages) for the same general solution. All the core logic between languages is mostly the same, so its easy to dig through and understand a different programming language / how its generally organized. One thing I remember learning about from experience polyglot programmers. Its extremely helpful to have an existing codebase you've written (such as todoMVC), and using that same example / porting it over in a different language to learn that languages nuances.

dmurthy 9y ago

A few years ago in my previous company there was a major push to convert a lot of the codebase to Python. A byproduct of this conversion was an easier way to build UI using predefined widgets. Yet no one was using this functionality due to the mindset of years of TCL. I honestly hated TCL. So I took it upon myself to explore the entire new code base and go about building UI panels. Initially I just couldn't understand how the code mapped out until one day I just printed out some of the code and read it repeatedly till it made sense. The whole exercise was an eye opener.

TheMagicHorsey 9y ago

I read code to learn all the time. And I don’t get paid to program.