14 ms·

Grokking big unfamiliar codebases

149 points by ibobev 4y ago

rapfaria 4y ago
You can send this list to everyone, and people will still struggle. Sometimes I'm just amazed at some folks that join the org and are able to finish tickets in no time. Other times, it doesn't matter how much help you provide: they start to get productive in one part of the system, but moving to another component is a challenge. Without help, they are incapable of advancing. > Let your instincts guide you if you think something feels much more difficult than you’d expect, drawing from prior experience as necessary. I wonder if this is the knack that so few software engineers possess, and I'm not talking about the the 10x bunch.
- Swizec 4y ago
  I think I have the knack. I don’t think it’s anything special. Read the code. Believe the code. Not what you think it’s doing, not what the comments imply, not even what variable/function/class names imply. What is the code actually doing? Great, now you can fix it. And remember: when you’ve eliminated the impossible, whatever is left, no matter how weird, is what the code is doing. Don’t be afraid to console log. I see all the time engineers wasting minutes and hours trying to reason about code when a good print statement could answer their question in seconds.
  - Jensson 4y ago
    I don't think many can read code efficiently, so people who can do that are special. People who can't have to rely on documentation or slowly trudging through.
    - Swizec 4y ago
      You are right. I think that part comes with experience. I’ve been coding my whole life (since 9 yo, am 35 now) so a lot of this stuff is very deeply ingrained.
    - xupybd 4y ago
      There is a mix. Some things you can learn from the code other things you need documentation. Often it's the domain knowledge that is hard to glean from the code. Especially when bug fixing. For example: The domain expert told me this output was wrong but the code appears to be doing exactly what it claims to. Trying to get a domain expert to set down and explain why the output is wrong is a skill that took me way to long to develop. Now sitting down with a pen and paper running through the logic with an expert is one of my most treasured information gathering techniques.
  - BurningFrog 4y ago
    One way to be sure what the code is doing is writing unit tests. Just dumb black boxing is fine: `assert f(0) == 0`. When it returns "Pancake", change to `assert f(0) == "Pancake"` ...repeat until you understand :)
  - Zanfa 4y ago
    > Read the code. Believe the code. Not what you think it’s doing, not what the comments imply, not even what variable/function/class names imply. What is the code actually doing? 100% agree, but in my experience there's one more step after this. What should the code be doing? If you're lucky, you have tests or human-readable documentation that explains business rules, but in legacy applications, odds are the only way to know is find an enduser that possesses the knowledge. These are the worst time sinks.
- Jtsummers 4y ago
  There seems to be a large contingent of people who learn almost exclusively by rote (this is not just in software engineering). A major problem they have is generalizing that knowledge to other systems or other parts of the same system. See the many people who struggle to learn a new programming language. I have worked with people who were baffled by *nix systems (especially the terminal interfaces) because their comprehension of "folders" or "directories" was a strictly graphical one (usually based on Windows' model, but they could usually adapt to another file browser at least). They even struggled to interact with filesystems programmatically as a consequence. I've had some success helping people break out of this, but it usually boils down to individual motivation. These are the people I struggle to work with (in a mentoring/teaching capacity) the most because they get extremely frustrated by variations and novelty. Give them a well-written procedure, though, and they're usually good. Just hope there aren't any knowledge assumptions you made in writing it that don't hold.
rambleraptor 4y ago
Aside - where was the diagram made?
- Jtsummers 4y ago
  Maybe not the case, but often that style was made using Excalidraw: https://excalidraw.com https://excalidraw.com. Playing around in it, the font at least appears the same.
  - yardshop 4y ago
    Haha, I had the same thought about the program and the opposite conclusion about the font! Many characters like g and p look similar, but the lower case t has a curl on the bottom in Excalidraw, and is straight in the article's diagram. Edit: on second look, it is the same t. I saw some images in a search for Excalidraw with a different t, but when I tried the actual site it was the same. Sorry for the noise!
    - Jtsummers 4y ago
      I'll reply here (because I simultaneously replied to your other comment but removed it). I also thought it was funny we had come to opposite conclusions based on the same data. Type out "Bottom-up" in excalidraw, it matches perfectly. The lower case `t` has a curl on the bottom in the diagram too. EDIT: Re Edit: No worries about noise. I had a good chuckle with our near simultaneous replies which was nice after the way our project has been going this year.
- yardshop 4y ago
  Looks like something one could make pretty easily with Excalidraw (https://excalidraw.com/ https://excalidraw.com/) although the font seems slightly different.
  - deleted 4y ago
    [deleted]
xupybd 4y ago
I hate the experience of a new code base when everything is so alien that at first it seems horrible. Then when you learn the ideas they're using you start to like the way everything is put together. Then you start finding navigation really easy. You're able to make changes quickly and can be sure they're safe. Then that project ends and start again. Each time it get easier as you see the same patterns but so much time gets sunk into learning something that you'll never see again.
- capableweb 4y ago
  > I hate the experience of a new code base when everything is so alien Personally I love that experience, of a fresh new codebase with some patterns and other things I haven't seen before, seeing what works well and what doesn't, what stinks and what shines. Once the exploration have been done, and things start to be obvious on how to achieve, I lose a bit of motivation as I can solve the problems in my head and now the boring part of actually typing it out begins.
  - xupybd 4y ago
    Ha, I wish there was a way we could split the labor on that. You do the learning, somehow transfer that to me and I do the bug fixing. I love fixing up a legacy system but don't enjoy the first few weeks of feeling unproductive. That has more to do with the pressure to get things done than the actual exploration.
    - Jtsummers 4y ago
      I do that with my teams. I have a technique where I turn existing code bases into literate programs. Largest I ever did this on was around 250k SLOC, but that didn't take me too long (you get fast with it if you practice). The result was a set of documents that I turned into presentations and diagrams (better than autogenerated ones) on the system to share with the team. I'll be doing that with one part of our system starting tomorrow, actually, because no one understands it and everyone is afraid of it (there is one automated test which is a massive end-to-end test that exercises, theoretically, every capability and takes 2 hours or so to run). The underlying code and repo are unaltered, this is a collection of org files that sit in tandem. I run org-tangle to verify that I haven't actually altered the code in my various adjustments to the literate program version. So people still see their regular cpp or whatever files and don't have to learn the tools I use (though I share if they ask, no one has ever asked). But they get the benefit of understanding the programs better. EDIT: By "didn't take me too long", it still took a few weeks. But it was a new-to-us system that the original contractor provided sans tests (there were directories and references to numerous automated tests that we weren't given and they wouldn't provide) and with useless auto-generated UML diagrams as "documentation". But a few focused weeks was probably a lot better than a few years of confusion and frustration.
deleted 4y ago
[deleted]
beyang 4y ago
This is a fantastic post on the challenges associated with diving into large codebases (sometimes called "source diving" or "code archaeology"). The vast majority of code in the world is in large private codebases. The size of legacy private code dwarfs that of the open source world (sizable as that might be these days) and the rate at which you can read/understand code is the limiting factor here for dev velocity. Some companies have invested in dedicated tools (like Code Search and Grok at Google) because they recognize the importance of the problem. Such systems were a primary source of inspiration for us when we started Sourcegraph—we wanted to bring the tech for large-scale code understanding to every company. In-depth posts like this are a big source of inspiration and ideas for us, so thanks to the author for writing this up.
yazaddaruvala 4y ago
Most of this makes sense, but ends up not being followed for non-technical reasons like the person's confidence, etc. I'd say, just know two things: 1. (From the post) "Let your instincts guide you if you think something feels much more difficult than you’d expect", and talk to your team about it, and change it for them. 2. There is a LOT of dead (unreachable / redundant) code in any enterprise codebase. Especially recently launched flag that haven't been "cleaned up yet". Just start cleaning this stuff up. Even the very obvious stuff gets you very well introduced to the codebase. Where I disagree with the post: > Taking notes is non-negotiable for me, not strictly as a memory aid, but as a means to communicate findings and verify them later. If you need to take a 2 line note, you're better off having written a 2 line code review. Even something simple like "metric.add("DOES_THIS_HAPPEN?", 1);" can be done with 0 confidence / testing or with minimal-confidence / minimal-testing you could instead write "Preconditions.checkState(..., "Will this be caught by our integration tests?");". The best notes are the ones that get pushed to production to validate assumptions.
- fsckboy 4y ago
  a person who doesn't understand a codebase shouldn't be adding "does this happen" all over the place. run in a debugger and set a breakpoint.
  - tejohnso 4y ago
    Agree on the first part but there are going to be plenty of cases where running in a debugger isn't sufficient to satisfy one's self that something never happens. Unless we're talking about "does this happen under this specific scenario I know about".
    - fsckboy 4y ago
      if something never happens, that's not a part of the codebase you need to understand right now. If your task is cleaning up, sure, but your task here is learning what happens, not what doesn't happen.
nerdponx 4y ago
I wonder if we can use the vector representations from the inner layers of LLMs to create some kind of semantic code and text analysis tools. Maybe such a tool can highlight salient areas, and tag regions of code as "boilerplate", "standard idiom in this language", "business logic", "performance optimization", et alia. As for taking notes, is there a IDE (or better yet a Vim plugin) that lets me annotate specific lines or regions with my own notes and comments? Or even annotate functions and classes that way, so I can see those comments in a pop-up box whenever I hover over an annotated function.
andrewstuart 4y ago
When working in a big unfamiliar code base, I: * have a goal to do something like, save some image data for example * just dig around trying to find where the right place is to get the data I need * hack in code to see if I can get the damn thing to work at all * once its working, try to do do it neatly if I can * don't worry too much about not understanding everything - it can take months to work out how a system works. This only comes through doing lots and lots of iterations of hacking in little fixes and along the way learning what you can
x86x87 4y ago
Just ask chatgpt to summarize it. How hard can it be?
- andrewstuart 4y ago
  It can't do that yet but I'm really excited about the day coming when AI can understand an entire code base. ChatGPT has already completely revolutionized programming for me, I loove forward to further advances. It's truly unbelievable, like have a permanent pair programmer with me who I can ask how things work, how to fix problems, and more importantly than anything else, ask for examples of how to do things, which I then hack into my code.
  - x86x87 4y ago
    I too look forward to machines rewriting my code and telling me I'm not smart enough to understand it when I object to their questionable output. I can see a 0.1x PeonGPT telling me that it got a ShipIt from a 10x TeamLeadGPT and we are moving forward with the code. Any concerns? Create a jira ticket which JiraGPT will promptly label as "tech debt" and will throw into the "backlog" for it never to look at it again.
- worik 4y ago
  A whole new take on `make world`
furyofantares 4y ago
I think something like Github Copilot can be a huge boost to getting up to speed in big unfamiliar codebases -- it will autocomplete what you're trying to do with lots of "unknown unknowns", patterns and APIs in the codebase that you don't yet know about.
- potatoman22 4y ago
  I've been thinking about LLMs' ability to help with this. If they really did "memorize" data as well as some people think they do, it'd be a great boost to enterprise software development.
eyelidlessness 4y ago
My two big pieces of advice (which I’ve also given to team members growing into senior roles): 1. Learn how and when to step into “external” code when debugging. How being mainly oriented around navigating call stacks that might be obfuscated by default, while establishing pattern recognition for cases where problems in your code show up in third party call sites versus simple cases of misconfiguration. 2. Sorry but… learn regex. Even if your language is static everything, you’re probably going to hit a wall using code navigation tooling. If you know the general format of things you’re looking to find and call stacks aren’t getting you there, regex is an incredibly powerful way to find what you’re looking for. Don’t rely on it for aggressive mass changes, but definitely get comfortable with it for finding potential root causes.
- worik 4y ago
  > Sorry but… learn regex. Do not be sorry. find and grep. How is it possible to exist without them?
  - eyelidlessness 4y ago
    I’m not really sorry, but people really hate regex. For reasons I comprehend but I use them on the job pretty much daily.
knlb2022 4y ago
Huh, I wrote about this a few months ago too (with a lot more whimsy): Eating Elephants -- https://docs.google.com/document/d/1c07-Zj6bUbYPwx7Zttd1N74oG2Ylk2EHPKpkUmdcsfw/ https://docs.google.com/document/d/1c07-Zj6bUbYPwx7Zttd1N74o... The thrust of both our articles is similar, except I think I ended up using many more words.
gorgoiler 4y ago
It is incredibly important to avoid putting tag-firebreaks in the code base — anything that stops a new developer from being able to navigate from symbol to definition, or from definition to all uses of that function. A couple of examples: ACTIONS = { “swim”: Swimmer, “skate”: Skater, } # …pages of code… def f(action, *args): … mod = ACTIONS[action] mod(*args).do() At face value, this pattern doesn’t do anything particularly wrong but anyone who arrives at f on their code safari won’t see any symbols for either Swimmer or Skater. It breaks their flow the same way switching languages might do. Sure, ACTIONS is only a hop away, but that’s a level of indirection that could have been avoided. Even worse, what if the class name was derived from a string? The fact that the action implementations are chosen at runtime also makes life a bit awkward but the primary issue is not being able to immediately see the action classes and jump to their implementations. Another one is calling shell scripts. We all have projects, I’m sure, where we have to shell out to some common legacy tool: def f(): run(“legacy.sh”) def g(): run(“legacy.sh”) Wrapping the script in a native function does wonders for making it easier to find, especially for the poor, sacred soul who deprecates it from the codebase one day: def f(): run_legacy() def g(): run_legacy() # …pages of code… def run_legacy(): run(“legacy.sh”) Any unfriendly pattern that breaks tag-hopping / IDE-integration is yet another small chafing of your colleagues‘ productivity. Think twice before you introduce that YAML file which defines behavior, instead of implementing it natively — your friends (and their jump-to-definition muscle memories) will thank you.
- worik 4y ago
  Really good. Think about the programmer reading this at write time. How to write large code bases that can be understood, every bit as important as knowing how to read them
  - Denzel 4y ago
    Software Engineering at Google has lots of valuable advice and best practices to accomplish this - https://abseil.io/resources/swe-book/html/toc.html https://abseil.io/resources/swe-book/html/toc.html
- 4y ago
psadri 4y ago
My first step is to learn the data model for the application. Then everything starts to make sense as code to read/write/present that model.
- worik 4y ago
  Yes. If there is a coherent data model.
jcranmer 4y ago
I've long had a goal to go through a similar explanation of how to do this, and I figured the best way to do this was to pick a large project whose source I had never seen before and record myself fixing a bug in it. This plan is foiled by the difficulty of finding large projects whose source I had never perused before just for fun, especially in my core area of competence. But some salient points I think I can make is: * A lot of people have pointed this out, and I think it's the most important thing: don't go into the project asking the high-level question "how does this work?" No, you want to start by figuring out something specific. If you're picking apart a command-line tool, for example, start by asking where one specific subcommand is implemented, and what it actually does. Once you've navigated yourself to such a starting position, you can start building a mental map from that particular location, and expand it to higher or lower levels of detail from there as necessary. * Use code search (e.g., grep) liberally, and start guessing for the keywords you want to search for in the code to reach your next piece of information. In my previous example of looking for a subcommand implementation, searching for a probably-unique part of the help message is a good way to guess where the code might be located. * If you don't understand when code fires, something like assert(false && "Did I happen?"); is useful for figuring out examples that execute a particular codepath (assuming, of course, that your codebase has a competent testsuite). Now you have tests that you know execute that code path, and you can start using a debugger to probe what program state looks like at that point. Also, get very acquainted very early in the process how to satisfactorily dump out that state. * Play the git blame game. Look up the history of the code you don't understand--track down the exact revision that added that code, and then go spelunking in code review or bug tracker history to understand why it came to be. If code seems to no longer be used, and you don't understand why, play the reverse blame game--look at where it was added, where it was used when it was added, and find where that code was removed, and (as above) go spelunking in more detail to understand why it came not to be. * Practice, practice, practice! While I don't think these are difficult skills to acquire, it's hard to talk about this in generalities because so much of the process relies on your experience knowing what things generally should be like as you try to match up the codebase to your (possibly incorrect) mental models of how it works.
- wpietri 4y ago
  > don't go into the project asking the high-level question "how does this work?" No, you want to start by figuring out something specific Should this be an or? For me, understanding big code bases has always been a pendulum between the big picture and details. Survey the overall scope, and then pick some very specific thing and examine it all the way through. Then go back to the big picture, integrate what I've learned, and come up with another very specific thing to investigate. For me this cycle is especially fun when on the trail of a bug. Get the big picture, and then dive in on a specific bug and beat it to death. Then ask what he bug told me about not just the code base and the architecture, but the development practices and the company culture. Pick another bug and run it to ground, seeing which of my notions were validated and which were challenged. Eventually, produce recommendations for broad, long-term changes. It's been a while since I did a gig like that, but they're such a joy when I get the chance.
helsontaveras18 4y ago
This sounds like an extremely fun job (or jobs?), how can I get it?
sandkoan 4y ago
I've been working on a semantic search engine for code that's particularly useful to those navigating unfamiliar codebases. Just write an English description of whatever you're looking for, and it'll show you the most relevant/similar functions from the repository. Afterall, it's often useful to see how things are done, and why. Search at least makes possible the former. Email me at govind <dot> gnanakumar <at> outlook <dot> com if you'd like to try out a beta—on account of potentially being hugged to death, I don't want to publicly display the link.
aurelius83 4y ago
One of the best pieces of writing I've come across that I still come back to about grokking complex codebases was written by Mitchell Hashimoto. https://mitchellh.com/writing/contributing-to-complex-projects https://mitchellh.com/writing/contributing-to-complex-projec...
zknill 4y ago
I work in a large java code base, where I'm often exposed to brand new code (written a while ago, or written by other teams). I built this tool to help me understand the unfamiliar: https://packagemap.co https://packagemap.co There's a parser that extracts all the classes and methods from the java code and a site that renders these in a directed graph. I use a kind of query syntax to filter the graph, and added features based on how I want to explore the code. The nodes in the graph carry an name like: "my.company.package.MyClass.myMethod" Then I can filter by wildcards: *MyClass* Or see just the classes with a line terminator: *.MyClass$ Or see the code that leads up to, or away from a method/type: ->*MyClass, *MyClass-> I've found making the problem visual has massively improved how I navigate the code. In code review I spot things that are very hard to see in an IDE or file-tree review UI (like github). Things like; where abstractions are being broken/leaking, or where types are being mis-used across packages. I wrote a summary of the common cases I find using this tool in a blog post[0] [0] https://blog.packagemap.co/posts/common-code-coupling-mistakes/ https://blog.packagemap.co/posts/common-code-coupling-mistak...
- aloer 4y ago
  I suggest you position the free tier more prominently on your pricing page. Took me a while to realize it's even there
  - zknill 4y ago
    Yes! Thanks
xtiansimon 4y ago
This is parallel topic—how do you grok an unfamiliar API? You have API access and a blank piece of paper, what are the first three things you’d write down?
- jrvarela56 4y ago
  No source, no docs? I'm assuming some app uses it: connect my phone to a proxy (I use mitmweb to inspect), close all apps except that one, and tinker with the app to see what requests come up.
effnorwood 4y ago
[dead]

149 points by ibobev 4y ago

You can send this list to everyone, and people will still struggle. Sometimes I'm just amazed at some folks that join the org and are able to finish tickets in no time. Other times, it doesn't matter how much help you provide: they start to get productive in one part of the system, but moving to another component is a challenge. Without help, they are incapable of advancing. > Let your instincts guide you if you think something feels much more difficult than you’d expect, drawing from prior experience as necessary. I wonder if this is the knack that so few software engineers possess, and I'm not talking about the the 10x bunch.

Swizec 4y ago
I think I have the knack. I don’t think it’s anything special. Read the code. Believe the code. Not what you think it’s doing, not what the comments imply, not even what variable/function/class names imply. What is the code actually doing? Great, now you can fix it. And remember: when you’ve eliminated the impossible, whatever is left, no matter how weird, is what the code is doing. Don’t be afraid to console log. I see all the time engineers wasting minutes and hours trying to reason about code when a good print statement could answer their question in seconds.
- Jensson 4y ago
  I don't think many can read code efficiently, so people who can do that are special. People who can't have to rely on documentation or slowly trudging through.
  - Swizec 4y ago
    You are right. I think that part comes with experience. I’ve been coding my whole life (since 9 yo, am 35 now) so a lot of this stuff is very deeply ingrained.
  - xupybd 4y ago
    There is a mix. Some things you can learn from the code other things you need documentation. Often it's the domain knowledge that is hard to glean from the code. Especially when bug fixing. For example: The domain expert told me this output was wrong but the code appears to be doing exactly what it claims to. Trying to get a domain expert to set down and explain why the output is wrong is a skill that took me way to long to develop. Now sitting down with a pen and paper running through the logic with an expert is one of my most treasured information gathering techniques.
- BurningFrog 4y ago
  One way to be sure what the code is doing is writing unit tests. Just dumb black boxing is fine: `assert f(0) == 0`. When it returns "Pancake", change to `assert f(0) == "Pancake"` ...repeat until you understand :)
- Zanfa 4y ago
  > Read the code. Believe the code. Not what you think it’s doing, not what the comments imply, not even what variable/function/class names imply. What is the code actually doing? 100% agree, but in my experience there's one more step after this. What should the code be doing? If you're lucky, you have tests or human-readable documentation that explains business rules, but in legacy applications, odds are the only way to know is find an enduser that possesses the knowledge. These are the worst time sinks.
Jtsummers 4y ago
There seems to be a large contingent of people who learn almost exclusively by rote (this is not just in software engineering). A major problem they have is generalizing that knowledge to other systems or other parts of the same system. See the many people who struggle to learn a new programming language. I have worked with people who were baffled by *nix systems (especially the terminal interfaces) because their comprehension of "folders" or "directories" was a strictly graphical one (usually based on Windows' model, but they could usually adapt to another file browser at least). They even struggled to interact with filesystems programmatically as a consequence. I've had some success helping people break out of this, but it usually boils down to individual motivation. These are the people I struggle to work with (in a mentoring/teaching capacity) the most because they get extremely frustrated by variations and novelty. Give them a well-written procedure, though, and they're usually good. Just hope there aren't any knowledge assumptions you made in writing it that don't hold.

rambleraptor 4y ago

Aside - where was the diagram made?

Jtsummers 4y ago
Maybe not the case, but often that style was made using Excalidraw: https://excalidraw.com https://excalidraw.com. Playing around in it, the font at least appears the same.
- yardshop 4y ago
  Haha, I had the same thought about the program and the opposite conclusion about the font! Many characters like g and p look similar, but the lower case t has a curl on the bottom in Excalidraw, and is straight in the article's diagram. Edit: on second look, it is the same t. I saw some images in a search for Excalidraw with a different t, but when I tried the actual site it was the same. Sorry for the noise!
  - Jtsummers 4y ago
    I'll reply here (because I simultaneously replied to your other comment but removed it). I also thought it was funny we had come to opposite conclusions based on the same data. Type out "Bottom-up" in excalidraw, it matches perfectly. The lower case `t` has a curl on the bottom in the diagram too. EDIT: Re Edit: No worries about noise. I had a good chuckle with our near simultaneous replies which was nice after the way our project has been going this year.
yardshop 4y ago
Looks like something one could make pretty easily with Excalidraw (https://excalidraw.com/ https://excalidraw.com/) although the font seems slightly different.
- deleted 4y ago
  [deleted]

xupybd 4y ago

I hate the experience of a new code base when everything is so alien that at first it seems horrible. Then when you learn the ideas they're using you start to like the way everything is put together. Then you start finding navigation really easy. You're able to make changes quickly and can be sure they're safe. Then that project ends and start again. Each time it get easier as you see the same patterns but so much time gets sunk into learning something that you'll never see again.

capableweb 4y ago
> I hate the experience of a new code base when everything is so alien Personally I love that experience, of a fresh new codebase with some patterns and other things I haven't seen before, seeing what works well and what doesn't, what stinks and what shines. Once the exploration have been done, and things start to be obvious on how to achieve, I lose a bit of motivation as I can solve the problems in my head and now the boring part of actually typing it out begins.
- xupybd 4y ago
  Ha, I wish there was a way we could split the labor on that. You do the learning, somehow transfer that to me and I do the bug fixing. I love fixing up a legacy system but don't enjoy the first few weeks of feeling unproductive. That has more to do with the pressure to get things done than the actual exploration.
  - Jtsummers 4y ago
    I do that with my teams. I have a technique where I turn existing code bases into literate programs. Largest I ever did this on was around 250k SLOC, but that didn't take me too long (you get fast with it if you practice). The result was a set of documents that I turned into presentations and diagrams (better than autogenerated ones) on the system to share with the team. I'll be doing that with one part of our system starting tomorrow, actually, because no one understands it and everyone is afraid of it (there is one automated test which is a massive end-to-end test that exercises, theoretically, every capability and takes 2 hours or so to run). The underlying code and repo are unaltered, this is a collection of org files that sit in tandem. I run org-tangle to verify that I haven't actually altered the code in my various adjustments to the literate program version. So people still see their regular cpp or whatever files and don't have to learn the tools I use (though I share if they ask, no one has ever asked). But they get the benefit of understanding the programs better. EDIT: By "didn't take me too long", it still took a few weeks. But it was a new-to-us system that the original contractor provided sans tests (there were directories and references to numerous automated tests that we weren't given and they wouldn't provide) and with useless auto-generated UML diagrams as "documentation". But a few focused weeks was probably a lot better than a few years of confusion and frustration.

deleted 4y ago

[deleted]

beyang 4y ago

This is a fantastic post on the challenges associated with diving into large codebases (sometimes called "source diving" or "code archaeology"). The vast majority of code in the world is in large private codebases. The size of legacy private code dwarfs that of the open source world (sizable as that might be these days) and the rate at which you can read/understand code is the limiting factor here for dev velocity. Some companies have invested in dedicated tools (like Code Search and Grok at Google) because they recognize the importance of the problem. Such systems were a primary source of inspiration for us when we started Sourcegraph—we wanted to bring the tech for large-scale code understanding to every company. In-depth posts like this are a big source of inspiration and ideas for us, so thanks to the author for writing this up.

yazaddaruvala 4y ago

Most of this makes sense, but ends up not being followed for non-technical reasons like the person's confidence, etc. I'd say, just know two things: 1. (From the post) "Let your instincts guide you if you think something feels much more difficult than you’d expect", and talk to your team about it, and change it for them. 2. There is a LOT of dead (unreachable / redundant) code in any enterprise codebase. Especially recently launched flag that haven't been "cleaned up yet". Just start cleaning this stuff up. Even the very obvious stuff gets you very well introduced to the codebase. Where I disagree with the post: > Taking notes is non-negotiable for me, not strictly as a memory aid, but as a means to communicate findings and verify them later. If you need to take a 2 line note, you're better off having written a 2 line code review. Even something simple like "metric.add("DOES_THIS_HAPPEN?", 1);" can be done with 0 confidence / testing or with minimal-confidence / minimal-testing you could instead write "Preconditions.checkState(..., "Will this be caught by our integration tests?");". The best notes are the ones that get pushed to production to validate assumptions.

fsckboy 4y ago
a person who doesn't understand a codebase shouldn't be adding "does this happen" all over the place. run in a debugger and set a breakpoint.
- tejohnso 4y ago
  Agree on the first part but there are going to be plenty of cases where running in a debugger isn't sufficient to satisfy one's self that something never happens. Unless we're talking about "does this happen under this specific scenario I know about".
  - fsckboy 4y ago
    if something never happens, that's not a part of the codebase you need to understand right now. If your task is cleaning up, sure, but your task here is learning what happens, not what doesn't happen.

nerdponx 4y ago

I wonder if we can use the vector representations from the inner layers of LLMs to create some kind of semantic code and text analysis tools. Maybe such a tool can highlight salient areas, and tag regions of code as "boilerplate", "standard idiom in this language", "business logic", "performance optimization", et alia. As for taking notes, is there a IDE (or better yet a Vim plugin) that lets me annotate specific lines or regions with my own notes and comments? Or even annotate functions and classes that way, so I can see those comments in a pop-up box whenever I hover over an annotated function.

andrewstuart 4y ago

When working in a big unfamiliar code base, I: * have a goal to do something like, save some image data for example * just dig around trying to find where the right place is to get the data I need * hack in code to see if I can get the damn thing to work at all * once its working, try to do do it neatly if I can * don't worry too much about not understanding everything - it can take months to work out how a system works. This only comes through doing lots and lots of iterations of hacking in little fixes and along the way learning what you can

x86x87 4y ago

Just ask chatgpt to summarize it. How hard can it be?

andrewstuart 4y ago
It can't do that yet but I'm really excited about the day coming when AI can understand an entire code base. ChatGPT has already completely revolutionized programming for me, I loove forward to further advances. It's truly unbelievable, like have a permanent pair programmer with me who I can ask how things work, how to fix problems, and more importantly than anything else, ask for examples of how to do things, which I then hack into my code.
- x86x87 4y ago
  I too look forward to machines rewriting my code and telling me I'm not smart enough to understand it when I object to their questionable output. I can see a 0.1x PeonGPT telling me that it got a ShipIt from a 10x TeamLeadGPT and we are moving forward with the code. Any concerns? Create a jira ticket which JiraGPT will promptly label as "tech debt" and will throw into the "backlog" for it never to look at it again.
worik 4y ago
A whole new take on `make world`

furyofantares 4y ago

I think something like Github Copilot can be a huge boost to getting up to speed in big unfamiliar codebases -- it will autocomplete what you're trying to do with lots of "unknown unknowns", patterns and APIs in the codebase that you don't yet know about.

potatoman22 4y ago
I've been thinking about LLMs' ability to help with this. If they really did "memorize" data as well as some people think they do, it'd be a great boost to enterprise software development.

eyelidlessness 4y ago

My two big pieces of advice (which I’ve also given to team members growing into senior roles): 1. Learn how and when to step into “external” code when debugging. How being mainly oriented around navigating call stacks that might be obfuscated by default, while establishing pattern recognition for cases where problems in your code show up in third party call sites versus simple cases of misconfiguration. 2. Sorry but… learn regex. Even if your language is static everything, you’re probably going to hit a wall using code navigation tooling. If you know the general format of things you’re looking to find and call stacks aren’t getting you there, regex is an incredibly powerful way to find what you’re looking for. Don’t rely on it for aggressive mass changes, but definitely get comfortable with it for finding potential root causes.

worik 4y ago
> Sorry but… learn regex. Do not be sorry. find and grep. How is it possible to exist without them?
- eyelidlessness 4y ago
  I’m not really sorry, but people really hate regex. For reasons I comprehend but I use them on the job pretty much daily.

knlb2022 4y ago

Huh, I wrote about this a few months ago too (with a lot more whimsy): Eating Elephants -- https://docs.google.com/document/d/1c07-Zj6bUbYPwx7Zttd1N74oG2Ylk2EHPKpkUmdcsfw/ https://docs.google.com/document/d/1c07-Zj6bUbYPwx7Zttd1N74o... The thrust of both our articles is similar, except I think I ended up using many more words.

gorgoiler 4y ago

It is incredibly important to avoid putting tag-firebreaks in the code base — anything that stops a new developer from being able to navigate from symbol to definition, or from definition to all uses of that function. A couple of examples: ACTIONS = { “swim”: Swimmer, “skate”: Skater, } # …pages of code… def f(action, *args): … mod = ACTIONS[action] mod(*args).do() At face value, this pattern doesn’t do anything particularly wrong but anyone who arrives at f on their code safari won’t see any symbols for either Swimmer or Skater. It breaks their flow the same way switching languages might do. Sure, ACTIONS is only a hop away, but that’s a level of indirection that could have been avoided. Even worse, what if the class name was derived from a string? The fact that the action implementations are chosen at runtime also makes life a bit awkward but the primary issue is not being able to immediately see the action classes and jump to their implementations. Another one is calling shell scripts. We all have projects, I’m sure, where we have to shell out to some common legacy tool: def f(): run(“legacy.sh”) def g(): run(“legacy.sh”) Wrapping the script in a native function does wonders for making it easier to find, especially for the poor, sacred soul who deprecates it from the codebase one day: def f(): run_legacy() def g(): run_legacy() # …pages of code… def run_legacy(): run(“legacy.sh”) Any unfriendly pattern that breaks tag-hopping / IDE-integration is yet another small chafing of your colleagues‘ productivity. Think twice before you introduce that YAML file which defines behavior, instead of implementing it natively — your friends (and their jump-to-definition muscle memories) will thank you.

worik 4y ago
Really good. Think about the programmer reading this at write time. How to write large code bases that can be understood, every bit as important as knowing how to read them
- Denzel 4y ago
  Software Engineering at Google has lots of valuable advice and best practices to accomplish this - https://abseil.io/resources/swe-book/html/toc.html https://abseil.io/resources/swe-book/html/toc.html
4y ago

psadri 4y ago

My first step is to learn the data model for the application. Then everything starts to make sense as code to read/write/present that model.

worik 4y ago
Yes. If there is a coherent data model.

jcranmer 4y ago

I've long had a goal to go through a similar explanation of how to do this, and I figured the best way to do this was to pick a large project whose source I had never seen before and record myself fixing a bug in it. This plan is foiled by the difficulty of finding large projects whose source I had never perused before just for fun, especially in my core area of competence. But some salient points I think I can make is: * A lot of people have pointed this out, and I think it's the most important thing: don't go into the project asking the high-level question "how does this work?" No, you want to start by figuring out something specific. If you're picking apart a command-line tool, for example, start by asking where one specific subcommand is implemented, and what it actually does. Once you've navigated yourself to such a starting position, you can start building a mental map from that particular location, and expand it to higher or lower levels of detail from there as necessary. * Use code search (e.g., grep) liberally, and start guessing for the keywords you want to search for in the code to reach your next piece of information. In my previous example of looking for a subcommand implementation, searching for a probably-unique part of the help message is a good way to guess where the code might be located. * If you don't understand when code fires, something like assert(false && "Did I happen?"); is useful for figuring out examples that execute a particular codepath (assuming, of course, that your codebase has a competent testsuite). Now you have tests that you know execute that code path, and you can start using a debugger to probe what program state looks like at that point. Also, get very acquainted very early in the process how to satisfactorily dump out that state. * Play the git blame game. Look up the history of the code you don't understand--track down the exact revision that added that code, and then go spelunking in code review or bug tracker history to understand why it came to be. If code seems to no longer be used, and you don't understand why, play the reverse blame game--look at where it was added, where it was used when it was added, and find where that code was removed, and (as above) go spelunking in more detail to understand why it came not to be. * Practice, practice, practice! While I don't think these are difficult skills to acquire, it's hard to talk about this in generalities because so much of the process relies on your experience knowing what things generally should be like as you try to match up the codebase to your (possibly incorrect) mental models of how it works.

wpietri 4y ago
> don't go into the project asking the high-level question "how does this work?" No, you want to start by figuring out something specific Should this be an or? For me, understanding big code bases has always been a pendulum between the big picture and details. Survey the overall scope, and then pick some very specific thing and examine it all the way through. Then go back to the big picture, integrate what I've learned, and come up with another very specific thing to investigate. For me this cycle is especially fun when on the trail of a bug. Get the big picture, and then dive in on a specific bug and beat it to death. Then ask what he bug told me about not just the code base and the architecture, but the development practices and the company culture. Pick another bug and run it to ground, seeing which of my notions were validated and which were challenged. Eventually, produce recommendations for broad, long-term changes. It's been a while since I did a gig like that, but they're such a joy when I get the chance.

helsontaveras18 4y ago

This sounds like an extremely fun job (or jobs?), how can I get it?

sandkoan 4y ago

I've been working on a semantic search engine for code that's particularly useful to those navigating unfamiliar codebases. Just write an English description of whatever you're looking for, and it'll show you the most relevant/similar functions from the repository. Afterall, it's often useful to see how things are done, and why. Search at least makes possible the former. Email me at govind <dot> gnanakumar <at> outlook <dot> com if you'd like to try out a beta—on account of potentially being hugged to death, I don't want to publicly display the link.

aurelius83 4y ago

One of the best pieces of writing I've come across that I still come back to about grokking complex codebases was written by Mitchell Hashimoto. https://mitchellh.com/writing/contributing-to-complex-projects https://mitchellh.com/writing/contributing-to-complex-projec...

zknill 4y ago

I work in a large java code base, where I'm often exposed to brand new code (written a while ago, or written by other teams). I built this tool to help me understand the unfamiliar: https://packagemap.co https://packagemap.co There's a parser that extracts all the classes and methods from the java code and a site that renders these in a directed graph. I use a kind of query syntax to filter the graph, and added features based on how I want to explore the code. The nodes in the graph carry an name like: "my.company.package.MyClass.myMethod" Then I can filter by wildcards: *MyClass* Or see just the classes with a line terminator: *.MyClass$ Or see the code that leads up to, or away from a method/type: ->*MyClass, *MyClass-> I've found making the problem visual has massively improved how I navigate the code. In code review I spot things that are very hard to see in an IDE or file-tree review UI (like github). Things like; where abstractions are being broken/leaking, or where types are being mis-used across packages. I wrote a summary of the common cases I find using this tool in a blog post[0] [0] https://blog.packagemap.co/posts/common-code-coupling-mistakes/ https://blog.packagemap.co/posts/common-code-coupling-mistak...

aloer 4y ago
I suggest you position the free tier more prominently on your pricing page. Took me a while to realize it's even there
- zknill 4y ago
  Yes! Thanks

xtiansimon 4y ago

This is parallel topic—how do you grok an unfamiliar API? You have API access and a blank piece of paper, what are the first three things you’d write down?

jrvarela56 4y ago
No source, no docs? I'm assuming some app uses it: connect my phone to a proxy (I use mitmweb to inspect), close all apps except that one, and tinker with the app to see what requests come up.

effnorwood 4y ago

[dead]