11 ms·

Plotting the source code “TODO” history of the most popular open source projects

• wiz21c 5y ago
  What about FIXME ?
• thenoblesunfish 5y ago
  Anyone got a (git-based) one liner to get this info for an arbitrary repo?
  • valyagolev 5y ago
    here you go git rebase -i --exec 'ack TODO | wc -l >> log' HEAD~20 (create a temp branch first!, then substitute your starting revision, and save-quit the editor that'll pop up)
  • unwind 5y ago
    As a starting point, git log --format=format:"%at %H" | sort -nr gives a list, with the oldest entry first, of just "hash timestamp" pairs, one per line. You can then use e.g. date -I --date='@1620720025' to convert the timestamp back to a human-readable date in ISO format, i.e. "2021-05-11". The next step would be to loop over the list, checkout each revision, grep and wc the TODOs, and collect into date buckets. Anyone? :)
  • pizza234 5y ago
    You've asked for it ;) This prints the years - you can the group and plot them as you wish (it should be fairly easy, but I wrestled enough with git). It's a non-rigorous script (eg. it assumes nobody's email/name includes an `YYYY-MM-DD`-like string, and that filenames don't include the colon character): grep -P '\bTODO\b' -n -R -- * | awk -F: '{ system("git blame "$1" -L "$2","$2) }' | perl -lne 'print /(\d{4})-\d\d-\d\d/' The working is actually fairly simple: - grep prints filenames and lines - awk captures the filename and line, and executes a git blame on it - perl matches the year and prints it In the Perl matching expression, month and day are not strictly necessary, but disambiguate potential 4-digit numbers in the email/name. I'm very underwhelmed by the lack of customization of the `git blame` command - `--porcelain` is also uncustomizable, which makes things even uglier. Note that `git blame` also mishandles some edges (printing "fatal: file [...] has only 1 line").
    • peff 5y ago
      What do you want `blame --porcelain` to do that it doesn't? Using: git blame --line-porcelain "$1" -L "$2,"$2" | perl -MPOSIX=strftime -lne '/^author-time (\d+)/ and print strftime("%Y", localtime($1))' I suppose it would be a little more convenient if you could ask `git blame` to format the whole line itself, but that wouldn't be part of the `--porcelain` output. All that said, that pipeline is quite slow on something like linux.git, as it runs a series of blames which will walk over the same history many times. I think: git log -STODO --format=%ad --date=short would be much faster (it's not _quite_ the same thing, as it counts TODOs which went away, but is a reasonable variant).
      • pizza234 5y ago
        > What do you want `blame --porcelain` to do that it doesn't? Using: It increases the complexity, due to time conversion. One can certainly solve the general problem by throwing enough awk/perl/sed at it, but an option to customize the blame output would make it significantly more ergonomic (and the oneliner much simpler).
  • cerved 5y ago
    I'll throw this into the mix git --no-pager grep -I --full-name --line-number 'TODO' |\ sed 's/\(^[^:]*\):\([0-9]\+\):.*/\1\n\2/' |\ xargs -d '\n' -n2 sh -c 'git --no-pager blame "$0" -L $1,$1' Not blazing fast but I think it does okay-ish. What it does: 1) git-grep for files that are checked in, not "binary" that contain the string 'TODO' 2) sed away the actual line contents (git-grep doesn't seem able to only output file:line-nr) 3) use xargs and sh to call git blame on that file:line-nr This shows the last time the TODO line was modified, ie: it may have been created 10 years ago but somebody modified the last yesterday edit: one might want to throw in --cached to git-grep to search the index and not just the current working-tree
• rplnt 5y ago
  Golang has/had TODOs in generatred code I assume?
• kuu 5y ago
  Interesting to see that most of them are almost always growing. It would be interesting to compare them to some other metrics, such as TODO/lines_of_code or TODO/num_contributors, to compare the TODO's with the size of the project. I guess that as project gets bigger, it also gets more TODO's
  • cies 5y ago
    > TODO/lines_of_code At least this is needed for any meaningful comparison between project (or even in projects themselves, as some double in SLOC count over a few months)
    • orthonormal 5y ago
      Ignoring the numbers on vertical axis, plots look like they are normalized to fit the plotting area. TODO/LoC should basically have the same form.
      • kuu 5y ago
        I don't think so. These plots are normalized by the max number of TODO's, but that may vastly differ from max TODO/LoC of each project.
        • cies 5y ago
          Indeed! When those numbers jump up/down in the graphs, there's prolly a code merge/purge cause to it.
• anoncow 5y ago
  The Golang Todo chart is interesting. It has a sharp peak in April 2018. Linux and Swift seem to be the most in number and uniform in growth.
  • forgotpwd16 5y ago
    I wonder what happened in Go there.
    • kgravenreuth 5y ago
      most likely code generation?
• Kipters 5y ago
  I thought Golang's number of TODOs was much much lower than the others until I looked at the scale
• kozziollek 5y ago
  Cool! That date axis though... PHP's 3 years 2011-2014 are much shorter than 2 years 2014-2016. NodeJS's years 2017-2020 that were ~50% longer than 2013-2017.
  • yiyus 5y ago
    I guess that each data point is a commit and they just made more commits in the 2014-2016 period than in 2011-2014. But it's just a guess.
• mekkkkkk 5y ago
  The growing number is hardly surprising, yet I don't know exactly what the implications are. There are a lot of different categories of TODOs, ranging from harmless ("it would be nice if this was improved at some point") to critical ("this is a really bad solution that needs to be fixed ASAP"). I wonder if these repos has official definitions of what a TODO entails.
  • mnahkies 5y ago
    Perhaps we should be including a priority in such comments, eg: TODO: high: don't use bogosort Would only be useful if it became a defacto standard though
    • esyir 5y ago
      At that point, make an issue and add it to the tracker.
    • schleiss 5y ago
      Yes, I agree. Priority and some context on what to do instead. Because as soon as it's a standard, big software vendors like Jetbrains could automatically categorize and mark the various lines.
    • spockz 5y ago
      Back when I used Eclipse there was another level called “FIXME”. Maybe it works in IntelliJ as well.
      • tirpen 5y ago
        It does. At least it works in PyCharm, so most likely it works in all IntelliJ based IDEs.
        • ExtraE 5y ago
          Yes, it does. In fact, you can configure it to use whatever you want. (I have it also match BUG as another priority)
• schleiss 5y ago
  OP here. I used `git log -G TODO --reverse -p -- . > ~/Desktop/test.txt` and used the results in PHP to aggregate the data as I couldn't think of the bash one liners in the other comments :(
  • pc86 5y ago
    Stupid question, is "TODO" in this instance case sensitive?
    • deleted 5y ago
      [deleted]
    • tastroder 5y ago
      It's case sensitive by default, you can add -i/--regexp-ignore-case to disable that.
      • thamer 5y ago
        You might also want -w for whole words, otherwise commits by your colleague Todor might cause some confusion.
• JulianMorrison 5y ago
  Interesting that some of them don't grow, or don't grow much, over time. Someone at PostgreSQL and Django is actually reading and fixing the TODOs.
  • jsmcgd 5y ago
    It would also be interesting to plot the number of TODOs against the size of the code base too. One would assume that as a project grows, the number of outstanding TODOs would grow too. Where and when this isn't true, might reveal something more interesting.
    • WrtCdEvrydy 5y ago
      We took some action on this internally at a place I worked. We had a couple of projects that had unit tests neglected so we enforced that you had to round up to the next closest 1% on the package.json for your merge to be approved (as well as adding some unit tests) In 3-4 weeks, code coverage slowly went from 20% to 73%.
    • hinkley 5y ago
      I bet it's more to do with rate of growth than total volume. I believe that one of the cognitive dissonances with people who think a lot of code is good news is that they become overwhelmed by how much they would actually write if they stuck to their convictions and so they start using TODOs to make themselves feel better about doing the wrong thing. Projects that grow slower I suspect have fewer TODOs.
  • zufallsheld 5y ago
    While it's probably the case for postgres or Django, todos getting less could also mean high code churn.
    • macksd 5y ago
      Yeah I noticed several cases were there to do's drop dramatically. I wondered if that was because that was a module that had a lot of to-dos that was also low quality, and the subject of a complete rewrite or replacement at some point. Golang was also interesting because of a huge increase followed by an almost equally huge decrease a short time later, both of them seemingly vertical. I wonder if something got reverted, or if they actually went back and addressed a bunch of to-dos.
• dep_b 5y ago
  Just today I announced a sweep through all of the TODO's and to either turn them into issues, stories or remove them. Biggest problem in Xcode is that it clogs up the warning list and the real warnings get swamped by them. But it's funny to see that Rust has less outstanding TODO's than our brand new 45KLoC project.
  • cerved 5y ago
    Can't you configure Xcode to filter out such warnings? TODOs offer valuable insight but they may not warrant turning into issues or stories. Removing valuable information because it's cluttering seems like a shame. Better if you can filter it in such a way that it doesn't clutter
    • moring 5y ago
      Quick idea: You could use a different wording that doesn't get recognized by Xcode but sticks out in a similar way. It would take some time to get used to it though.
    • dep_b 5y ago
      I think it's actually my linter generating these warnings! But I do like them to be resolved so I consider them issues.
      • cerved 5y ago
        everybody wants todos resolved but chances are that if a decision is made to fix, make issues or nuke them, they'll just get nuked instead
  • the_duke 5y ago
    TODOS are great documentation. They often reveal design decisions, suboptimal implementations and the thought process of the creator. This information is often completely lost in issues that noone will ever look at again. I also like to do TODO sweeps, but with a bias towards rewriting the into documentation or leaving them in if the are actionable.
• AJRF 5y ago
  I was just thinking the other day that searching for TODO is probably a very good way to search a project for potential bugs or security issues. E.g; I see a bunch of todos in Firebase iOS SDK that look kind of interesting to an attacker. Without looking into how the methods are called I can't say if they are actually exploitable (and I am sure Firebase is fuzzed to high-hell) but it was a little seed planted in my head.
  • _kbh_ 5y ago
    for a great example of this just have a look at the following macOS privesc the source of which came with the handy comment "deal with OOB". https://blog.zecops.com/research/from-a-comment-to-a-cve-content-filter-strikes-again/ https://blog.zecops.com/research/from-a-comment-to-a-cve-con...
  • lucb1e 5y ago
    Am security tester. Can confirm. Sometimes the vulnerabilities are just handed to you on a dark-themed platter and I don't look them in the mouth.
• atoav 5y ago
  Django doing really well (or nobody dares to write TODOs anymore)
• dkarras 5y ago
  I'd like to see a todos added / todos disappeared graph that would account for the velocity of development of a project. For some, while the todos grow, the relative "todo per LOC" might decrease etc.
• varjag 5y ago
  Tangential: magit-todos is a great package for Emacs/magit users to keep track of todos in a project.
• 38932ur98u 5y ago
  Would be interesting to add major release versions to the graphs as well
• Gravityloss 5y ago
  What about amount of TODOs per character or line of code?
  • staticshock 5y ago
    Agreed, this could make the graphs much more comparable and maybe reflective of project culture. Comparing something like the linux kernel to VueJS is nonsensical without any normalization with respect to overall repo size.
• pabs3 5y ago
  I wonder what other strings people use like this. So far I have seen FIXME TODO HACK XXX BROKEN.
  • DominikD 5y ago
    I tend to use my initials for the TODOs I introduced.
  • yakubin 5y ago
    At work we use "TODO(JIRA-XXXX):", where JIRA-XXXX is the Jira ticket for the TODO. Every TODO needs an accompanying Jira ticket. Otherwise it won't pass code review.
    • jrochkind1 5y ago
      If it's got an accompanying JIRA ticket, what do you experience as the value of also including the `TODO` in a source comment, over just the jira ticket alone? [edit: reasonable answers below, thanks!]
      • yakubin 5y ago
        When reading the source code, you immediately see an acknowledgement of the deficiencies, instead of assuming that everything is as it should be, or needing to investigate what is ok and what is not. It also maintains a link between the ticket and the location in source code throughout future source code changes.
      • gbear605 5y ago
        Suppose a future developer is working on a separate ticket that touches the same code. If there’s an inline TODO, the future developer knows that the TODO needs something changed, which can help them understand how the code works and they might wind up resolving that TODO as part of the second ticket. If there isn’t an inline TODO, the issue might be resolved without that first ticket ever being touched. I see it inevitably leading to a lot of dead meaningless tickets crowding up the backlog. If an even later developer then was assigned one of those unknowingly-resolved tickets, they might spend a significant amount of time looking through the codebase to find where the code needs to be fixed, only to realize later that they’re looking for nothing.
• jpswade 5y ago
  Todos are generally terrible practice in code, as they often don’t give any indication how they are going to get to done. They never get prioritised and very rarely does anyone get to do them. So what’s the point? It feels like a todo is really only there to serve as an excuse for suboptimal solutions.
  • _Understated_ 5y ago
    I dunno... I think they have their place. I'm working on a personal project right now and in order to see how things look, if they work etc, I've got a bunch of vanilla js frontend code and //TODO in a few places to call an API and get actual data. It works great for now as I've hardcoded everything, got it looking broadly like the finished product, and it means that I just have to do the API calls (and programme the API too, of course). I use them a lot.
    • eXpl0it3r 5y ago
      On a personal project, I can see this working, on a team project, you're not gaining anything with TODOs, because the chances that you or your colleagues actually go back and fix/implement the TODO are close to zero. Not only are you rarely gonna have the time to fix the TODO, but weeks, months, years later when you run into a TODO in your code, you have no idea what the actual requirements were, why it was not implemented, why it hasn't hurt anyone and whether anyone is actually needing it. Thus the TODO comment will remain forever, as you can't figure out what to do about it, without investing a lot of time and energy on requirement engineering. Personally, I've started to block PRs with TODO comments that aren't directly mentioning the future implementation story/bug. As such, even if the TODO is forgotten in some way, you at least will find a reference point to what should have been done here.
      • berkes 5y ago
        > because the chances that you or your colleagues actually go back and fix/implement the TODO are close to zero. This depends entirely on the team, company and/or work, though. It certainly is not a given. > TODO in your code, you have no idea what the actual requirements were, why it was not implemented, why it hasn't hurt anyone and whether anyone is actually needing it This depends on the task you are TODO-ing. Sure, if it is a "TODO: seems broken, fix." or "TODO: make sure that users don't see this", you are putting not just the wrong things in TODOs you are not giving them enough context. Compare that with a "TODO: this duplicates the routine in FooBars#bar_bar, but we cannot move this to a generic helper until the BarBar can handle both ActiveUsers and PendingUsers. Once that polymorphism is implemented, this can be DRYd up", which gives context, predicaments, and communicates that the author knows it is suboptimal, and explains how the author would've fixed it.
• twelvechairs 5y ago
  Is it controversial to say that those with low TODOs are pretty clearly the cleanest packages I enjoy working with most? (Postgres, Rust, Django, VueJS, maybe Python)
  • mceachen 5y ago
    Use and semantics of TODOs are decidedly not consistent across these projects. Just because you (and I) see correlation with our expected biases shouldn't be construed as proof: merely interesting chart wiggles. I think the shape (monotonic increase or sawtooth) can be used to see how a project handles either missing features of technical debt.
    • hinkley 5y ago
      But if the semantics speak to project philosophy differences that result in a worse or better experience, then it is important that they don't have the same semantics.
  • actinium226 5y ago
    Well, is the converse true? Are the packages with high TODOs the dirtiest packages that you least enjoy working with?
    • hinkley 5y ago
      "I'll do it later" is one of the more challenging personality faults to deal with in coworkers. There are very few people you can trust to make that statement, while most of the rest behave as if you should trust them as well, even though everybody knows they won't in fact do that later. Not only will they not do that later, but at some point they will compare their productivity to someone who ended up having to 'do that later' for them, causing their other work to suffer.
• snorberhuis 5y ago
  I always suggest TODO's to be replaced during a Code Review by: 1. A ticket number that will be picked up shortly if should still be part of a larger change. In a healthy team, this is done within two weeks and you know where to perform changes when you pick it up. 2. You do not add a TODO, but explain your current understanding of what is wrong and what should be done. This way you can refresh the knowledge if it ever again is touched. With a simple TODO, this knowledge is usually not writtend down.
  • stevenhuang 5y ago
    Or do both. Might as well add TODO: to make it stand out as a thing that can be improved while also making it greppable.
    • mihi 5y ago
      Might also be cool to automatically create these tickets (when commited to master?). Then you don't forget, and even if they end up not that detailed, you at least get a nice list of all of them.
      • actinium226 5y ago
        TODO: Automatically create tickets based on TODO comments. We'll get to it. Someday.
        • ableal 5y ago
          https://en.wiktionary.org/wiki/round_tuit https://en.wiktionary.org/wiki/round_tuit There used to be an ASCII art version occasionally slapped on Usenet posts: "here's a round tuit, you can go ahead."
          • hinkley 5y ago
            Caption: "An artist's impression of a round tuit."
      • 5y ago
• travisgriggs 5y ago
  It was interesting to see that Swift has 2K+. Seemed kinda high when I consider it's youth and uptake relative to some of its plotted peers. I don't know wether to suspect that's because it has a) an overly parliamentary development process that just creates lots of bookkeeping side affects b) a very aspirational development community that is busy writing tons of "try to take over the world" goals to improve in various and sundry ways c) indicates a lot of short sighted/highly focused language evolutions that leave a long trail of todos because that kind of "we have no big picture" creates a lot of corner/edge cases that need "todo" signs to document them d) something else?
• garblegarble 5y ago
  I'd be quite interested in seeing data on the age of TODOs over time - for instance, are there lots of old TODOs sitting around gathering dust while newer TODOs get fixed, or are they getting worked through?
• egypturnash 5y ago
  Damn, what happened to Typescript in May/June 2018? It jumped from around 730 TODOs to 3000, and has never really come back down. Wikipedia's list of versions suggests that this was probably related to version 3 happening in July 2019.
• shireboy 5y ago
  Well, I'm glad I'm not the only one who never gets around to my //TODO#s. One related tip for devs: I've started adding "You are here" as a placemark for where I'm working in the code. So for example, on friday, if I want to pick up quickly next monday, I add "//TODO: YOU ARE HERE Finish doing foo". Then on monday, I search for "You are here" and pick up where I left off easier. Saves me a few cycles, though if I'm honest, I have quite a few of those hanging out too.
  • superdimwit 5y ago
    even better, don't leave it as a comment. Leave it uncommented, as a syntax error, so when you try and run your code on Monday morning you are reminded of it!
  • berkes 5y ago
    I've been annotating my work several times a day, with `INK`, from "leave some water in the well", a productivity hack from Hemingway[0]. I forgot how I went from "Water in the well" to "ink the well", though. It's been a while since I started doing it, and I wrote a blog-post[1] with some scripts and helpers that I still use. [0] https://www.fastcompany.com/3021905/hemingways-secret-to-maintaining-productive-momentum https://www.fastcompany.com/3021905/hemingways-secret-to-mai... [1] https://berk.es/2012/05/30/leave-some-ink-in-the-well/ https://berk.es/2012/05/30/leave-some-ink-in-the-well/
    • hinkley 5y ago
      > in other words, never end a day’s work without knowing how you are going to start the next day. This turns out to be very hard advice for developers to follow. I don't say that as a complaint about Hemingway, but as a complaint about developer neuroses. I have tried many, many times to convince people to associate their 'sense of completion' not with the act of getting their changes into master but the act of committing their changes (or pushing it to a branch, if you are PR-driven). It works less than half the time, and almost always with the more junior people... So many incidents of someone staying late to finish something, pushing it, then coming in late the next day (because they stayed late) to a bunch of upset coworkers who had to clean up the mess they made. Completionism will be the death of us all.
• geon 5y ago
  How did golang gain and lose 12k todos in an instant?
• javier10e6 5y ago
  TODO: Look Ma Im doling out work for whoever reads this. Are you the intern? You are it.