15 ms·
I know lines of code as a measure of productivity has been criticized to death here and elsewhere but today I went and did some analysis on past two years of my
by msoad 4y ago
I know lines of code as a measure of productivity has been criticized to death here and elsewhere but today I went and did some analysis on past two years of my team members contributions and saw the team members that I know they are productive are modifying code a ton more than others. It can be subtraction of code too but overall more changes correlated with my perception of their contributions. My perception is pretty informed because I'm their manager and I know this team and this code base very well.
- johannes1234321 4y agoAmount of code can be an indicator, however as soon as developers know it is being used as metric it is easy to manipulate.
- AnonCoward42 4y ago"When a measure becomes a target, it ceases to be a good measure." (short form of Goodhart's law)
- expazl 4y agoNo, it's not easy to manipulate. You're thinking in linear terms which isn't what lines of code changed is useful as. You can't use it to distinguish which of 4 different productive programmers is most or least productive. However it's extremely easy to distinguish the non-productive programmer in the group of 5. And it's not easy to fake, because as soon as that developer start merging in a ton of bogus or useless edits to push up their numbers then everyone on the team, including the manager will call them on their bullshit. It's just like shift times for workers in manual labor jobs. You can't tell which of the people who are meeting up 8 hours a day is most productive based on if they where there for 8 hours and 12 minutes or 8 hours and 25 minutes. But you sure as hell can easily see that if someone is showing up 4-5 hours on a nine-to-five job, then something is wrong. And if that person tries to "fake it" but showing up for 8 hours but spends half the time slacking off, but coworkers and managers will call him on the bullshit.
- BlargMcLarg 4y agoYour entire point revolves around the idea of coworkers ratting out and being smart enough to know. Not every place is so cutthroat and pro-employer this is a given. The idea the manager would call out individuals is laughable too. Most managers barely know how to code themselves. Give it another few decades before your manager looking through commits is a standard, you're more likely to see alert tools be prevalent before that.
- still_grokking 4y agoIt is easy to manipulate. You just need to create a shitload of bloated bullshit that somehow "works" (at least more or less). That doesn't make you productive. Actually that's even counterproductive as it creates technical debt. But by the bogus metric that would be positive.
- vidarh 4y agoCreating bloated bullshit has even more value in terms of measured changes than the initial bloat because it also creates the opportunity to fix it up later for even more changes.
- wsc981 4y agoIf the team verifies PRs, then it should be hard to get most bogus commits merged in the repo though.
- still_grokking 4y agoSure, in an ideal world no bugs and not even bad code would slip through review. I'm still to find the place where this is true in software development.
- startupsfail 4y agoNot really. One can always keep refactoring and churning things around. Search-replace to rename some long function name into something even longer. If you’ve managed to enforce the 80 char rule, this would likely result in things linted differently. Boom, massive change. No one can object to a good refactor.
- newZWhoDis 4y agoLuckily for elon then all those Twitter engineers had no idea to be boosting their LoC deltas for the past 2 years.
- quickthrower2 4y agoThey do now and they will. I bet there are bash scripts sitting on Macbooks to do all kinds of shenanigans with git going forward.
- gedy 4y agoI think the key point is modified, not just net new lines. About 15 years ago I worked for a genius manager who had his underling write a script to measure productivity this way, and they forgot to account for removing lines of code. They were so pleased with the one guy who just loved to copy paste crap over and over, versus us who were actually refactoring or reusing existing code.
- malux85 4y agoLines of code written Lines of code rewritten (changed N days after) Lines of code removed Lines of code that contributed to linting problems Lines of code that contributed to security issues (as reported by the static analyser) Average complexity measures CI/CD build failure rates Lines of code reviewed Gitlab has plugins to generate reports like this, and if you have enough data (multi year preferably) you can use this as a springboard for deeper analysis of members of the team, I find all these metrics together (non taken as absolutes because there’s many assumptions built into each one) can give a good overview of individuals in a team
- BlargMcLarg 4y agoReview + written encourages buddy-buddy dynamics. Build failure rates encourages frustration as long as you don't provide ways to run the exact tests in isolation locally, under the same rules as CI. Meanwhile, branch triggers and pre-push hooks trivialize this further. Most of these metrics are strange given they are trivial to check pre-merge. Meanwhile, running these statistics pre-merge discourages small commits and sharing code prior to having ideal builds. The moment this gets out is the moment devs will game your statistics and any benefit you gain goes out the door. And almost every one of these, even combined, is easy to game.
- malux85 4y agoCorrect, we do not tell the devs these are being measured
- BlargMcLarg 4y agoI cannot tell if you are aware you're setting devs up for failure or not. Half your mentioned metrics are tackled by hard forcing your devs to run branch CI prior to merging and giving them a slap on the wrist for not running CI checks locally pre-push. If they are important enough to measure, why not hard force them and focus on the remaining metrics instead?
- 4y ago
- greenthrow 4y agoThings LOC doesn't measure: - patents written / rewritten / meetings with lawyers / issued - mentoring of other engineers - PRs reviewed - talks given - meetings attended - RFCs / ADRs written - customer calls attended - quality of code (the original and best reason that LOC is bad.) Et cetera, et cetera If you are truly an engineering manager, unless all your direct reports are fairly junior, I sincerely hope you realize soon that LOC is a pretty terrible measure of engineer contribution. I am old enough to remember when the industry finally came around to accepting LOC is a worthless measure of engineer productivity in the late 90s. I am not exactly shocked but dismayed that this conversation needs to be rehashed. Number of tasks closed is a better measure than LOC and it is still a terrible measure.
- koonsolo 4y agoAt the company I'm currently working, they accept that seniors crunch out less code than juniors. Because of all the "side jobs" you listed.
- quickthrower2 4y agoThey shouldn’t just “accept it”. They shouldn’t care at all! Equivalent would be saying “our sales rep only said 2300 words today but that is ok, we can accept it he was visiting an old customer not cold calling like the juniors”
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- armchairhacker 4y agoLOC modified is a fine rough estimate. The issue is that it's a rough estimate and shouldn't be used in things like who to promote or fire without deeper investigation. Usually LOC modified correlates with actual productivity but there are lots of exceptions. Like if someone copies an open-source library directly into the repo, commits a formatting change, is working on a critical section of code, is working in a different language, is writing new features while everyone else is fixing bugs, has been told they are going to be measured by LOC, etc. The fact that you actually analyze your team members' contributions, know the codebase, and measure productivity in more ways than LOC is a good sign. The problem is when managers exclusively measure LOC to gauge productivity and make decisions. While LOC and productivity correlate more often than not, I bet you can spot at least some cases where the LOC is not an accurate measure.
- TedDoesntTalk 4y ago> LOC modified is a fine rough estimate Maybe for junior engineers and that’s it. I want my seniors and leads mentoring, doing code reviews, designing, communicating with the business about impact and the like, training, and yes, coding, but not as much as juniors.
- dbish 4y agoAgreed that the more senior you are the less time percent-wise you will allocate to just coding but for strong developers this doesn’t mean your output drops compared to juniors, it drops compared to what you could do if you were only heads down coding. It’s easier in many ways to have higher output as you become more senior if you stay close to the code, know more about the code base, and can be much faster, all the best devs I’ve loved working with are like this. Many of the less productive senior folks that drop in their output and don’t code much (i.e. modify < 100 loc in a month as a normal occurrence) become architecture astronauts which is not imho what a senior dev should be doing as they will lose touch and eventually their guidance is less useful on the complex matters where they are most needed which in turn means their team and company impact is not what it should be.
- 4y ago
- gburt 4y agoCounting lines of code, commits, changesets or any other simple metric will destroy your culture. The team _will_ find out, and then instead of contributing to the success of the business in earnest, they’ll be doing stupid things like maximizing their changesets or racing for “easy” large changes like deleting a module. It doesn’t matter whether those values do or do not correlate with reality (IMO, if they do, it is for relatively junior engineers only). If you give off the smell of measuring people like that, you will ruin any collaborative team environment and you risk never being able to recover that. There’s a good chance you’ll chase away excellent engineers with this sort of low-effort metrics management too.
- startupsfail 4y agoMassive amount of code, particularly when written by a single engineer more often ends up being a liability, not an asset. There are exceptions, but these are extremely rare. Look for code created in collaboration by multiple engineers, with contributors or reviewers across multiple teams. This code is more likely to be an asset. (Unless created by Google for external use or by xOoglers. Such code outside Google is a near guaranteed liability.)
- bigDinosaur 4y agoThis is a very general statement that I'm not sure is necessarily true at all?
- yellowapple 4y agoI reckon I could count the counterexamples on one hand.
- fredophile 4y agoOn the balance sheet I'd call functionality an asset and code a liability. Everything else bring equal more code is more to maintain as things change, search through for bugs, etc.
- tobyjsullivan 4y ago
- grogers 4y agoLOC definitely positively correlates to productivity, it just isn't everything. One year I modified over a million lines, because we decided that we wanted to stored some generated data in a config file that was checked into git, rather than in the DB. If you can ignore certain commits like this, the correlation is probably even better, but use it as just one data point. Another major confounder is that super deep investigations that fix major problems often have relatively trivial solutions at the end. But the effort to debug and fix them (that is largely hidden by conventional metrics) is enormous. The people working on these types of things are often your best devs (because you know they will actually get to the root cause), but if you are only looking at LOC you wouldn't know that.
- oxfordmale 4y agoAccording to Goodhart's Law, when a measure becomes a target, it cease to be a good measure. For example, I have just written a Snowflake stored procedure that auto generates SQL code and then executes it. I am going to call this stored procedure from multiple places in my Python codebase. Now if my manager would look at the number of lines of code I would still write the stored procedure, however, rather than auto executing the SQL code, I would just copy and paste the auto generated code into my Python codebase increasing my LOC for that week.
- meowmeowmoo 4y ago> Now if my manager would look at the number of lines of code I would still write the stored procedure, however, rather than auto executing the SQL code, I would just copy and paste the auto generated code into my Python codebase increasing my LOC for that week. Not everyone panders exclusively to optimizing grading metrics. I wouldn’t do that even if my manager looked at lines of code because I care about the quality of my work. I’m confident a lot of people share my sentiment.
- deergomoo 4y agoYou wouldn’t, but maybe some other people on your team would, or maybe some other team. Even if it’s just a small minority, all of a sudden you’re performing worse than them, despite doing an objectively better job. Hence, it’s not a good measure.
- habitue 4y agoSure, maybe if your manager passively mentioned looking at the stat, you wouldn't. If your manager brought it up in performance reviews as a reason you weren't getting a raise or being promoted, or suggested you might be laid off if you didn't get your LOC metric up, you would immediately change your behavior (or hopefully quit).
- cortesoft 4y agoIf you are already good, productive professionals, than what is the point of counting lines of code at all? If you are trying to use lines of code to measure productivity, the only reason is because you think some members of the team are not being productive. Sure, the good and productive team members won’t try to game the metrics… but the bad programmers, who you are trying to find, sure will. The good programmers won’t game the system, but that doesn’t matter because they are going to be fine no matter how they are evaluated. So what is the point of measuring lines of code? You aren’t gaining any new information.
- soheil 4y agoSo fixing indentation or spaces, changing case, renaming functions systematically are all things that can be done with a quick search and replace across the entire code base. I have a feeling the most experienced/"productive" engineers are the ones confident enough to make those changes, but that doesn't necessarily translate into more lines = better programmer.
- diob 4y agoAs with anything it depends. I'm definitely a code churn out machine (when I'm motivated at least), but I'm not going to say that's a good thing. I've seen others like me who leave a wake of tech debt, and others who don't. Who knows which one I am? People seem to like me though. As others point out though, the moment something becomes a metric it ceases to be useful. Check out the book "The Tyranny of Metrics" for a breakdown of it.
- saagarjha 4y agoThis can be approximately true on certain teams but falls over for others, like people working on diagnosing issues, security, or performance. Those jobs are engineering-related but generally require spending a lot of time thinking about code rather than writing it, so the contributions from it are not easy to see in a codebase directly.
- Tomis02 4y ago> did some analysis on past two years of my team members contributions and saw the team members that I know they are productive are modifying code a ton more than others Depends what you mean by "productive". If you mean "appear to be busy" then yes. But by using this metric you disadvantage people who measure twice before cutting once, people who think deeply about a problem and find a simpler way to solve things. In my experience, more junior people tend to spin their wheels a lot, and out of that spinning comes out a lot of code. Code that is often broken and then is is fixed again, and again. And such people appear to be more productive and busier, at a first glance.
- ruszki 4y agoMaybe the more productive people are just quicker in “measuring twice before cutting once”.
- saghm 4y agoI think that this can sometimes be the case, but I think it's mostly due to a couple of factors that end up making this sort of analysis not very useful at a smaller timescale. First, tasks that end up requiring a large number of lines of code changes (e.g. a large new feature or a giant refactor) will be more likely to be done by people on the team who are the most experienced and successful, whereas smaller scoped tasks tend to be assigned to junior engineers or new team members. This makes the association between lines of code and productivity for an individual engineer a bit circular; if you're the one assigning tasks as their manager, the engineers you think are more productive will be assigned the larger scoped tasks, so of course they'll have more lines of code. The other factor is that summing up changes over a long period of time pretty much by definition ends up being biased towards people who have been on the team longer; even if two engineers made the same number of per unit of time, the one who will has been on the team longer will obviously been the one who has made more changes, and that's not even taking into account the fact that people's productivity should ideally increase over time given the increased familiarity with the codebase, the tooling, the problem domain, etc. If the longest tenured team members are the most productive, you'd expect them to have the most changed lines of code based on that correlation alone.
- eyelidlessness 4y agoFascinating take. Anyway, I’ll likely be contributing deletion of at least 1000 loc from a codebase I maintain, sometime in the next couple weeks. My rate of code change in the month leading up to that will have been approximately zero. A week of that was on the road. The rest was the real work, and quite a bit more of the real work preceded that week on the road. No lines of code changed for weeks on end. That work was instrumental in being able to make a -1000 loc change. Glad I work with people who communicate about the work being done rather than communicating with commit logs.
- shubhamjain 4y agoI wrote about a blog post about this: https://shubhamjain.co/2020/05/17/being-fast-matters https://shubhamjain.co/2020/05/17/being-fast-matters. Basically, as your get better, you also get faster. It's an unpopular opinion but great programmers can often perform wonderful in a short amount of time. Hell, it took Linus just a day to start using Git for managing Git[1]. It's a product of experience, thinking clearly, and being knowledgable about good tools for the job. People will retort: "But, but, but... what about pushing junk?" Well, if you've decent code review system how will that get merged? Pushing junk is often a less of a problem provided you've a good engineering culture. The best teams I worked with were just plain fast. The worst ones had plenty of engineers hiding behind meetings, reviews, and bike-shedding. [1]: https://medium.datadriveninvestor.com/when-overachievers-self-doubt-the-birth-of-git-780503960577 https://medium.datadriveninvestor.com/when-overachievers-sel...