9 ms·

The idea of measuring individual developer productivity is kind of absurd to me. I'm not saying that what we do is magic, there are just so many variables. Mea

The idea of measuring individual developer productivity is kind of absurd to me. I'm not saying that what we do is magic, there are just so many variables. Measuring story points or lines of code is kind of the opposite of productivity. This encourages developers to do as much meaningless work as possible. I'd want a developer to make a task simpler or use an existing tool which means less time writing code. The value of them saving that work from knowing another way is high but hard to measure. What you want is measuring business outcomes but those are hard to attribute to a particular developer. I think unfortunately we're left with our subjective judgement here. I think we'd do better admitting to ourselves that we can't measure this than to pretend we have some sort of science here.

mrbadguy 2y ago
Well said. There’s a noticeable lack of judgement in many places these days; people think they can abdicate it to metrics and data but this is mistaken.
jckahn 2y ago
I don’t think it’s that we _can’t_ accurately measure developer value/productivity, it’s that doing so isn’t feasible with any methodology we currently have. As you said, we’re not doing magic, therefore our value is measurable. Measuring just requires an impractical degree of time and energy.
suzzer99 2y ago
Dev productivity is like quantum mechanics. The second you try to measure it, the wave function collapses, and you've fundamentally altered the thing you were trying to measure. That said, at every place I've worked you could get a starting point idea of who the top devs were by looking at overall code commits. This Tim sensei situation may be more common than I think, but I've never run into it.
- elevatedastalt 2y ago
  Thanks to Goodhart's law, it's true for any measure that becomes a target.
- szundi 2y ago
  [dead]
- lolinder 2y ago
  > at every place I've worked you could get a starting point idea of who the top devs were by looking at overall code commits. This works for Junior through Senior level roles, but it falls apart quickly when you have Staff+ roles in your company. Engineers in those roles still code, but they write a fraction of what they used to and that is by design—you want these people engineering entire initiatives, integrations, and migrations. This kind of work is essential and should be done by your top devs, but it will lead to many fewer commits than you get out of people who are working on single features. With a few exceptions for projects with a high degree of source-level complexity, a Staff+ engineer who's committing as much as your Senior engineers is probably either misleveled or misused.
  - suzzer99 2y ago
    Yeah, I've always worked at non-tech companies with pretty small teams and no roles like that. But it seems like any place with Staff+ engineers should know better than to try to stack them up against more junior devs based on some metric.
    - philjohn 2y ago
      Case in point, I worked with a Staff software engineer (I was also the same level) who consistently, half over half, had zero diffs to their name. Because they were leaning more into a product archetype - which it turns out they were very suited to.
  - Aeolun 2y ago
    I have a decent number of commits as a staff engineer, but barely any are on ‘features’ as measured by story points. We’re building the things that other people use to deliver features better and faster.
- winwang 2y ago
  With my minor dabbling in game theory, I've considered funny angles like secret-ish internal metrics and punish those who simply game the incentives. Example: bot detections and banwaves in MMOs. Instead of instantly banning plausible bots, the company has to hide its (ever-changing) internal algo and ban in waves instead. Basically, treating (non-)productivity like bot detection, lol.
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]
  - closeparen 2y ago
    A few years ago we started tracking both office attendance and PR count. Management swore up and down that they understood the nuances, and these metrics would only be used to follow up on outliers. But then one middle manager sets a target. His peers, not to be outdone, set more ambitious targets. And their underlings follow up aggressively with whomever is below target. For a few months there was an implicit "no vacations" policy in much of the company, until they updated the attendance dashboard to account for PTO. And now an engineer's position in the stack rank is presumed to be her position in the PR count rank barring very strong evidence to the contrary. The approach you outline is probably optimal, but I don't think middle management as a culture would be capable of pulling it off. You give them a KPI in a dashboard, they're going to compete on it, in a very open and simple way. Metrics are hard to argue with, and nuance requires constant vigilance to maintain.
- gopher_space 2y ago
  I've worked on projects where every single ounce of enthusiasm is coming out of one mediocre developer. Nobody pays attention to team dynamics like this, but they're all over the place.
- nijave 2y ago
  I like looking at lines removed and, in context with, lines added or PR count. Someone more experienced will be refactoring and removing old stuff while adding new stuff and will have some balance (barring vendoring or moving file trees between repos)
  - suzzer99 2y ago
    Lines removed should count like 5x lines added. I don't trust any developer who doesn't get a thrill out of removing code.
- __turbobrew__ 2y ago
  > That said, at every place I've worked you could get a starting point idea of who the top devs were by looking at overall code commits Yea, in the current place I work at we do not measure coding performance metrics, but if you look at the top commiters to the repo — by number of commits — you will see all of the people who bring the most value to the company. Even at staff eng the best engineers are still the ones who crank out code, albeit maybe as not as much as a senior engineer. Staff engineers who only commit 1 a month are usually the ones who don’t bring value to the company, especially given their high pay.
  - hamburglar 2y ago
    At my last gig I spent the last year and a half as staff engineer and made almost no commits because I was constantly writing proposals, defending them to execs, doing architecture reviews, doing design consultations, and planning long term responses to large incidents. I know for a fact that I brought a ton of value to the company but it was very uncoupled to commits. I didn’t really like it because I need to code, so I’ve moved on to a role where I commit like a maniac again. :)
    - jimbokun 2y ago
      Just check all of your proposals and design documents into source control, and you’ll still have plenty of commits!
    - __turbobrew__ 2y ago
      Yea it is dependent on your company. When I think of highly functioning staff engineers Im thinking of people like Jeff Dean and Mitchell Hashimoto. Similarly at companies I have been at, even the highest level principal super staff++ engineers still code. They do the high level strategy and design, but they are still writing PRs at least every week or two. If you are spending 100% of your time on politics and architecture as a staff eng I think that shows a somewhat dysfunctional organization.
    - milesrout 2y ago
      Writing proposals, doing architecture reviews and doing design consultations sounds to me largely like busywork and bureaucracy rather than real and productive work. Maybe in a business where the cost of iteration is very high, it makes sense to do this. But in a good business you should not be doing architecture reviews and writing proposals, you should be doing refactoring and creating prototypes, respectively.
- closeparen 2y ago
  You probably can't do it observationally. You could give several developers the same project, and probably see pretty big differences in timeline and quality. But no one is going to put in the work to run that kind of assessment at scale with the level of realism and time duration (probably several weeks) to make it meaningful.
- razeh 2y ago
  This is true for more than dev productivity.
- xdavidliu 2y ago
  > The second you try to measure it, the wave function collapses, and you've fundamentally altered the thing you were trying to measure. Related to Goodhart's law: "When a measure becomes a target, it ceases to be a good measure". I'm wondering if this is only true if the act of measurement is transparent (so that the people being measured can game them). If the act is opaque enough such that the form of the metric cannot easily be guessed SEO-style, then maybe wave function collapse can still be avoided?
- naikrovek 2y ago
  https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law says (almost) exactly this.
OnionBlender 2y ago
I had a director that was obsessed with github enterprise stats. He forbid people from squashing commits and told people to commit every day, even if you're in the middle of something. This was so that he could see who was writing the most code. One of our interns was close to the end of his term and this director wanted to hire him. He thought the intern was amazing based on the amount of code he wrote. The problem was that this intern was bad, so we had him write unit tests. However, he was also bad at writing unit tests. He would run the code and then write tests that enforced the current results, instead of considering that the current behaviour might be incorrect. Thankfully we didn't hire him after myself and others explained why the intern had so many commits and lines of code written.
- lisper 2y ago
  > who was writing the most code There's yer problem right there. Code quantity is not correlated with value. In fact, it can be negatively correlated with value if it's buggy and laden with technical debt. Measuring productivity by lines of code produced actively discourages writing clean, maintainable, bug-free code.
  - thesuperbigfrog 2y ago
    > Code quantity is not correlated with value. In fact, it can be negatively correlated with value if it's buggy and laden with technical debt. ** "No Code" or Nihilist Software Engineering ** No code runs faster than no code. No code has fewer bugs than no code. No code uses less memory than no code. No code is easier to understand than no code. No code is the best way to have secure and reliable applications. Write nothing; deploy nowhere. One of my most productive days was throwing away 1,000 lines of code. -- Ken Thompson The cheapest, fastest, and most reliable components are those that aren’t there. -- Gordon Bell Deleted code is debugged code. -- Jeff Sickel Measuring programming progress by lines of code is like measuring aircraft building progress by weight. -- Bill Gates * Master Foo and the Ten Thousand Lines * Master Foo once said to a visiting programmer: “There is more Unix-nature in one line of shell script than there is in ten thousand lines of C.” The programmer, who was very proud of his mastery of C, said: “How can this be? C is the language in which the very kernel of Unix is implemented!” Master Foo replied: “That is so. Nevertheless, there is more Unix-nature in one line of shell script than there is in ten thousand lines of C.” The programmer grew distressed. “But through the C language we experience the enlightenment of the Patriarch Ritchie! We become as one with the operating system and the machine, reaping matchless performance!” Master Foo replied: “All that you say is true. But there is still more Unix-nature in one line of shell script than there is in ten thousand lines of C.” The programmer scoffed at Master Foo and rose to depart. But Master Foo nodded to his student Nubi, who wrote a line of shell script on a nearby whiteboard, and said: “Master programmer, consider this pipeline. Implemented in pure C, would it not span ten thousand lines?” The programmer muttered through his beard, contemplating what Nubi had written. Finally he agreed that it was so. “And how many hours would you require to implement and debug that C program?” asked Nubi. “Many,” admitted the visiting programmer. “But only a fool would spend the time to do that when so many more worthy tasks await him.” “And who better understands the Unix-nature?” Master Foo asked. “Is it he who writes the ten thousand lines, or he who, perceiving the emptiness of the task, gains merit by not coding?” Upon hearing this, the programmer was enlightened. Source: http://www.catb.org/~esr/writings/unix-koans/ten-thousand.html http://www.catb.org/~esr/writings/unix-koans/ten-thousand.ht...
fsndz 2y ago
the worst programmer nowadays is the vibe coder. vibe coding is so overrated imo https://www.lycee.ai/blog/why-vibe-coding-is-overrated https://www.lycee.ai/blog/why-vibe-coding-is-overrated
- sanex 2y ago
  I've been writing code professionally for a decade but I've never written so much code in my free time because I can just vibe code it all. I wouldn't do it at work and I probably wouldn't trust a juniors vibes as much as my own but no tools have made me feel quite so powerful.
  - fsndz 2y ago
    it's definitely nice for prototypes, but nothing more complex.
    - eclipxe 2y ago
      I used to feel that way but my perspective over the last 6 months has changed greatly. You can absolutely build very complex things in this style
      - fsndz 2y ago
        show me an example of such a complex thing obtained after just writing prompts
        godelski 2y ago
        Sometimes you enter a codebase and it looks like there's some obscure attempt to summon a Lovecraftian entity made of spaghetti and duct tape. Those codebases are both the shitiest and most complex stuff I've ever seen. They sure don't work well and I'm not even sure do a good job at summoning the old ones. I don't find LLM code significantly differing from that kind of monstrosity. It sure is complex.
      - 2y ago
slowtrek 2y ago
Could a company keep a subjective poor performer on for the lifespan of the company? As in, what is the plus or minus in overall revenue or profit from this charity? What if all companies did that? Could we distribute the "burden" of the charity across all companies for a better society? My point is, I don't even know if metrics are good or bad, we may need to look at why we see each other like this. Is it so offensive to the mind of the captain of a ship that they may have a few of not the best sailors? It's a chance for them to be on a ship, go on a journey. The concept of a "free ride" appears to be a serious moral hazard for us, but I can't figure out why.
- ponector 2y ago
  Are you going to accept such free rider in your team? You will eventually do both your and his tasks but for the same or worse pay. Given you have a fixed budget for the team - do you want to split it more ways? Where can I send a CV?
  - slowtrek 2y ago
    If we were to do this, we wouldn't just pick someone so out of the range of what's needed. So basically, if you reframe your question, would I hire a 70 average student on a team of 90 average students? Yes. We would not pick the 50 average students. This is all possible within reason, the kind of stuff being discussed in this thread is to purge people who are 80 average students (free riders in a 95 average class). Will we need to tutor and support the 70 average student? Yes. Why would we do this? It's good for the soul. The stuff I'm talking about has no place in business, as far as we care to understand as a society at the moment.
    - rpmisms 2y ago
      My dangerous opinion is that the purpose of a company is to provide a living for its employees.
      - slowtrek 2y ago
nijave 2y ago
I don't think productivity itself is absurd but it's definitely a hard thing to do, especially in small time intervals. I think it's important to measure employee performance and you can definitely can a macro look and say "what have you accomplished in the last year" which might be measured of delivery of "t-shirt" sized projects
matusp 2y ago
It is not absurd. From manager's point of view, when they are deciding to let someone go or promote them, they often need some arguments. And since software is invisible, it is often hard to just say straight away who is over/underperforming and why you think so apart from a "vibe" you are having. Measuring some relevant metrics can strengthen your argument and/or your overview of the situation. I don't think that the blog makes a compelling argument why measuring productivity is bad per se. It makes a compelling argument that metrics should not be interpreted blindly, but the metrics in this case identified a guy that was doing something unusual, and the managers managed to interpret why this is the case. But if it was an IC that is supposed to deliver, or if you did not want "Tim" to spend his time coaching people, this could still have been valuable info.
- morsecodist 2y ago
  I don't think it is absurd to reason about individual contributions, just to focus on measuring them with metrics. I have been on the management side as well though I admit I am much more of an IC. But the way I think that should go is laddering up to business value by describing what the person did and why that contribution was important. So for example I would say something like: X deserves a promotion because their work was important to delivering project A on time. Project A was difficult but worthwhile because it allowed us as a business to meet goals 1 and 2. That said I mostly work in smaller orgs. I have never been in a situation where a manager would be so removed from a team they would need a sort of proxy metric to direct them where to focus their attention to understand what the people on their team are doing day to day. I can see how this would get more difficult as a company grows.
bob1029 2y ago
> What you want is measuring business outcomes but those are hard to attribute to a particular developer. I think we could solve this by eliminating some middle tiers and putting the developers on the actual customer calls every week. Each senior developer owns at least one customer. That sort of thing. It's a completely different ball game when you are a developer on some B2B/SaaS and you are answering directly to the customer regarding your work. There's no one to hide behind - your teammates are now aside you and can only render supplemental aid. Once you have developers answering directly to the customers, you have a simple, ~boolean, qualitative metric that virtually any manager/investor can get behind: Is the customer happy? If the developers are keeping the customers happy, then they are performing. If the customer is unhappy, then there is a performance issue. Whether or not most or all of it can be attributed to blockers on the customer's side is irrelevant at the senior levels of play. Developers should be helping the customers get unblocked. Offer meetings & screen share time. Generally, make yourself as available as possible. If you do this correctly, the customer is likely to take note and provide you with some amount of cover (i.e., not be in a raging fit when they call your CEO). There is no reality in which you can inject more middle management or process to get developers to take more accountability. The only thing that works is to get them out of the bullshit matrix and in front of the customers. They need to experience how it feels to work directly with a happy customer and then realize how important it is to keep them that way. This path also happens to solve a whole host of other maladies in technology business, most notably shiny rabbit chases. When you have the fear of god in you regarding the client, you aren't going to be as inclined to play in traffic with a NuSQL engine on their watch.
- eek2121 2y ago
  lol no, I have worked with companies that do that. Many developers have very specific personality types, and they also think in technical terms rather than customer terms. It isn’t a bad thing, it is why they are good at what they do. The most successful companies I have worked with have a good product manager that can take customer input and work with a technical manager to balance priorities/effort. The technical manager consults the team before making decisions (such as SCRUM meetings if using that) Issues come into play when folks start distrusting developers. When we say it will take 4 weeks or 8 weeks to implement something, there is a reason for that. We know the code. We know how much of a PITA it is to work with, and we aren’t being misleading. On the flip side, we have been trained to give conservative estimates thanks to crap management and unexpected pitfalls, so we try to understand promise and over deliver. If management could recognize this, everyone would be happy, provided they recognize that 8-week timeline is fine and they don’t promise something different to the client and trust devs to do their thing. EDIT: Managers also tend to forget that we aren't a machine in a factory. We have good days, bad days, and everything in between. We excel with using our brains, however our brains suffer from anything between lack of sleep, depression, and other nonsense due to just plain having a bad day. I feel like it is more noticeable with us because we rely on our brains so much more than other folks in other professions do. Shoot, even random noise in an office, whether working at home or at an office, can hurt productivity. Now I have myself missing dev work. Hoping to go back soon. Currently unemployed.
WalterBright 2y ago
Everyone on a programming team knows who the good and not-so-good programmers are.
- dilyevsky 2y ago
  Kind of meaningless statement though. Even if they do, which Im not sure is true in general case because you have to be good yourself to know who’s really good, they are not telling you anyway.
  - WalterBright 2y ago
    In every organization I've worked in, everyone knew who the productive and non-productive people were. Just like in school. Everyone knew who the good teachers were, and who the good students were. I don't fathom how you can work with other people and not be aware of it.
    - dilyevsky 2y ago
      Frankly, I think you might be confusing "productive" with "smart" or "easy to work with". I've had several past team mates at several companies gush to me about staff eng Bob and while I agreed Bob was a cool guy (or just talked good game) I also knew his last few projects were a failure, it was 100% his fault, and management took notice. It happened in the other direction too but not as often. Alternatively you might be underestimating how oblivious some (most?) people are of those things.
      - WalterBright 2y ago
        > I think you might be confusing "productive" with "smart" or "easy to work with". Working in teams for most of my career, and especially when I make or lose money depending on how the team performs, I am not confusing them. At work, I am not really interested in how smart they are or how easy to work with they are, or how nice a person they are. At work I'm interested in results. I've had workers come to me and complain about "Bob" who has annoying habits and an abrasive personality. I defend the ones that produce results. Outside of work, I value smart people who are good friends. I've had several friends who were laid off for poor performance, yet we continued as friends. (Being somewhat on the spectrum myself, I recognize that in others and give them a lot of slack.) > you might be underestimating how oblivious some (most?) people are of those things. The oblivious ones were the ones who got laid off. They honestly believed they were god's gift to the company. Frankly, it was tragic. The ones I kept in contact with would ruefully admit to me years later that the company was right in laying them off.
godelski 2y ago
> The idea of measuring individual developer productivity is kind of absurd to me. It is absurd to do in a lot of settings. The other day we had a similar story with the measurement of "value" just the other day[0]. I think there's been a big bureaucratic takeover where people are measuring for the sake of measuring rather than recognizing that these things are proxies and you need to be careful to ensure your proxy is properly aligned with the actual thing you want to measure. You see this in research and academia by trying to measure by number of papers/citations/venue/novelty. You see this in the workforce with things like lines of code, story points, and any of that other bullshit. You see this with how people think about money. Like just think what a million dollars a month would mean what you __*couldn't*__ do wit that money. The problem is that we've created scoring systems and people see the ultimate goal as maximizing their scores. This doesn't matter if the score is actually meaningful or not, but we'll sure as hell passionately argue that they do. I'll refer to my comment to [0] for more[1]. I've been calling this concept: Goodhart's Hell The truth is that every system has noise in it. I completely understand wanting to remove all possible noise from the system, but after a certain point, attempts to remove noise are actually ignoring noise. Maybe the problem is that people don't recognize that randomness is itself a measurement of uncertainty. There is no measuring device that is without uncertainty. Embrace the noise. Remove as much as you can, but you need to embrace what is impossible to remove. You create more noise by ignoring the noise. [0] https://news.ycombinator.com/item?id=43447616 https://news.ycombinator.com/item?id=43447616 [1] https://news.ycombinator.com/item?id=43448860 https://news.ycombinator.com/item?id=43448860
BobbyTables2 2y ago
Where I work, the CI pipelines measure changes in code coverage. If I refactor repetitive code into something more concise and maintainable —- code coverage goes DOWN! (Now the proportion of untested code is larger!) Yeah, measurement is hard…
jasonkester 2y ago
This is why I like working on my own stuff. All that nonsense goes away and I’m left with just the two numbers that matter to me: - how much money did it bring in this month, and - how many hours did I have to work to make that happen. The goal is to make the first number as big as possible while keeping the other as close to zero as possible.
Sparkyte 2y ago
If you are not committing any code that is a problem. But software development is more than just code. It is also more than just the individual.
naikrovek 2y ago
> The idea of measuring individual developer productivity is kind of absurd to me. that idea is very attractive to every manager on planet earth. they seem to quickly lose their ability to maintain contact with reality and they start using metrics to track everything, rewarding those who game the metrics best.
Grimblewald 2y ago
but also, the reason we come together to work as groups is so that we can achieve more than we would as the sum of the parts. The same is true for horses, two horses that work well together put out more power than two individual horses. So the problem becomes when you identify one of your horses has 1.5x the output of the other, and think firing the weaker horse will lead to minimal productivity gains, you end up with egg on your face when you realise you are not left with a horse that works 1.5x, but rather a horse now working at 0.5x what what it was in a team. The exact same thing is true for team work, team sports, etc. There is no one person 'holding it all together' if there is, then that person needs way better support and realistically, should get the entire departments pay when the remaining department is let go.

mrbadguy 2y ago

Well said. There’s a noticeable lack of judgement in many places these days; people think they can abdicate it to metrics and data but this is mistaken.

jckahn 2y ago

I don’t think it’s that we _can’t_ accurately measure developer value/productivity, it’s that doing so isn’t feasible with any methodology we currently have. As you said, we’re not doing magic, therefore our value is measurable. Measuring just requires an impractical degree of time and energy.

suzzer99 2y ago

Dev productivity is like quantum mechanics. The second you try to measure it, the wave function collapses, and you've fundamentally altered the thing you were trying to measure. That said, at every place I've worked you could get a starting point idea of who the top devs were by looking at overall code commits. This Tim sensei situation may be more common than I think, but I've never run into it.

elevatedastalt 2y ago
Thanks to Goodhart's law, it's true for any measure that becomes a target.
szundi 2y ago
[dead]
lolinder 2y ago
> at every place I've worked you could get a starting point idea of who the top devs were by looking at overall code commits. This works for Junior through Senior level roles, but it falls apart quickly when you have Staff+ roles in your company. Engineers in those roles still code, but they write a fraction of what they used to and that is by design—you want these people engineering entire initiatives, integrations, and migrations. This kind of work is essential and should be done by your top devs, but it will lead to many fewer commits than you get out of people who are working on single features. With a few exceptions for projects with a high degree of source-level complexity, a Staff+ engineer who's committing as much as your Senior engineers is probably either misleveled or misused.
- suzzer99 2y ago
  Yeah, I've always worked at non-tech companies with pretty small teams and no roles like that. But it seems like any place with Staff+ engineers should know better than to try to stack them up against more junior devs based on some metric.
  - philjohn 2y ago
    Case in point, I worked with a Staff software engineer (I was also the same level) who consistently, half over half, had zero diffs to their name. Because they were leaning more into a product archetype - which it turns out they were very suited to.
- Aeolun 2y ago
  I have a decent number of commits as a staff engineer, but barely any are on ‘features’ as measured by story points. We’re building the things that other people use to deliver features better and faster.
winwang 2y ago
With my minor dabbling in game theory, I've considered funny angles like secret-ish internal metrics and punish those who simply game the incentives. Example: bot detections and banwaves in MMOs. Instead of instantly banning plausible bots, the company has to hide its (ever-changing) internal algo and ban in waves instead. Basically, treating (non-)productivity like bot detection, lol.
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
- closeparen 2y ago
  A few years ago we started tracking both office attendance and PR count. Management swore up and down that they understood the nuances, and these metrics would only be used to follow up on outliers. But then one middle manager sets a target. His peers, not to be outdone, set more ambitious targets. And their underlings follow up aggressively with whomever is below target. For a few months there was an implicit "no vacations" policy in much of the company, until they updated the attendance dashboard to account for PTO. And now an engineer's position in the stack rank is presumed to be her position in the PR count rank barring very strong evidence to the contrary. The approach you outline is probably optimal, but I don't think middle management as a culture would be capable of pulling it off. You give them a KPI in a dashboard, they're going to compete on it, in a very open and simple way. Metrics are hard to argue with, and nuance requires constant vigilance to maintain.
gopher_space 2y ago
I've worked on projects where every single ounce of enthusiasm is coming out of one mediocre developer. Nobody pays attention to team dynamics like this, but they're all over the place.
nijave 2y ago
I like looking at lines removed and, in context with, lines added or PR count. Someone more experienced will be refactoring and removing old stuff while adding new stuff and will have some balance (barring vendoring or moving file trees between repos)
- suzzer99 2y ago
  Lines removed should count like 5x lines added. I don't trust any developer who doesn't get a thrill out of removing code.
__turbobrew__ 2y ago
> That said, at every place I've worked you could get a starting point idea of who the top devs were by looking at overall code commits Yea, in the current place I work at we do not measure coding performance metrics, but if you look at the top commiters to the repo — by number of commits — you will see all of the people who bring the most value to the company. Even at staff eng the best engineers are still the ones who crank out code, albeit maybe as not as much as a senior engineer. Staff engineers who only commit 1 a month are usually the ones who don’t bring value to the company, especially given their high pay.
- hamburglar 2y ago
  At my last gig I spent the last year and a half as staff engineer and made almost no commits because I was constantly writing proposals, defending them to execs, doing architecture reviews, doing design consultations, and planning long term responses to large incidents. I know for a fact that I brought a ton of value to the company but it was very uncoupled to commits. I didn’t really like it because I need to code, so I’ve moved on to a role where I commit like a maniac again. :)
  - jimbokun 2y ago
    Just check all of your proposals and design documents into source control, and you’ll still have plenty of commits!
  - __turbobrew__ 2y ago
    Yea it is dependent on your company. When I think of highly functioning staff engineers Im thinking of people like Jeff Dean and Mitchell Hashimoto. Similarly at companies I have been at, even the highest level principal super staff++ engineers still code. They do the high level strategy and design, but they are still writing PRs at least every week or two. If you are spending 100% of your time on politics and architecture as a staff eng I think that shows a somewhat dysfunctional organization.
  - milesrout 2y ago
    Writing proposals, doing architecture reviews and doing design consultations sounds to me largely like busywork and bureaucracy rather than real and productive work. Maybe in a business where the cost of iteration is very high, it makes sense to do this. But in a good business you should not be doing architecture reviews and writing proposals, you should be doing refactoring and creating prototypes, respectively.
closeparen 2y ago
You probably can't do it observationally. You could give several developers the same project, and probably see pretty big differences in timeline and quality. But no one is going to put in the work to run that kind of assessment at scale with the level of realism and time duration (probably several weeks) to make it meaningful.
razeh 2y ago
This is true for more than dev productivity.
xdavidliu 2y ago
> The second you try to measure it, the wave function collapses, and you've fundamentally altered the thing you were trying to measure. Related to Goodhart's law: "When a measure becomes a target, it ceases to be a good measure". I'm wondering if this is only true if the act of measurement is transparent (so that the people being measured can game them). If the act is opaque enough such that the form of the metric cannot easily be guessed SEO-style, then maybe wave function collapse can still be avoided?
naikrovek 2y ago
https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law says (almost) exactly this.

OnionBlender 2y ago

I had a director that was obsessed with github enterprise stats. He forbid people from squashing commits and told people to commit every day, even if you're in the middle of something. This was so that he could see who was writing the most code. One of our interns was close to the end of his term and this director wanted to hire him. He thought the intern was amazing based on the amount of code he wrote. The problem was that this intern was bad, so we had him write unit tests. However, he was also bad at writing unit tests. He would run the code and then write tests that enforced the current results, instead of considering that the current behaviour might be incorrect. Thankfully we didn't hire him after myself and others explained why the intern had so many commits and lines of code written.

lisper 2y ago
> who was writing the most code There's yer problem right there. Code quantity is not correlated with value. In fact, it can be negatively correlated with value if it's buggy and laden with technical debt. Measuring productivity by lines of code produced actively discourages writing clean, maintainable, bug-free code.
- thesuperbigfrog 2y ago
  > Code quantity is not correlated with value. In fact, it can be negatively correlated with value if it's buggy and laden with technical debt. ** "No Code" or Nihilist Software Engineering ** No code runs faster than no code. No code has fewer bugs than no code. No code uses less memory than no code. No code is easier to understand than no code. No code is the best way to have secure and reliable applications. Write nothing; deploy nowhere. One of my most productive days was throwing away 1,000 lines of code. -- Ken Thompson The cheapest, fastest, and most reliable components are those that aren’t there. -- Gordon Bell Deleted code is debugged code. -- Jeff Sickel Measuring programming progress by lines of code is like measuring aircraft building progress by weight. -- Bill Gates * Master Foo and the Ten Thousand Lines * Master Foo once said to a visiting programmer: “There is more Unix-nature in one line of shell script than there is in ten thousand lines of C.” The programmer, who was very proud of his mastery of C, said: “How can this be? C is the language in which the very kernel of Unix is implemented!” Master Foo replied: “That is so. Nevertheless, there is more Unix-nature in one line of shell script than there is in ten thousand lines of C.” The programmer grew distressed. “But through the C language we experience the enlightenment of the Patriarch Ritchie! We become as one with the operating system and the machine, reaping matchless performance!” Master Foo replied: “All that you say is true. But there is still more Unix-nature in one line of shell script than there is in ten thousand lines of C.” The programmer scoffed at Master Foo and rose to depart. But Master Foo nodded to his student Nubi, who wrote a line of shell script on a nearby whiteboard, and said: “Master programmer, consider this pipeline. Implemented in pure C, would it not span ten thousand lines?” The programmer muttered through his beard, contemplating what Nubi had written. Finally he agreed that it was so. “And how many hours would you require to implement and debug that C program?” asked Nubi. “Many,” admitted the visiting programmer. “But only a fool would spend the time to do that when so many more worthy tasks await him.” “And who better understands the Unix-nature?” Master Foo asked. “Is it he who writes the ten thousand lines, or he who, perceiving the emptiness of the task, gains merit by not coding?” Upon hearing this, the programmer was enlightened. Source: http://www.catb.org/~esr/writings/unix-koans/ten-thousand.html http://www.catb.org/~esr/writings/unix-koans/ten-thousand.ht...

fsndz 2y ago

the worst programmer nowadays is the vibe coder. vibe coding is so overrated imo https://www.lycee.ai/blog/why-vibe-coding-is-overrated https://www.lycee.ai/blog/why-vibe-coding-is-overrated

sanex 2y ago
I've been writing code professionally for a decade but I've never written so much code in my free time because I can just vibe code it all. I wouldn't do it at work and I probably wouldn't trust a juniors vibes as much as my own but no tools have made me feel quite so powerful.
- fsndz 2y ago
  it's definitely nice for prototypes, but nothing more complex.
  - eclipxe 2y ago
    I used to feel that way but my perspective over the last 6 months has changed greatly. You can absolutely build very complex things in this style
    - fsndz 2y ago
      show me an example of such a complex thing obtained after just writing prompts
      - godelski 2y ago
        Sometimes you enter a codebase and it looks like there's some obscure attempt to summon a Lovecraftian entity made of spaghetti and duct tape. Those codebases are both the shitiest and most complex stuff I've ever seen. They sure don't work well and I'm not even sure do a good job at summoning the old ones. I don't find LLM code significantly differing from that kind of monstrosity. It sure is complex.
    - 2y ago

slowtrek 2y ago

Could a company keep a subjective poor performer on for the lifespan of the company? As in, what is the plus or minus in overall revenue or profit from this charity? What if all companies did that? Could we distribute the "burden" of the charity across all companies for a better society? My point is, I don't even know if metrics are good or bad, we may need to look at why we see each other like this. Is it so offensive to the mind of the captain of a ship that they may have a few of not the best sailors? It's a chance for them to be on a ship, go on a journey. The concept of a "free ride" appears to be a serious moral hazard for us, but I can't figure out why.

ponector 2y ago
Are you going to accept such free rider in your team? You will eventually do both your and his tasks but for the same or worse pay. Given you have a fixed budget for the team - do you want to split it more ways? Where can I send a CV?
- slowtrek 2y ago
  If we were to do this, we wouldn't just pick someone so out of the range of what's needed. So basically, if you reframe your question, would I hire a 70 average student on a team of 90 average students? Yes. We would not pick the 50 average students. This is all possible within reason, the kind of stuff being discussed in this thread is to purge people who are 80 average students (free riders in a 95 average class). Will we need to tutor and support the 70 average student? Yes. Why would we do this? It's good for the soul. The stuff I'm talking about has no place in business, as far as we care to understand as a society at the moment.
  - rpmisms 2y ago
    My dangerous opinion is that the purpose of a company is to provide a living for its employees.
    - slowtrek 2y ago

nijave 2y ago

I don't think productivity itself is absurd but it's definitely a hard thing to do, especially in small time intervals. I think it's important to measure employee performance and you can definitely can a macro look and say "what have you accomplished in the last year" which might be measured of delivery of "t-shirt" sized projects

matusp 2y ago

It is not absurd. From manager's point of view, when they are deciding to let someone go or promote them, they often need some arguments. And since software is invisible, it is often hard to just say straight away who is over/underperforming and why you think so apart from a "vibe" you are having. Measuring some relevant metrics can strengthen your argument and/or your overview of the situation. I don't think that the blog makes a compelling argument why measuring productivity is bad per se. It makes a compelling argument that metrics should not be interpreted blindly, but the metrics in this case identified a guy that was doing something unusual, and the managers managed to interpret why this is the case. But if it was an IC that is supposed to deliver, or if you did not want "Tim" to spend his time coaching people, this could still have been valuable info.

morsecodist 2y ago
I don't think it is absurd to reason about individual contributions, just to focus on measuring them with metrics. I have been on the management side as well though I admit I am much more of an IC. But the way I think that should go is laddering up to business value by describing what the person did and why that contribution was important. So for example I would say something like: X deserves a promotion because their work was important to delivering project A on time. Project A was difficult but worthwhile because it allowed us as a business to meet goals 1 and 2. That said I mostly work in smaller orgs. I have never been in a situation where a manager would be so removed from a team they would need a sort of proxy metric to direct them where to focus their attention to understand what the people on their team are doing day to day. I can see how this would get more difficult as a company grows.

bob1029 2y ago

> What you want is measuring business outcomes but those are hard to attribute to a particular developer. I think we could solve this by eliminating some middle tiers and putting the developers on the actual customer calls every week. Each senior developer owns at least one customer. That sort of thing. It's a completely different ball game when you are a developer on some B2B/SaaS and you are answering directly to the customer regarding your work. There's no one to hide behind - your teammates are now aside you and can only render supplemental aid. Once you have developers answering directly to the customers, you have a simple, ~boolean, qualitative metric that virtually any manager/investor can get behind: Is the customer happy? If the developers are keeping the customers happy, then they are performing. If the customer is unhappy, then there is a performance issue. Whether or not most or all of it can be attributed to blockers on the customer's side is irrelevant at the senior levels of play. Developers should be helping the customers get unblocked. Offer meetings & screen share time. Generally, make yourself as available as possible. If you do this correctly, the customer is likely to take note and provide you with some amount of cover (i.e., not be in a raging fit when they call your CEO). There is no reality in which you can inject more middle management or process to get developers to take more accountability. The only thing that works is to get them out of the bullshit matrix and in front of the customers. They need to experience how it feels to work directly with a happy customer and then realize how important it is to keep them that way. This path also happens to solve a whole host of other maladies in technology business, most notably shiny rabbit chases. When you have the fear of god in you regarding the client, you aren't going to be as inclined to play in traffic with a NuSQL engine on their watch.

eek2121 2y ago
lol no, I have worked with companies that do that. Many developers have very specific personality types, and they also think in technical terms rather than customer terms. It isn’t a bad thing, it is why they are good at what they do. The most successful companies I have worked with have a good product manager that can take customer input and work with a technical manager to balance priorities/effort. The technical manager consults the team before making decisions (such as SCRUM meetings if using that) Issues come into play when folks start distrusting developers. When we say it will take 4 weeks or 8 weeks to implement something, there is a reason for that. We know the code. We know how much of a PITA it is to work with, and we aren’t being misleading. On the flip side, we have been trained to give conservative estimates thanks to crap management and unexpected pitfalls, so we try to understand promise and over deliver. If management could recognize this, everyone would be happy, provided they recognize that 8-week timeline is fine and they don’t promise something different to the client and trust devs to do their thing. EDIT: Managers also tend to forget that we aren't a machine in a factory. We have good days, bad days, and everything in between. We excel with using our brains, however our brains suffer from anything between lack of sleep, depression, and other nonsense due to just plain having a bad day. I feel like it is more noticeable with us because we rely on our brains so much more than other folks in other professions do. Shoot, even random noise in an office, whether working at home or at an office, can hurt productivity. Now I have myself missing dev work. Hoping to go back soon. Currently unemployed.

WalterBright 2y ago

Everyone on a programming team knows who the good and not-so-good programmers are.

dilyevsky 2y ago
Kind of meaningless statement though. Even if they do, which Im not sure is true in general case because you have to be good yourself to know who’s really good, they are not telling you anyway.
- WalterBright 2y ago
  In every organization I've worked in, everyone knew who the productive and non-productive people were. Just like in school. Everyone knew who the good teachers were, and who the good students were. I don't fathom how you can work with other people and not be aware of it.
  - dilyevsky 2y ago
    Frankly, I think you might be confusing "productive" with "smart" or "easy to work with". I've had several past team mates at several companies gush to me about staff eng Bob and while I agreed Bob was a cool guy (or just talked good game) I also knew his last few projects were a failure, it was 100% his fault, and management took notice. It happened in the other direction too but not as often. Alternatively you might be underestimating how oblivious some (most?) people are of those things.
    - WalterBright 2y ago
      > I think you might be confusing "productive" with "smart" or "easy to work with". Working in teams for most of my career, and especially when I make or lose money depending on how the team performs, I am not confusing them. At work, I am not really interested in how smart they are or how easy to work with they are, or how nice a person they are. At work I'm interested in results. I've had workers come to me and complain about "Bob" who has annoying habits and an abrasive personality. I defend the ones that produce results. Outside of work, I value smart people who are good friends. I've had several friends who were laid off for poor performance, yet we continued as friends. (Being somewhat on the spectrum myself, I recognize that in others and give them a lot of slack.) > you might be underestimating how oblivious some (most?) people are of those things. The oblivious ones were the ones who got laid off. They honestly believed they were god's gift to the company. Frankly, it was tragic. The ones I kept in contact with would ruefully admit to me years later that the company was right in laying them off.

godelski 2y ago

> The idea of measuring individual developer productivity is kind of absurd to me. It is absurd to do in a lot of settings. The other day we had a similar story with the measurement of "value" just the other day[0]. I think there's been a big bureaucratic takeover where people are measuring for the sake of measuring rather than recognizing that these things are proxies and you need to be careful to ensure your proxy is properly aligned with the actual thing you want to measure. You see this in research and academia by trying to measure by number of papers/citations/venue/novelty. You see this in the workforce with things like lines of code, story points, and any of that other bullshit. You see this with how people think about money. Like just think what a million dollars a month would mean what you __*couldn't*__ do wit that money. The problem is that we've created scoring systems and people see the ultimate goal as maximizing their scores. This doesn't matter if the score is actually meaningful or not, but we'll sure as hell passionately argue that they do. I'll refer to my comment to [0] for more[1]. I've been calling this concept: Goodhart's Hell The truth is that every system has noise in it. I completely understand wanting to remove all possible noise from the system, but after a certain point, attempts to remove noise are actually ignoring noise. Maybe the problem is that people don't recognize that randomness is itself a measurement of uncertainty. There is no measuring device that is without uncertainty. Embrace the noise. Remove as much as you can, but you need to embrace what is impossible to remove. You create more noise by ignoring the noise. [0] https://news.ycombinator.com/item?id=43447616 https://news.ycombinator.com/item?id=43447616 [1] https://news.ycombinator.com/item?id=43448860 https://news.ycombinator.com/item?id=43448860

BobbyTables2 2y ago

Where I work, the CI pipelines measure changes in code coverage. If I refactor repetitive code into something more concise and maintainable —- code coverage goes DOWN! (Now the proportion of untested code is larger!) Yeah, measurement is hard…

jasonkester 2y ago

This is why I like working on my own stuff. All that nonsense goes away and I’m left with just the two numbers that matter to me: - how much money did it bring in this month, and - how many hours did I have to work to make that happen. The goal is to make the first number as big as possible while keeping the other as close to zero as possible.

Sparkyte 2y ago

If you are not committing any code that is a problem. But software development is more than just code. It is also more than just the individual.

naikrovek 2y ago

> The idea of measuring individual developer productivity is kind of absurd to me. that idea is very attractive to every manager on planet earth. they seem to quickly lose their ability to maintain contact with reality and they start using metrics to track everything, rewarding those who game the metrics best.

Grimblewald 2y ago

but also, the reason we come together to work as groups is so that we can achieve more than we would as the sum of the parts. The same is true for horses, two horses that work well together put out more power than two individual horses. So the problem becomes when you identify one of your horses has 1.5x the output of the other, and think firing the weaker horse will lead to minimal productivity gains, you end up with egg on your face when you realise you are not left with a horse that works 1.5x, but rather a horse now working at 0.5x what what it was in a team. The exact same thing is true for team work, team sports, etc. There is no one person 'holding it all together' if there is, then that person needs way better support and realistically, should get the entire departments pay when the remaining department is let go.