9 ms·

Never believe stories like this, measuring productivity is an impossible problem. Microsoft spend something in excess of 3 billion dollars trying to quantify p

Never believe stories like this, measuring productivity is an impossible problem. Microsoft spend something in excess of 3 billion dollars trying to quantify productivity in the 90s and they couldn't do it. One of my college professors worked on this exact problem for the Army in the 80s, they created a formal domain specific language for quantifying employee productivity it took them 20 years and several hundred million dollars before they finally realized it wasn't generalizable. They had to scrap the entire project. It isnt possible to measure employee productivity because its impossible to properly quantify the relative importance of a task. Properly scoping problems is a massive task and finally its impossible to adjust for employees gaming the system, collusion or simply malicious behavior. Employees always figure out how to abuse productivity metrics.

yhoneycomb 6y ago
> Employees always figure out how to abuse productivity metrics. If people can abuse your productivity metric, then it's a bad measure of productivity - not a fault of the worker.
- jessaustin 6y ago
  Parent agrees, and further observes that all measures of productivity are "bad".
  - yhoneycomb 6y ago
    Parent does not agree, if parent agreed, then parent would not have used the word "abuse", which implies that the worker is the one at fault. There are many nuances to the English language that are difficult for non-native speakers to pick up.
    - montenegrohugo 6y ago
      > There are many nuances to the English language that are difficult for non-native speakers to pick up. Being condescending is bad, but being condescending whilst also being clearly wrong is doubly so. Get out.
    - t-writescode 6y ago
      English as first language speaker here. Your comment at the end was rude. And I think you’re wrong. I think your parent post agrees with you. Not every conversation includes the requisite specificity of language to make declarations of perfect understanding of original writer’s intent. We will need OP to confirm.
    - xupybd 6y ago
      I think the parent comment reads like it was written by a native speaker. I don't see how the term abuse means that the parent doesn't agree?
    - ponker 6y ago
      Is there a word for the pedantry of an uninformed idiot?
      - jessaustin 6y ago
        I nominate parent: "honeycombing".
- 37ruudueuej 6y ago
  Strongly disagree, by definition it's the abuse that compromises the metric not the abusability. The distinction is important because it means that with a large and rapid enough rotation of obfuscated targets you can find out what you need while mitigating the bulk of intentional metric abuse.
xwdv 6y ago
Just because it cannot be measured to a high precision does not mean it can't be measured. People clearly have some sense of when they are and aren't productive.
- fractionalhare 6y ago
  That doesn't necessarily mean it's quantifiable, rather than qualifiable.
- sgt101 6y ago
  A subjective sense that fails to account for the productivity of the organization over all or the productivity of the team they are in as a whole.
baron_harkonnen 6y ago
> Never believe stories like this, measuring productivity is an impossible problem I want to start off by saying that I don't completely disagree with you, but all of us should take a step back and realize how an absurd of a position this is for us to be in a society. This immeasurability is inversely proportional to the tangible, real value created. You can, for example, absolutely measure to the dollar the productivity of a plumber and you can easily compare two different plumbers, and see if environmental factors improve or decrease their productivity. If you own a factory making screws you can unquestionably measure the valued added and therefor productivity of each and every employee. If you have a farm and you hire seasonal laborers to help you harvest, again you can easily measure the exact productivity of each worker, determine if a hot day makes them less productive etc. Software engineers are a step removed from this in part because the things we make are of questionable value (especially in the extremes). Front-end contract engineers certainly seem to have measurable productivity. If someone can build you a working store front in 40 hours, you can compare this to another one how can do a worse job in 100 hours at the same rate. But as software products grow more complex and the actual value more vague, then software engineer productivity likewise gets harder and harder to measure. And I think a real question every engineer who has worked at a large company has asked, even if they work their asses off, is "do I really create any value?" Software engineers, again, are still closer to the side that you can measure real productivity because in at least most cases software itself is important. When you move on to other "office workers", those who are really struggling now, that value again is very hard to measure and therefore so is productivity. The question that we need to ask more often is, if productivity is so hard to measure, maybe it is possibly because we might not be creating as much value as we have largely tricked ourselves into believing we have.
- Barrin92 6y ago
  >The question that we need to ask more often is, if productivity is so hard to measure, maybe it is possibly because we might not be creating as much value as we have largely tricked ourselves into believing we have. at a very low level productivity is hard to measure because it's tacit and implicit rather than explicit. The reason why we can measure the productivity of a business is because it responds to price signals and you can put a literal number on what they put out. You can try running some sort of market scheme within your company but at some point the trade-offs are bad and it adds too much complexity and creates strange incentives. For software in particular if anything the opposite of your assumption is likely true. It adds a lot of value without necessarily being able to put a price tag on it. It's pretty hard to measure what wikipedia or vim is worth in terms of productivity as long as they're not charging, but that doesn't mean it's not worth anything.
sk5t 6y ago
Great background info; it reminds me--albeit on a greater scale--of an earlier HN thread to the effect of "how hard could possibly be to show street names on maps at with globally correct-seeming detail at arbitrary zoom level," which does sound kind of simple, if you've never tried to do it for real, but which has frustrated many smart people for a long time.
bsder 6y ago
> Never believe stories like this, measuring productivity is an impossible problem. I totally disagree. That's the whole point of your manager and reviews. In addition, JPMorgan isn't just programmers. It's a lot of people who talk to people and process paper--a LOT of paper. And they can quite clearly measure how much paper is being processed per person. > Employees always figure out how to abuse productivity metrics. That's a different problem from being able to see that productivity is up or down. Now, the larger question is whether productivity is down because of work-from-home or productivity is down because the economy is in a complete shambles.
- jrochkind1 6y ago
  Working somewhere where it's thought that 'whole point of my manager' is measuring my productivity sounds like a real nightmare.
- andreilys 6y ago
  The review process is a complete sham. If you honestly believe it’s a measure of your productivity and not a post-hoc rationalization based on a restricted promo and bonus pool I have a bridge to sell you. Most managers will think an employee who spends 10 hours working and 30 hours schmoozing/promoting their work is more productive than someone who spends 60 hours grinding the tasks given to them.
  - WalterBright 6y ago
    It's smart for an employee to let the manager know what he's been doing. It's just like a business has to do some promotion or the business will fail - even great products don't sell themselves.
  - bsder 6y ago
    > The review process is a complete sham. It's a complete sham--up until the moment it's about to fire you. Yeah, I've been in the "Look, that deadline is HARD. If you don't figure out how to do something useful by it, the whole GROUP, including me and you, are fired. So, are you gonna figure something out or should I fire you now and get someone else?" That can wake you up, reeeaaaal quick. > Most managers will think an employee who spends 10 hours working and 30 hours schmoozing/promoting their work is more productive than someone who spends 60 hours grinding the tasks given to them. And you know what? The "schmoozer" might actually be more productive. One of the articles on HN recently was about how senior people tend to avoid work that might get thrown away. "Grinders" are useful, but they can also go grinding in the wrong direction or keep grinding on something that no longer needs to be done because conditions changed and they aren't paying attention. This is especially true if they get too irascible so nobody wants to talk to them. The "schmoozer" might see that the assignment given doesn't quite match up with what the end customer (either internal or external) actually needs and adjust things appropriately. Or, perhaps, just delivers something "close enough" but takes a lot less time and makes the recipient happier. As I became increasingly senior I spent far more time "schmoozing" than getting concrete engineering done.
Aunche 6y ago
> Employees always figure out how to abuse productivity metrics. Targets aren't the same things as measures. Measures of productivity are easy, but targets are not. For example, for a sufficiently large group of people (e.g. all young developers at JP Morgan), you can measure their productivity with how many lines of code they write on average. As long as you aren't using lines of code for performance evaluation, there is no reason any developer will perform needless large refactors or find other means for increasing their delta. Edit: changed metric to target
- Thorentis 6y ago
  Yes, but that proves OP's point. If the productivity metric was equal to the measure of lines of code, then they would figure out that they can abuse the metric by artificially adjusting the output of the measurement.
  - Aunche 6y ago
    Oops. I meant target rather than metric.
- andreilys 6y ago
  Lines of code on average is a terrible measure of productivity. I’ve solved problems with 6 lines of code that took me a week to think of. Other days I write 600 lines of code. I would argue that the former 6 lines I wrote is much more productive than the 600 lines.
  - Aunche 6y ago
    For an individual, lines of code is a bad measure. Some problems are harder than other and some people do more design or leadership work. However, for the collective this all should average out.
    - tsimionescu 6y ago
      No, the problem is that, in general, the more lines of code you have, the worse your program is. Perhaps people working at home feel less pressure to just code, and are thinking more about the problem and writing less code to solve the same problems - a massive win in productivity, which would appear as a loss by your measure. On the other hand, perhaps people are indeed writing the same old code that they always were, just writing less of it. So lines of code are not a good metric. Average number of features/user stories/tasks delivered may be a decent metric (at the group level, not for individual developers), as long as it doesn't become a target. Delivery VS estimate could also be OK, in the same way.
jimmytucson 6y ago
This is what makes me so uncomfortable with hiring software developers. I don’t know how good of a job they’re doing, or what’s taking them so long, or whether another group of developers would do a better job faster. If I hire painters, I can tell if they’re doing a good job by looking at it. I know what’s taking them so long. They’re using slower drying drywall compound, or they’re having to mask off because they’re not good with a brush. There are fields of work where it’s easy to gauge productivity, but software development is not one of them.
- icelancer 6y ago
  >> There are fields of work where it’s easy to gauge productivity, but software development is not one of them. I manage a team of developers / analysts and it's straightforward (though not always easy) to measure this over several six-week blocks of time for the team. It is not, however, easy to measure it for any single developer or analyst, and there's a lot of variance when you try to look any more granular than the team. I suspect it's doable for small groups that work together. Or even larger groups. (I have no experience here.) But for individuals? I agree. No shot.
- onion2k 6y ago
  If I hire painters, I can tell if they’re doing a good job by looking at it. It's easy to make a job look good for a day, but on some jobs if you skip the prep work, lining, filling, priming, undercoat, additional coats, clearcoat, and finishing the paint job is going to look pretty bad after a year. You can't tell how good a painter is unless you know a bit about painting things. The same is true of software. In order to know if a developer is productive you really need to know quite a lot about the domain they're writing software for.
WalterBright 6y ago
On the other hand, everyone in the office knows who the productive and unproductive people are. It's also true that many people have a very different assessment of their productivity than their colleagues do of them. I've seen some pretty massive disconnects there, which usually lead to trouble.
cryptonector 6y ago
At Sun we experienced a new metrics fad every other year. Once they tried to use bug report clearance rate, so a friend said "I'm going to code myself a minivan" at a meeting. That metric didn't last. You have to have a great deal of sympathy for the management of software-heavy companies. They've no idea how to manage software development. Nobody really knows how to. It sucks. Oh, well, I suppose there is a way: mind the bottom line, do not over-hire, and have good managers of the old sort: the type that know how to keep tabs on their direct reports without being overbearing. Business schools won't go for that.
- MrMorden 6y ago
  Dilbert had this punchline in 1995: https://dilbert.com/strip/1995-11-13 https://dilbert.com/strip/1995-11-13 It's possible one of Scott Adams's moles was at this meeting, since they seem to be omnipresent and omniscient.
  - cryptonector 6y ago
    Ahhh, I'd missed that. Thanks for setting me straight!
- andyjohnson0 6y ago
  Goodhart's Law in action: "When a measure becomes a target, it ceases to be a good measure.” https://en.m.wikipedia.org/wiki/Goodhart%27s_law https://en.m.wikipedia.org/wiki/Goodhart%27s_law
  - seventhtiger 6y ago
    This sounds true and I've seen it in action. But I always wonder about the implications. Must we keep our most important measures target free? That sounds insane, although I can see how it would improve the measurement. So the questions is how are we supposed to set targets without our most important measures?
- swarnie_ 6y ago
  > "I'm going to code myself a minivan" Could you explain this term? I thought i was a native English speaker but this has lost me.
baryphonic 6y ago
100% correct. If it were possible to measure productivity, the Soviet Union would have won the Cold War and cleaned the capitalists' clocks.
alexbanks 6y ago
And if you were going to believe a company's metrics on productivity, JP Morgan is almost certainly not the place to start.
alephu5 6y ago
It doesn't surprise me that your friend failed on this project but it isn't obvious a priori that measuring productivity is futile. But in my current job they're trying to create KPIs for us, so I've thought about this a bit and can certainly recognise the difficulty. As an engineer in general how can you quantify the value of maintenance? On paper it's just a cost and it's impossible to know what would have happened without it. The worst strategy is just to measure easy things like story points or number of tasks completed because it ruins the work culture. In those cases you're not going to take time to review pull requests carefully, advise colleagues, propose new projects or work on difficult tasks. It seems impossible to measure these, at least to me.
Kiro 6y ago
> Employees always figure out how to abuse productivity metrics. Yes, and working from home makes it even easier. Your post basically says that any measured productivity gain from WFH can't be trusted but I don't understand why it would apply to a measured productivity decrease.
1e-9 6y ago
> Never believe stories like this, measuring productivity is an impossible problem. While I do agree that many productivity metrics are useless and even counterproductive, measuring productivity in a useful way is most certainly possible. For example, sales organizations measure sales per unit time for each sales person. I doubt you will find a successful sales organization not doing this.
- Robin_Message 6y ago
  Even then, what if one salesperson builds up better longer term relationships (e.g. by not pushing options the customer doesn't really need) and thus their future sales will be better but their initial productivity looks poor? Or a team player who is good at opening conversations but not closing? (I know, always be closing) But yes, with a quantifiable and accurate and business-orientated metric, you can measure some form of productivity. It is still hard for most roles in most organisations.
  - 1e-9 6y ago
    Modifying a now trite-but-true saying from the statistical modeling world: “All productivity metrics are wrong, but some are useful.” I imagine there will always be an infinite number of ways an employee could add value that would be inadequately reflected in any finite collection of metrics, but that does not eliminate the usefulness of good metrics. Managerial judgement is always needed to account for exceptions that require accommodation.
    - bob33212 6y ago
      Metrics will never be perfect, they are an approximation of the real world. And some things are unknowable. Maybe a customer goes bankrupt 3 months into a project and you never collect the full revenue. One metric shows that a salesperson closed that deal and another metric shows that his revenue per customer isn't good.
  - metacork 6y ago
    A successful team structure will make it difficult to measure individual productivity because team members spend some time on shared team priorities rather than just individual priorities. I enjoy team work and I push hard against managers focusing on individual performance metrics because it destroys team work.
lanevorockz 6y ago
Quick correction, this problem is far from impossible. A multivariate model and about 5/10 years of data and you can easily get an answer to that.
jariel 6y ago
This is hyperbole. There are many ways to measure productivity depending on the task, and often you can simply ask managers who have a good feel for how much progress is being made. It may be that remoters are 'trying hard' but that the limitations of remote are not overcome, but this is still a lack of productivity.
SergeAx 6y ago
We can't measure productivity of creative workers to the cent, but we of course can use various proxies to assess the productivity. This often can lead to unwanted outcomes (like Uber's criteria of being innovative led to microservices hell), but in general you can't dismiss those proxy measurements as totally irrelevant.
klmadfejno 6y ago
I don't agree that measuring productivity is impossible. It's obviously possible for some roles, especially sales, which seems to be the focus here. That said, it's dubious whether or not they've got anything real here. They say productivity fell, especially on monday and friday. Probably implies they're looking at: * Login time on the VPN * Number of meetings or some other proxy for activity * Sales numbers The first one might be right. People might be spending less time working, when working remotely. The mistake is in assuming that they would be contributing any sort of value if they were in the office and logged in 100% of the time. The second one might be right too, but again, may not correspond to anything The third one, if true, probably has less to do with the productivity of the workers and more to do with the economic situation. But who knows? They may have something real.
SilasX 6y ago
That seems like way too strong a claim to make, since it would also mean that you can’t claim anyone is just as productive after going remote.
_glass 6y ago
Management is part of the social sciences, and there there was a big shift away from the quantitative methods (e.g. measuring productivity) towards qualitative methods, where one can better understand why managers are now saying that the productivity fell. And what is behind those reasons.
hnarn 6y ago
I would argue that measuring productivity is only possible if you’re trying to detect the lack of it. Ie it’s definitely possible to see if someone’s slacking behind the rest if whatever output they’re expected to create is way less than the average of all colleagues, but at some point you have to accept that the confidence intervals are pretty vague. In a situation where workload is at least sort of randomly distributed, which it normally is given time, there’s no other good explanation for statistical deviancy than simply less “work” being done. But again, this only works for detecting problems. As you point out, and I agree with you, as soon as you attempt to measure “change in productivity” you’re on very thin ice.

yhoneycomb 6y ago

> Employees always figure out how to abuse productivity metrics. If people can abuse your productivity metric, then it's a bad measure of productivity - not a fault of the worker.

jessaustin 6y ago
Parent agrees, and further observes that all measures of productivity are "bad".
- yhoneycomb 6y ago
  Parent does not agree, if parent agreed, then parent would not have used the word "abuse", which implies that the worker is the one at fault. There are many nuances to the English language that are difficult for non-native speakers to pick up.
  - montenegrohugo 6y ago
    > There are many nuances to the English language that are difficult for non-native speakers to pick up. Being condescending is bad, but being condescending whilst also being clearly wrong is doubly so. Get out.
  - t-writescode 6y ago
    English as first language speaker here. Your comment at the end was rude. And I think you’re wrong. I think your parent post agrees with you. Not every conversation includes the requisite specificity of language to make declarations of perfect understanding of original writer’s intent. We will need OP to confirm.
  - xupybd 6y ago
    I think the parent comment reads like it was written by a native speaker. I don't see how the term abuse means that the parent doesn't agree?
  - ponker 6y ago
    Is there a word for the pedantry of an uninformed idiot?
    - jessaustin 6y ago
      I nominate parent: "honeycombing".
37ruudueuej 6y ago
Strongly disagree, by definition it's the abuse that compromises the metric not the abusability. The distinction is important because it means that with a large and rapid enough rotation of obfuscated targets you can find out what you need while mitigating the bulk of intentional metric abuse.

xwdv 6y ago

Just because it cannot be measured to a high precision does not mean it can't be measured. People clearly have some sense of when they are and aren't productive.

fractionalhare 6y ago
That doesn't necessarily mean it's quantifiable, rather than qualifiable.
sgt101 6y ago
A subjective sense that fails to account for the productivity of the organization over all or the productivity of the team they are in as a whole.

baron_harkonnen 6y ago

> Never believe stories like this, measuring productivity is an impossible problem I want to start off by saying that I don't completely disagree with you, but all of us should take a step back and realize how an absurd of a position this is for us to be in a society. This immeasurability is inversely proportional to the tangible, real value created. You can, for example, absolutely measure to the dollar the productivity of a plumber and you can easily compare two different plumbers, and see if environmental factors improve or decrease their productivity. If you own a factory making screws you can unquestionably measure the valued added and therefor productivity of each and every employee. If you have a farm and you hire seasonal laborers to help you harvest, again you can easily measure the exact productivity of each worker, determine if a hot day makes them less productive etc. Software engineers are a step removed from this in part because the things we make are of questionable value (especially in the extremes). Front-end contract engineers certainly seem to have measurable productivity. If someone can build you a working store front in 40 hours, you can compare this to another one how can do a worse job in 100 hours at the same rate. But as software products grow more complex and the actual value more vague, then software engineer productivity likewise gets harder and harder to measure. And I think a real question every engineer who has worked at a large company has asked, even if they work their asses off, is "do I really create any value?" Software engineers, again, are still closer to the side that you can measure real productivity because in at least most cases software itself is important. When you move on to other "office workers", those who are really struggling now, that value again is very hard to measure and therefore so is productivity. The question that we need to ask more often is, if productivity is so hard to measure, maybe it is possibly because we might not be creating as much value as we have largely tricked ourselves into believing we have.

Barrin92 6y ago
>The question that we need to ask more often is, if productivity is so hard to measure, maybe it is possibly because we might not be creating as much value as we have largely tricked ourselves into believing we have. at a very low level productivity is hard to measure because it's tacit and implicit rather than explicit. The reason why we can measure the productivity of a business is because it responds to price signals and you can put a literal number on what they put out. You can try running some sort of market scheme within your company but at some point the trade-offs are bad and it adds too much complexity and creates strange incentives. For software in particular if anything the opposite of your assumption is likely true. It adds a lot of value without necessarily being able to put a price tag on it. It's pretty hard to measure what wikipedia or vim is worth in terms of productivity as long as they're not charging, but that doesn't mean it's not worth anything.

sk5t 6y ago

Great background info; it reminds me--albeit on a greater scale--of an earlier HN thread to the effect of "how hard could possibly be to show street names on maps at with globally correct-seeming detail at arbitrary zoom level," which does sound kind of simple, if you've never tried to do it for real, but which has frustrated many smart people for a long time.

bsder 6y ago

> Never believe stories like this, measuring productivity is an impossible problem. I totally disagree. That's the whole point of your manager and reviews. In addition, JPMorgan isn't just programmers. It's a lot of people who talk to people and process paper--a LOT of paper. And they can quite clearly measure how much paper is being processed per person. > Employees always figure out how to abuse productivity metrics. That's a different problem from being able to see that productivity is up or down. Now, the larger question is whether productivity is down because of work-from-home or productivity is down because the economy is in a complete shambles.

jrochkind1 6y ago
Working somewhere where it's thought that 'whole point of my manager' is measuring my productivity sounds like a real nightmare.
andreilys 6y ago
The review process is a complete sham. If you honestly believe it’s a measure of your productivity and not a post-hoc rationalization based on a restricted promo and bonus pool I have a bridge to sell you. Most managers will think an employee who spends 10 hours working and 30 hours schmoozing/promoting their work is more productive than someone who spends 60 hours grinding the tasks given to them.
- WalterBright 6y ago
  It's smart for an employee to let the manager know what he's been doing. It's just like a business has to do some promotion or the business will fail - even great products don't sell themselves.
- bsder 6y ago
  > The review process is a complete sham. It's a complete sham--up until the moment it's about to fire you. Yeah, I've been in the "Look, that deadline is HARD. If you don't figure out how to do something useful by it, the whole GROUP, including me and you, are fired. So, are you gonna figure something out or should I fire you now and get someone else?" That can wake you up, reeeaaaal quick. > Most managers will think an employee who spends 10 hours working and 30 hours schmoozing/promoting their work is more productive than someone who spends 60 hours grinding the tasks given to them. And you know what? The "schmoozer" might actually be more productive. One of the articles on HN recently was about how senior people tend to avoid work that might get thrown away. "Grinders" are useful, but they can also go grinding in the wrong direction or keep grinding on something that no longer needs to be done because conditions changed and they aren't paying attention. This is especially true if they get too irascible so nobody wants to talk to them. The "schmoozer" might see that the assignment given doesn't quite match up with what the end customer (either internal or external) actually needs and adjust things appropriately. Or, perhaps, just delivers something "close enough" but takes a lot less time and makes the recipient happier. As I became increasingly senior I spent far more time "schmoozing" than getting concrete engineering done.

Aunche 6y ago

> Employees always figure out how to abuse productivity metrics. Targets aren't the same things as measures. Measures of productivity are easy, but targets are not. For example, for a sufficiently large group of people (e.g. all young developers at JP Morgan), you can measure their productivity with how many lines of code they write on average. As long as you aren't using lines of code for performance evaluation, there is no reason any developer will perform needless large refactors or find other means for increasing their delta. Edit: changed metric to target

Thorentis 6y ago
Yes, but that proves OP's point. If the productivity metric was equal to the measure of lines of code, then they would figure out that they can abuse the metric by artificially adjusting the output of the measurement.
- Aunche 6y ago
  Oops. I meant target rather than metric.
andreilys 6y ago
Lines of code on average is a terrible measure of productivity. I’ve solved problems with 6 lines of code that took me a week to think of. Other days I write 600 lines of code. I would argue that the former 6 lines I wrote is much more productive than the 600 lines.
- Aunche 6y ago
  For an individual, lines of code is a bad measure. Some problems are harder than other and some people do more design or leadership work. However, for the collective this all should average out.
  - tsimionescu 6y ago
    No, the problem is that, in general, the more lines of code you have, the worse your program is. Perhaps people working at home feel less pressure to just code, and are thinking more about the problem and writing less code to solve the same problems - a massive win in productivity, which would appear as a loss by your measure. On the other hand, perhaps people are indeed writing the same old code that they always were, just writing less of it. So lines of code are not a good metric. Average number of features/user stories/tasks delivered may be a decent metric (at the group level, not for individual developers), as long as it doesn't become a target. Delivery VS estimate could also be OK, in the same way.

jimmytucson 6y ago

This is what makes me so uncomfortable with hiring software developers. I don’t know how good of a job they’re doing, or what’s taking them so long, or whether another group of developers would do a better job faster. If I hire painters, I can tell if they’re doing a good job by looking at it. I know what’s taking them so long. They’re using slower drying drywall compound, or they’re having to mask off because they’re not good with a brush. There are fields of work where it’s easy to gauge productivity, but software development is not one of them.

icelancer 6y ago
>> There are fields of work where it’s easy to gauge productivity, but software development is not one of them. I manage a team of developers / analysts and it's straightforward (though not always easy) to measure this over several six-week blocks of time for the team. It is not, however, easy to measure it for any single developer or analyst, and there's a lot of variance when you try to look any more granular than the team. I suspect it's doable for small groups that work together. Or even larger groups. (I have no experience here.) But for individuals? I agree. No shot.
onion2k 6y ago
If I hire painters, I can tell if they’re doing a good job by looking at it. It's easy to make a job look good for a day, but on some jobs if you skip the prep work, lining, filling, priming, undercoat, additional coats, clearcoat, and finishing the paint job is going to look pretty bad after a year. You can't tell how good a painter is unless you know a bit about painting things. The same is true of software. In order to know if a developer is productive you really need to know quite a lot about the domain they're writing software for.

WalterBright 6y ago

On the other hand, everyone in the office knows who the productive and unproductive people are. It's also true that many people have a very different assessment of their productivity than their colleagues do of them. I've seen some pretty massive disconnects there, which usually lead to trouble.

cryptonector 6y ago

At Sun we experienced a new metrics fad every other year. Once they tried to use bug report clearance rate, so a friend said "I'm going to code myself a minivan" at a meeting. That metric didn't last. You have to have a great deal of sympathy for the management of software-heavy companies. They've no idea how to manage software development. Nobody really knows how to. It sucks. Oh, well, I suppose there is a way: mind the bottom line, do not over-hire, and have good managers of the old sort: the type that know how to keep tabs on their direct reports without being overbearing. Business schools won't go for that.

MrMorden 6y ago
Dilbert had this punchline in 1995: https://dilbert.com/strip/1995-11-13 https://dilbert.com/strip/1995-11-13 It's possible one of Scott Adams's moles was at this meeting, since they seem to be omnipresent and omniscient.
- cryptonector 6y ago
  Ahhh, I'd missed that. Thanks for setting me straight!
andyjohnson0 6y ago
Goodhart's Law in action: "When a measure becomes a target, it ceases to be a good measure.” https://en.m.wikipedia.org/wiki/Goodhart%27s_law https://en.m.wikipedia.org/wiki/Goodhart%27s_law
- seventhtiger 6y ago
  This sounds true and I've seen it in action. But I always wonder about the implications. Must we keep our most important measures target free? That sounds insane, although I can see how it would improve the measurement. So the questions is how are we supposed to set targets without our most important measures?
swarnie_ 6y ago
> "I'm going to code myself a minivan" Could you explain this term? I thought i was a native English speaker but this has lost me.

baryphonic 6y ago

100% correct. If it were possible to measure productivity, the Soviet Union would have won the Cold War and cleaned the capitalists' clocks.

alexbanks 6y ago

And if you were going to believe a company's metrics on productivity, JP Morgan is almost certainly not the place to start.

alephu5 6y ago

It doesn't surprise me that your friend failed on this project but it isn't obvious a priori that measuring productivity is futile. But in my current job they're trying to create KPIs for us, so I've thought about this a bit and can certainly recognise the difficulty. As an engineer in general how can you quantify the value of maintenance? On paper it's just a cost and it's impossible to know what would have happened without it. The worst strategy is just to measure easy things like story points or number of tasks completed because it ruins the work culture. In those cases you're not going to take time to review pull requests carefully, advise colleagues, propose new projects or work on difficult tasks. It seems impossible to measure these, at least to me.

Kiro 6y ago

> Employees always figure out how to abuse productivity metrics. Yes, and working from home makes it even easier. Your post basically says that any measured productivity gain from WFH can't be trusted but I don't understand why it would apply to a measured productivity decrease.

1e-9 6y ago

> Never believe stories like this, measuring productivity is an impossible problem. While I do agree that many productivity metrics are useless and even counterproductive, measuring productivity in a useful way is most certainly possible. For example, sales organizations measure sales per unit time for each sales person. I doubt you will find a successful sales organization not doing this.

Robin_Message 6y ago
Even then, what if one salesperson builds up better longer term relationships (e.g. by not pushing options the customer doesn't really need) and thus their future sales will be better but their initial productivity looks poor? Or a team player who is good at opening conversations but not closing? (I know, always be closing) But yes, with a quantifiable and accurate and business-orientated metric, you can measure some form of productivity. It is still hard for most roles in most organisations.
- 1e-9 6y ago
  Modifying a now trite-but-true saying from the statistical modeling world: “All productivity metrics are wrong, but some are useful.” I imagine there will always be an infinite number of ways an employee could add value that would be inadequately reflected in any finite collection of metrics, but that does not eliminate the usefulness of good metrics. Managerial judgement is always needed to account for exceptions that require accommodation.
  - bob33212 6y ago
    Metrics will never be perfect, they are an approximation of the real world. And some things are unknowable. Maybe a customer goes bankrupt 3 months into a project and you never collect the full revenue. One metric shows that a salesperson closed that deal and another metric shows that his revenue per customer isn't good.
- metacork 6y ago
  A successful team structure will make it difficult to measure individual productivity because team members spend some time on shared team priorities rather than just individual priorities. I enjoy team work and I push hard against managers focusing on individual performance metrics because it destroys team work.

lanevorockz 6y ago

Quick correction, this problem is far from impossible. A multivariate model and about 5/10 years of data and you can easily get an answer to that.

jariel 6y ago

This is hyperbole. There are many ways to measure productivity depending on the task, and often you can simply ask managers who have a good feel for how much progress is being made. It may be that remoters are 'trying hard' but that the limitations of remote are not overcome, but this is still a lack of productivity.

SergeAx 6y ago

We can't measure productivity of creative workers to the cent, but we of course can use various proxies to assess the productivity. This often can lead to unwanted outcomes (like Uber's criteria of being innovative led to microservices hell), but in general you can't dismiss those proxy measurements as totally irrelevant.

klmadfejno 6y ago

I don't agree that measuring productivity is impossible. It's obviously possible for some roles, especially sales, which seems to be the focus here. That said, it's dubious whether or not they've got anything real here. They say productivity fell, especially on monday and friday. Probably implies they're looking at: * Login time on the VPN * Number of meetings or some other proxy for activity * Sales numbers The first one might be right. People might be spending less time working, when working remotely. The mistake is in assuming that they would be contributing any sort of value if they were in the office and logged in 100% of the time. The second one might be right too, but again, may not correspond to anything The third one, if true, probably has less to do with the productivity of the workers and more to do with the economic situation. But who knows? They may have something real.

SilasX 6y ago

That seems like way too strong a claim to make, since it would also mean that you can’t claim anyone is just as productive after going remote.

_glass 6y ago

Management is part of the social sciences, and there there was a big shift away from the quantitative methods (e.g. measuring productivity) towards qualitative methods, where one can better understand why managers are now saying that the productivity fell. And what is behind those reasons.

hnarn 6y ago

I would argue that measuring productivity is only possible if you’re trying to detect the lack of it. Ie it’s definitely possible to see if someone’s slacking behind the rest if whatever output they’re expected to create is way less than the average of all colleagues, but at some point you have to accept that the confidence intervals are pretty vague. In a situation where workload is at least sort of randomly distributed, which it normally is given time, there’s no other good explanation for statistical deviancy than simply less “work” being done. But again, this only works for detecting problems. As you point out, and I agree with you, as soon as you attempt to measure “change in productivity” you’re on very thin ice.