4 ms·

> Like can we determine the productivity of doctors, lawyers, journalists, or pastry chefs? Yes, yes we can. Programmers really need to stop this cope about u

> Like can we determine the productivity of doctors, lawyers, journalists, or pastry chefs? Yes, yes we can. Programmers really need to stop this cope about us being such special snowflakes that we can't be assessed and that our maangers just need to take that we're worth keeping around on good faith.

bheadmaster 1y ago
> Yes, yes we can. Of course we can. But can we do it in a meaningful way, such that the metric itself doesn't become a subject to optimization? "When a measure becomes a target, it ceases to be a good measure"
- AllegedAlec 1y ago
  > the metric itself doesn't become a subject to optimization By making the metrics part of a sustaintable company-wide goal. If there's a company-wide goal to increase X kind of revenue by Y% making actionable targets on how a team can contribute (not lazy shit like "our changes should contribute Z% of that Y%"), and within that create for a person another smaller metric based on that.
  - StrangeDoctor 1y ago
    This would be difficult to apply to R&D orgs or anything seen as a typical cost center. Also, medical facilities… you certainly could define it as profit, but that bothers me and many other people. You could define it as patients seen, or “cured” but that incentivizes very quick but probably poor care. You could define it as intensity of treatment or amount of care given, but you’d probably end up in a situation where 1 incredibly sick person has every doctor treating them. You could define it as…
    - dickersnoodle 1y ago
      This is the kind of thing that pops up when you try to substitute metrics for judgment. It reminds me of the catastrophic fiscal and monetary policies that emerge from economic theories that rest on bad assumptions (like people being rational actors in an economic system) that are there to make the math work.
  - bheadmaster 1y ago
    It's all so great in theory, where you get to imagine an ideal world in which all our incentives align and we're all rowing on a big boat towards success. In real world, most things don't work out that way. What metrics do you use to measure surgeons' success? If you use fatality rate, then as a result surgeons will refuse to do more risky surgeries which will put their ratings at risk, which makes the healthcare worse, instead of better.
latexr 1y ago
> Yes, yes we can. Could you make an effort to explain how, or at the very least link to some reasoning? Otherwise your comment is basically the equivalent of “nuh-uh”, which doesn’t meaningfully contribute to the discussion. > Programmers really need to stop this cope about us being such special snowflakes Which is not at all what is happening in your parent comment. On the contrary, they’re putting developers on even footing with other professions.
- rcxdude 1y ago
  >Could you make an effort to explain how, or at the very least link to some reasoning? Otherwise your comment is basically the equivalent of “nuh-uh”, which doesn’t meaningfully contribute to the discussion. You can look at the kind of work they're doing, how effective their solutions are, and how long it takes them to do it. That's the basics of it across a wide range of professions. Now, there's no one-size-fits-all metric or formula you can just calculate based on objective facts for most of this, because the work is more varied than e.g. factory work, but it's also not impossible to make the comparison, if you actually understand the work reasonably and you use judgement. In the case of this study, because the assignment of the comparison they were doing was random, then just measuring time to completion across a range of tasks is a perfectly reasonable metric, because there's nothing to really bias the outcome, just a lot of factors that add noise instead. But it is worth noting that the result is a very broad average, and there is likely a very complicated distribution of details underneath, which is much harder to measure.
  - skydhash 1y ago
    > You can look at the kind of work they're doing, how effective their solutions are, and how long it takes them to do it. That's the basics of it across a wide range of professions. Now, there's no one-size-fits-all metric or formula you can just calculate based on objective facts for most of this, because the work is more varied than e.g. factory work, but it's also not impossible to make the comparison, if you actually understand the work reasonably and you use judgement. AKA, be subjective! Which people are wary of, because what it brings is politics and tribalism.
jaredklewis 1y ago
News to me. How do you determine the productivity of a doctor? Patients seen? Patients cured? (for real, where did you get that data?) Number of medicines prescribed? Procedures performed? Does a triple bypass surgery count the same a pap smear? Hours worked? Amount of help they provided to colleagues? Easy to come up with another 100 other metrics that might be worth looking out. How are they all weighted? Like I get that in SWE (like all other fields), managers have to make judgement calls and try to evaluate which reports contribute the most, but the GP post seemed surprised that this wasn't a solved problem by now, which just seems incomprehensible to me.
- AllegedAlec 1y ago
  > How do you determine the productivity of a doctor? At the end of the road. Patient outcome and contentedness compared to others with similar indications. Patients seen and all that is that sort of short-term BS that you see everywhere that's giving metrics a bad name. It'd be like determining a mechanic's productivity by how many times he twisted a wrench.
  - bheadmaster 1y ago
    > Patient outcome and contentedness compared to others with similar indications Which would incentivise doctors to refuse to treat patients who are more ill, lest they risk their ratings go down.
    - deleted 1y ago
      [deleted]
  - jaredklewis 1y ago
    > At the end of the road. Patient outcome and contentedness compared to others with similar indications. Well I would first of all remark that this doesn’t seem and to be how it’s normally done as I’ve never been asked to rate my “contentedness” or similar with my medical care. And where is the “end of the road?” Most medical interventions could be plausibly evaluated at all manner of different intervals. Also, “similar indications” is doing a lot of work here. Patient outcomes are often influenced more by the individual than the doctor. By the time you bucket all the patients by age, diet, activity level, smoking status, alcohol intake, metabolic health, bmi, family history, etc…buckets are going to be pretty tiny. Clinics and hospitals aren’t that big, there won’t be anything to compare. If you only bucket the most obvious categories like age, you’ll have comparisons, but it will just be noise.
tomcam 1y ago
So what methods do you use?
UncleMeat 1y ago
"Software engineers can be qualitatively assessed for the purposes of pay and promotion" and "software engineers can have their productivity measured and quantified" are two very different things.

bheadmaster 1y ago

> Yes, yes we can. Of course we can. But can we do it in a meaningful way, such that the metric itself doesn't become a subject to optimization? "When a measure becomes a target, it ceases to be a good measure"

AllegedAlec 1y ago
> the metric itself doesn't become a subject to optimization By making the metrics part of a sustaintable company-wide goal. If there's a company-wide goal to increase X kind of revenue by Y% making actionable targets on how a team can contribute (not lazy shit like "our changes should contribute Z% of that Y%"), and within that create for a person another smaller metric based on that.
- StrangeDoctor 1y ago
  This would be difficult to apply to R&D orgs or anything seen as a typical cost center. Also, medical facilities… you certainly could define it as profit, but that bothers me and many other people. You could define it as patients seen, or “cured” but that incentivizes very quick but probably poor care. You could define it as intensity of treatment or amount of care given, but you’d probably end up in a situation where 1 incredibly sick person has every doctor treating them. You could define it as…
  - dickersnoodle 1y ago
    This is the kind of thing that pops up when you try to substitute metrics for judgment. It reminds me of the catastrophic fiscal and monetary policies that emerge from economic theories that rest on bad assumptions (like people being rational actors in an economic system) that are there to make the math work.
- bheadmaster 1y ago
  It's all so great in theory, where you get to imagine an ideal world in which all our incentives align and we're all rowing on a big boat towards success. In real world, most things don't work out that way. What metrics do you use to measure surgeons' success? If you use fatality rate, then as a result surgeons will refuse to do more risky surgeries which will put their ratings at risk, which makes the healthcare worse, instead of better.

latexr 1y ago

> Yes, yes we can. Could you make an effort to explain how, or at the very least link to some reasoning? Otherwise your comment is basically the equivalent of “nuh-uh”, which doesn’t meaningfully contribute to the discussion. > Programmers really need to stop this cope about us being such special snowflakes Which is not at all what is happening in your parent comment. On the contrary, they’re putting developers on even footing with other professions.

rcxdude 1y ago
>Could you make an effort to explain how, or at the very least link to some reasoning? Otherwise your comment is basically the equivalent of “nuh-uh”, which doesn’t meaningfully contribute to the discussion. You can look at the kind of work they're doing, how effective their solutions are, and how long it takes them to do it. That's the basics of it across a wide range of professions. Now, there's no one-size-fits-all metric or formula you can just calculate based on objective facts for most of this, because the work is more varied than e.g. factory work, but it's also not impossible to make the comparison, if you actually understand the work reasonably and you use judgement. In the case of this study, because the assignment of the comparison they were doing was random, then just measuring time to completion across a range of tasks is a perfectly reasonable metric, because there's nothing to really bias the outcome, just a lot of factors that add noise instead. But it is worth noting that the result is a very broad average, and there is likely a very complicated distribution of details underneath, which is much harder to measure.
- skydhash 1y ago
  > You can look at the kind of work they're doing, how effective their solutions are, and how long it takes them to do it. That's the basics of it across a wide range of professions. Now, there's no one-size-fits-all metric or formula you can just calculate based on objective facts for most of this, because the work is more varied than e.g. factory work, but it's also not impossible to make the comparison, if you actually understand the work reasonably and you use judgement. AKA, be subjective! Which people are wary of, because what it brings is politics and tribalism.

jaredklewis 1y ago

News to me. How do you determine the productivity of a doctor? Patients seen? Patients cured? (for real, where did you get that data?) Number of medicines prescribed? Procedures performed? Does a triple bypass surgery count the same a pap smear? Hours worked? Amount of help they provided to colleagues? Easy to come up with another 100 other metrics that might be worth looking out. How are they all weighted? Like I get that in SWE (like all other fields), managers have to make judgement calls and try to evaluate which reports contribute the most, but the GP post seemed surprised that this wasn't a solved problem by now, which just seems incomprehensible to me.

AllegedAlec 1y ago
> How do you determine the productivity of a doctor? At the end of the road. Patient outcome and contentedness compared to others with similar indications. Patients seen and all that is that sort of short-term BS that you see everywhere that's giving metrics a bad name. It'd be like determining a mechanic's productivity by how many times he twisted a wrench.
- bheadmaster 1y ago
  > Patient outcome and contentedness compared to others with similar indications Which would incentivise doctors to refuse to treat patients who are more ill, lest they risk their ratings go down.
  - deleted 1y ago
    [deleted]
- jaredklewis 1y ago
  > At the end of the road. Patient outcome and contentedness compared to others with similar indications. Well I would first of all remark that this doesn’t seem and to be how it’s normally done as I’ve never been asked to rate my “contentedness” or similar with my medical care. And where is the “end of the road?” Most medical interventions could be plausibly evaluated at all manner of different intervals. Also, “similar indications” is doing a lot of work here. Patient outcomes are often influenced more by the individual than the doctor. By the time you bucket all the patients by age, diet, activity level, smoking status, alcohol intake, metabolic health, bmi, family history, etc…buckets are going to be pretty tiny. Clinics and hospitals aren’t that big, there won’t be anything to compare. If you only bucket the most obvious categories like age, you’ll have comparisons, but it will just be noise.

tomcam 1y ago

So what methods do you use?

UncleMeat 1y ago

"Software engineers can be qualitatively assessed for the purposes of pay and promotion" and "software engineers can have their productivity measured and quantified" are two very different things.