3 ms·

> Good developer experience increases self-reported productivity. So do ineffective DX, most times. There's "proper" way to assess productivity gains that don'

by herval 3y ago

> Good developer experience increases self-reported productivity. So do ineffective DX, most times. There's "proper" way to assess productivity gains that don't rely on subjective self-reporting (eg the DORA metrics). The "improving feedback loops" item on this report makes sense on that light - the other two are questionable as a direct proxy

Paul-Craft 3y ago
> So do ineffective DX, most times. I don't find this hard to believe, but do you know of any research showing this?
- herval 3y ago
  I haven't looked for research tbh, just personal experience. It's kinda hard to prove a negative tbh (how do you organize a research that can measure if something subjectively "bad" made people subjectively less effective?). I've been to multiple organizations that invested heavily in DX, and as an external observer you could see they became actually slower (eg one went from test-in-prod and deploy-to-prod to a convoluted CI pipeline) - it made engineers feel like they were doing a lot less work (you just push code and forget), but in reality, an entire department was hired to deal with issues in prod, and the teams didn't really deliver anything faster at all. But 100% of the devs swore it was better/faster/more efficient. On another, the added free time converted directly into more complexity, so tons of overengineered microservices (sometimes to the tune of 2-3x the number of individuals on a team) ensued - it got so easy to do X, everyone overdid it.
makeitdouble 3y ago
> DORA metrics I generally agree with your point, with the caveat that metrics like DORA are the first to be gamed when applied to an organization. It's the usual cycle of applying the metrics first in an informal way, then using it for team/process evaluations as it seems to be somewhat reliable, then teams start to reajust how they make the tickets and the deploy cycle in ways that dramatically increase the metrics while reality hasn't changed much. Analyzing productivity with completely objective metrics will I think always bear these flaws (I'd assume even measuring things at the highest level as "wages vs revenue" would be gamed on how wages and revenue are calculated ?)
- herval 3y ago
  when applied as a metric, yes - so is any metric that becomes a goal (goodhart's law?). To _assess_ an organization, they're pretty solid

by herval 3y ago

Paul-Craft 3y ago

> So do ineffective DX, most times. I don't find this hard to believe, but do you know of any research showing this?

herval 3y ago
I haven't looked for research tbh, just personal experience. It's kinda hard to prove a negative tbh (how do you organize a research that can measure if something subjectively "bad" made people subjectively less effective?). I've been to multiple organizations that invested heavily in DX, and as an external observer you could see they became actually slower (eg one went from test-in-prod and deploy-to-prod to a convoluted CI pipeline) - it made engineers feel like they were doing a lot less work (you just push code and forget), but in reality, an entire department was hired to deal with issues in prod, and the teams didn't really deliver anything faster at all. But 100% of the devs swore it was better/faster/more efficient. On another, the added free time converted directly into more complexity, so tons of overengineered microservices (sometimes to the tune of 2-3x the number of individuals on a team) ensued - it got so easy to do X, everyone overdid it.

makeitdouble 3y ago

> DORA metrics I generally agree with your point, with the caveat that metrics like DORA are the first to be gamed when applied to an organization. It's the usual cycle of applying the metrics first in an informal way, then using it for team/process evaluations as it seems to be somewhat reliable, then teams start to reajust how they make the tickets and the deploy cycle in ways that dramatically increase the metrics while reality hasn't changed much. Analyzing productivity with completely objective metrics will I think always bear these flaws (I'd assume even measuring things at the highest level as "wages vs revenue" would be gamed on how wages and revenue are calculated ?)

herval 3y ago
when applied as a metric, yes - so is any metric that becomes a goal (goodhart's law?). To _assess_ an organization, they're pretty solid