5 ms·

Surely relying on deployments as a measure of developer productivity is pretty flawed. You could deploy the same code over two deployments instead of one or dep

Surely relying on deployments as a measure of developer productivity is pretty flawed. You could deploy the same code over two deployments instead of one or deploy code that has no meaningful impact on the product and the metrics would look good but actual value added would be low or zero.

gajus 6y ago
Deploying small changes derisks large outages. The smaller the change, the easier it is to revert it. This in turn translates to less time spent debugging and panic-shifting priorities to fix production deployments. Therefore, there is a relationship. As to low value contributions, their consensus process seems to filter out the extreme cases.
mumblemumble 6y ago
This is where metrics and KPIs part ways. Metrics are often poor proxies for the thing you actually want to drive, but a thoughtful leader can easily mitigate that effect by monitoring several metrics and understanding how they combine to form a more holistic picture. That can all fall apart rather quickly as soon as you turn something you're measuring into a performance indicator that will be imposed on people as a way of assessing how well they're doing their job. If there's any room at all to game the numbers, then you've just created a huge incentive to do so, and it will happen. At which point the thing you're measuring has stopped being a metric in any meaningful sense of the term.
- mdgrech23 6y ago
  I can think of a good example of this. I worked at a fast food chain in high school. Our goal was 30 seconds from the time a car pulled up and started to order over the loud speaker to the time they pulled away w/ their food. We would rush the customer through the ordering process to the detriment of customer service and often have them pull over to the side to collect their order to cheat the clock.
  - mumblemumble 6y ago
    I used to work in the kitchen at a restaurant that did similar things with time to get food out to tables. I don't know exactly how much beef I wasted due to keeping anticipatory burgers on the grill, even when I had zero open orders in the queue. All I know is, when I die, I'll have to have a very uncomfortable conversation with St Peter while a whole lot of angry cattle are scowling over his shoulders at me. Such a waste.
hoorayimhelping 6y ago
If you're trying to build a system and a culture that deploys frequently and often, this is a great way to do it. You want engineers deploying smaller changes more frequently - smaller changes are easier to review, easier to reason about if something goes wrong and easier to revert if they need to be. You want to incentivize people to deploy as often as possible because it forces people to address the pain points of deploying. The idea is you build a cultural idea and drive towards a goal, in this case deploying on the first day, and use the cultural goal to address the technical pain points. All of this seems odd for coinbase though. We did stuff like that at Etsy, because the key insight was that it's cheaper to fix errors than it is to prevent them in an e-commerce context. A hiccup on an e-commerce page just forces a reload and it's usually not a big deal. A hiccup in a bank transaction is much more frightening.
pgroves 6y ago
The same code split across two pull requests is preferable to the rest of the team (assuming they are actually trying to understand the PRs). Sort of a clever way to incentivize that.
- JamesSwift 6y ago
  I'd argue its a way to _hopefully_ incentivize that. Number of deploys has no inherent value though, so optimizing it opens you up to pointless progress. How about, instead, you optimize _time to deploy_. i.e. the delta between code submitted for review and code running in production. Then, who cares if we didn't deploy 5 times today. Maybe we didn't have 5 things that needed updated.

gajus 6y ago

Deploying small changes derisks large outages. The smaller the change, the easier it is to revert it. This in turn translates to less time spent debugging and panic-shifting priorities to fix production deployments. Therefore, there is a relationship. As to low value contributions, their consensus process seems to filter out the extreme cases.

mumblemumble 6y ago

This is where metrics and KPIs part ways. Metrics are often poor proxies for the thing you actually want to drive, but a thoughtful leader can easily mitigate that effect by monitoring several metrics and understanding how they combine to form a more holistic picture. That can all fall apart rather quickly as soon as you turn something you're measuring into a performance indicator that will be imposed on people as a way of assessing how well they're doing their job. If there's any room at all to game the numbers, then you've just created a huge incentive to do so, and it will happen. At which point the thing you're measuring has stopped being a metric in any meaningful sense of the term.

mdgrech23 6y ago
I can think of a good example of this. I worked at a fast food chain in high school. Our goal was 30 seconds from the time a car pulled up and started to order over the loud speaker to the time they pulled away w/ their food. We would rush the customer through the ordering process to the detriment of customer service and often have them pull over to the side to collect their order to cheat the clock.
- mumblemumble 6y ago
  I used to work in the kitchen at a restaurant that did similar things with time to get food out to tables. I don't know exactly how much beef I wasted due to keeping anticipatory burgers on the grill, even when I had zero open orders in the queue. All I know is, when I die, I'll have to have a very uncomfortable conversation with St Peter while a whole lot of angry cattle are scowling over his shoulders at me. Such a waste.

hoorayimhelping 6y ago

If you're trying to build a system and a culture that deploys frequently and often, this is a great way to do it. You want engineers deploying smaller changes more frequently - smaller changes are easier to review, easier to reason about if something goes wrong and easier to revert if they need to be. You want to incentivize people to deploy as often as possible because it forces people to address the pain points of deploying. The idea is you build a cultural idea and drive towards a goal, in this case deploying on the first day, and use the cultural goal to address the technical pain points. All of this seems odd for coinbase though. We did stuff like that at Etsy, because the key insight was that it's cheaper to fix errors than it is to prevent them in an e-commerce context. A hiccup on an e-commerce page just forces a reload and it's usually not a big deal. A hiccup in a bank transaction is much more frightening.

pgroves 6y ago

The same code split across two pull requests is preferable to the rest of the team (assuming they are actually trying to understand the PRs). Sort of a clever way to incentivize that.

JamesSwift 6y ago
I'd argue its a way to _hopefully_ incentivize that. Number of deploys has no inherent value though, so optimizing it opens you up to pointless progress. How about, instead, you optimize _time to deploy_. i.e. the delta between code submitted for review and code running in production. Then, who cares if we didn't deploy 5 times today. Maybe we didn't have 5 things that needed updated.