4 ms·

So how do you measure performance or success of an engineering department within a bigger corp? Not to evaluate salaries or budgets, but to know, are we moving

So how do you measure performance or success of an engineering department within a bigger corp? Not to evaluate salaries or budgets, but to know, are we moving in the right direction? Are we improving or are we getting worse? From what I see most metrics on an individual or team level can/will be gamed leading to worse outcomes. Metrics on higher levels like on the level of the product or even revenue of the company/product depends on many other players outside of the control of the engineering team (product management, operations, QA, regulatory, marketing, sales, etc). This of course it's due to the fact that delivering value to customers is a team effort. But how can you know/show your department is on the right track?

save_ferris 6y ago
Personally, having worked for a few big companies in my career, I believe it's absolutely possible for a company to be too big. It might be an impious opinion, but there absolutely comes a time in a company's existence that internal politics begins to undermine the mission of the entire organization. Of course, I haven't worked everywhere and I recognize that this opinion is based purely on my own experience. But companies that generate billions upon billions in revenue seem to care less and less about functional process improvement and more about short-term profit gains. Because ultimately, the C suite is paid based on stock performance for most large companies, and that becomes the only metric that matters. In that situation, everything falls in line behind it.
solidasparagus 6y ago
Why do you want to measure the productivity of the engineers in isolation when their ability to deliver value depends on other people? Too often the answer to that questions boils down to 'ammunition for the blame game'. If your engineers can't deliver value because of say product, who cares how productive they are? Measure business outcomes - you will get what you measure. If engineering is being measured by the ability to deliver business outcomes but engineers can't because of other people, the software engineers are going to let you know* and it's an important thing to learn about and fix. *this assumes you have competent managers and engineers who are willing to communicate with those managers, but if you don't have that, this is sort of a moot point.
kqr 6y ago
In Accelerate the authors explain the construct they use in the State of DevOps reports. It consists of four metrics: - Deployment frequency - Lead time for changes - Failure rate of deployments - Time to restore service These collectively capture some sense of "quality and quantity" of changes. What they don't capture, obviously, are whether all changes are meaningful and valuable to the customer. However doing well on these metrics is nearly a prerequisite for doing a good job of bringing the voice of the customer into the process, so maybe trying to capture customer value created is step 2.

save_ferris 6y ago

Personally, having worked for a few big companies in my career, I believe it's absolutely possible for a company to be too big. It might be an impious opinion, but there absolutely comes a time in a company's existence that internal politics begins to undermine the mission of the entire organization. Of course, I haven't worked everywhere and I recognize that this opinion is based purely on my own experience. But companies that generate billions upon billions in revenue seem to care less and less about functional process improvement and more about short-term profit gains. Because ultimately, the C suite is paid based on stock performance for most large companies, and that becomes the only metric that matters. In that situation, everything falls in line behind it.

solidasparagus 6y ago

Why do you want to measure the productivity of the engineers in isolation when their ability to deliver value depends on other people? Too often the answer to that questions boils down to 'ammunition for the blame game'. If your engineers can't deliver value because of say product, who cares how productive they are? Measure business outcomes - you will get what you measure. If engineering is being measured by the ability to deliver business outcomes but engineers can't because of other people, the software engineers are going to let you know* and it's an important thing to learn about and fix. *this assumes you have competent managers and engineers who are willing to communicate with those managers, but if you don't have that, this is sort of a moot point.

kqr 6y ago

In Accelerate the authors explain the construct they use in the State of DevOps reports. It consists of four metrics: - Deployment frequency - Lead time for changes - Failure rate of deployments - Time to restore service These collectively capture some sense of "quality and quantity" of changes. What they don't capture, obviously, are whether all changes are meaningful and valuable to the customer. However doing well on these metrics is nearly a prerequisite for doing a good job of bringing the voice of the customer into the process, so maybe trying to capture customer value created is step 2.