5 ms·

One of the most well-known companies using what the author calls “impact frame” is Facebook. There, engineers’ promotions and perf used to be heavily dependent

One of the most well-known companies using what the author calls “impact frame” is Facebook. There, engineers’ promotions and perf used to be heavily dependent on their impact. Developers needed to be aware what the business impact of their work was, thus they needed to measure (using the excellent tooling available). This model worked very well while the company was growing. Lots of features core to Facebook today were created by engineers, not product managers. Newsfeed. GraphQL. A/B testing frameworks. The difficulty comes both when growth slows - and it’s hard to have impact without cannibalising another team’s impact - as well as when having more impact actually causes harm to the company on a different direction. While engineers were optimising for engagement, tweaking the news feed algorithm to show items you were likely to click on, sensationalist content (sometimes: fake news) was over-promoted. While the ads team optimised for ad engagement by exposing various metrics, they exposed metrics that turned to public uproar later on. The idea of only optimising for impact is a great short-term strategy. Long-term it is not nearly as black-and-white. My take is the optimum is in-between. Engineers should have an understanding and input on the business impact. But the strategy and prioritisation framework does need to come top-down, aligned with the long-term goals of the business.

jbay808 6y ago
Incentivizing your staff to optimize a measure is always dangerous. You'd better be darn sure that that measure is a very good approximation of your utility function.
CoolGuySteve 6y ago
I always think of the Business Dudes episode of Adventure Time: https://www.youtube.com/watch?v=_mxB_OlSsKI https://www.youtube.com/watch?v=_mxB_OlSsKI It's so blunt that it's great, the business dudes are so adept at optimization while ignoring externalities. "These poor souls are lost without jobs, we can't ignore their plight!"
davedx 6y ago
> My take is the optimum is in-between. Engineers should have an understanding and input on the business impact. But the strategy and prioritisation framework does need to come top-down, aligned with the long-term goals of the business. I used to work for a small-ish Dutch energy company, and they went to a great effort at implementing this kind of framework, I thought it was great. We basically had huge quarterly roadmaps pasted onto one big wall of the office, and you would frequently see the product, marketing, sales and C-level people stood there discussing and prioritising. The transparency here meant anyone could wander up and see what the strategy was for the quarter and therefore what would be prioritised. It was sometimes gloriously messy but you really felt included and incentivised to work towards the company's common goals. Wish everyone did this.
- andyjohnson0 6y ago
  Transparent, inclusive, low-ego. Seems like a very Dutch way to organise things. Like you, I also wish with was more common.
29athrowaway 6y ago
There is also the bus factor. If critical business functions depend on a piece of software maintained by one person, the organization has a problem. If you decide to become dependent on a piece of software, that software better be well documented, tested, audited and maintained by a group of people with certain level of redundancy in case one of them is not available or leaves the organization.
- someguydave 6y ago
  Preventing the destruction of the company in the case of low probability risks is its own impact.
  - 29athrowaway 6y ago
    Let's say the probability of someone leaving their job in a given day is 1/100. If you have 100 employees, from which each one of them is working on some critical piece of software that nobody fully understand, eventually you get a death spiral of orphaned projects.
zeku 6y ago
If you head to youtube and lookup a channel called something like: JomaTech you will find a guy who just recently quit Facebook. He was a Data Scientist Analyst as they call it there(according to him). It seemed he liked working at FB but he mentioned several times how your impact is your tracked metric and if you didn't have impact then who cares what you're doing. I think it can be flawed because if your team doesn't like you and youre a Data Scientist? Well now they aren't going to listen to your compelling data and your impact is a 0. Or if they have some other motive, it could even be one person with power on your team! Obviously I'm describing the worst, but it seems a flawed system.
tstrimple 6y ago
One thing I learned at my time at Microsoft is when a measurement becomes a metric, it ceases to be a good measurement. So much effort goes into gaming metrics which roll up to impact people's salaries that you can no longer really trust the results.
- kthejoker2 6y ago
  https://en.m.wikipedia.org/wiki/Goodhart%27s_law https://en.m.wikipedia.org/wiki/Goodhart%27s_law

jbay808 6y ago

Incentivizing your staff to optimize a measure is always dangerous. You'd better be darn sure that that measure is a very good approximation of your utility function.

CoolGuySteve 6y ago

I always think of the Business Dudes episode of Adventure Time: https://www.youtube.com/watch?v=_mxB_OlSsKI https://www.youtube.com/watch?v=_mxB_OlSsKI It's so blunt that it's great, the business dudes are so adept at optimization while ignoring externalities. "These poor souls are lost without jobs, we can't ignore their plight!"

davedx 6y ago

> My take is the optimum is in-between. Engineers should have an understanding and input on the business impact. But the strategy and prioritisation framework does need to come top-down, aligned with the long-term goals of the business. I used to work for a small-ish Dutch energy company, and they went to a great effort at implementing this kind of framework, I thought it was great. We basically had huge quarterly roadmaps pasted onto one big wall of the office, and you would frequently see the product, marketing, sales and C-level people stood there discussing and prioritising. The transparency here meant anyone could wander up and see what the strategy was for the quarter and therefore what would be prioritised. It was sometimes gloriously messy but you really felt included and incentivised to work towards the company's common goals. Wish everyone did this.

andyjohnson0 6y ago
Transparent, inclusive, low-ego. Seems like a very Dutch way to organise things. Like you, I also wish with was more common.

29athrowaway 6y ago

There is also the bus factor. If critical business functions depend on a piece of software maintained by one person, the organization has a problem. If you decide to become dependent on a piece of software, that software better be well documented, tested, audited and maintained by a group of people with certain level of redundancy in case one of them is not available or leaves the organization.

someguydave 6y ago
Preventing the destruction of the company in the case of low probability risks is its own impact.
- 29athrowaway 6y ago
  Let's say the probability of someone leaving their job in a given day is 1/100. If you have 100 employees, from which each one of them is working on some critical piece of software that nobody fully understand, eventually you get a death spiral of orphaned projects.

zeku 6y ago

If you head to youtube and lookup a channel called something like: JomaTech you will find a guy who just recently quit Facebook. He was a Data Scientist Analyst as they call it there(according to him). It seemed he liked working at FB but he mentioned several times how your impact is your tracked metric and if you didn't have impact then who cares what you're doing. I think it can be flawed because if your team doesn't like you and youre a Data Scientist? Well now they aren't going to listen to your compelling data and your impact is a 0. Or if they have some other motive, it could even be one person with power on your team! Obviously I'm describing the worst, but it seems a flawed system.

tstrimple 6y ago

One thing I learned at my time at Microsoft is when a measurement becomes a metric, it ceases to be a good measurement. So much effort goes into gaming metrics which roll up to impact people's salaries that you can no longer really trust the results.

kthejoker2 6y ago
https://en.m.wikipedia.org/wiki/Goodhart%27s_law https://en.m.wikipedia.org/wiki/Goodhart%27s_law