8 ms·

Now having worked in a few smaller teams since my time at FB, I realize how marvellous it is to not have to worry about OKRs, performance reviews and all that m

Now having worked in a few smaller teams since my time at FB, I realize how marvellous it is to not have to worry about OKRs, performance reviews and all that malarkey. It was so draining to run yourself as a mini PR/brand within one's team (/one's department) and market oneself (one's team) to your manager (/your company). So much mental bandwidth was spent. I also, from a systems-thinking POV, just fundamentally reject the value in OKRs as a meaningful proxy for the value you (/your team) are producing unless you are an entirely mechanistic function. By mechanistic I mean you have a clear discrete Input<>Output expectation. Every single time an element of human activity is reduced to a metric, you lose something, and if you do it enough times, you've effectively produced a metric that is completely distinct from the thing you intended on measuring in the first place. For example: let's say you're in the suicide prevention team at FB, and your OKR is 'number of suicides averted'. Well, that sounds good, but unbeknownst to you, perverse incentives have kicked in and invisibly imbued your metric-chasing experiments a dark undertone. A new risk model might start flagging more content as potentially suicidal to boost numbers, leading to over-intervention that could traumatize users or waste resources. Or perhaps the surface we use to measure success, a UX or user activity metric, might actually be uncorrelated with crisis aversion or... and this is where it hits: the thing that's instrumental in crises is social media itself. The metric becomes a game, and it games _you_. Your team thinks you're doing X, but you're really doing Y. Because you're disattached from the real problem and comforted by incredibly lossy proxies.

friendzis 2y ago
I have semi-humorously dropped a comment defining Goodhart's law. The problem you are describing is nothing else but Goodhart's law in action: A measure stops being a good measure, i.e. be a proxy for something, once there are objectives attached to it. In other words, attaching goals to a metric invalidates previous causal relationship. That's neither bad, not good. It's a property of goal setting. The problematic part is still treating the measure as if it had causal relationship to something when that relationship has already been invalidated. In your example, number of suicides ceases to be comparable between pre and post OKR timeframes, however if you look closely, this particular goal is based on a metric the underlying OKR targets invalidate. Yes, sometimes you get these weird tautologies where you have to change the whole framework/process to make something both targetable and measurable simultaneously, potentially losing comparability to past data.
qznc 2y ago
As a team within a larger company, your purpose is to contribute to the larger goals. How do you know if you are doing that? As this "suicide prevention team", how do you know if you are doing a good job? I agree with you that proxy metrics easily distract you into doing Y instead of X. My opinion is that you need to iterate on your metrics then. Not having metrics means it all depends on the gut feeling of executives. It surely is draining to be clear about your goals. I fear we cannot really be politically correct and sufficiently honest even. What is the real goal of having a suicide prevention team? It might be token effort after some incident, then the actual goal would be to as cheap as possible while still maintaining the illusion. It might be to prevent future PR disasters, then collection helpful evidence for lawyers should be part of the job. This touches hard ethical questions and these should become evident when discussing the purpose of a team.
- paganel 2y ago
  > How do you know if you are doing that? You don't, as an individual "unit", which is part of the problem, i.e. modern management's focus in trying to split teams/big companies down to its "elementary" unit, the employee. > Not having metrics means it all depends on the gut feeling of executives. And that's why you need good executives, executives who have good guts. You cannot automate your way into being successful, at the end of it all running a company is still pretty much a social endeavour, one that cannot be partitioned down to individual units, neither can its success or failure be explained by those individual units alone.
  - 0xEF 2y ago
    It's exceptionally hard to find good executives due to nepotism and other problems with hiring them that result in the wrong people in charge of the wrong thing more frequently than not, though. So we need metrics., lest we get swept away in the Trickle-down Incompetence.
    - paganel 2y ago
      Yeah, I agree that it's a very difficult problem to solve and that getting the balance right between using metrics and having access to good "guts" is essential in a company's forward success, but that's the state of affairs we are in right now, for better or worse. I also think that the companies that matter getting bigger and bigger, with no actual failure on the horizon (such as bankruptcy) in case of strategically wrong decisions doesn't help things one bit, because in those cases management failure is in many cases rewarded as there's no immediate and adverse affect on the life of the company itself. We need some return to creative destruction, otherwise we'll be left re-arranging the chairs on the deck of the Titanic (I see this type of discussion on this particular subject as part of that metaphorical re-arrangement) until the proverbial iceberg will strike.
michaelt 2y ago
The times I've seen OKRs and metrics used effectively, they met two criteria. (1) it was always a metric closely aligned with what customers want and which drives revenue for the company; and (2) it didn't just impose a requirement to do work, it also provided political cover for not doing other, competing work. For example, if you're designing GPUs and/or GPU drivers? If your next-generation product has the aim of providing 25% more frames per second in "Baldur's Gate 3" and "Call of Duty 6" while maintaining the same quality - that would be a good objective for the team, as it's closely aligned with what your customers want. And if someone should come to you and tell you there's a lot of people streaming these days, and they think you should optimise gaming FPS and h264 compression at the same time? It's a sensible request but it's also a distraction; proper goal setting will let you say "great idea, but not this quarter". But there are a lot of fields of endeavour where it's not possible to meet these criteria - like the suicide prevention team from your example.
- ricardobeat 2y ago
  100% agree. There is a basic level of honesty required for this to work, which seems to have gone missing due to the "evolution" of corporate culture and the cash-rich environment which made actual productivity a secondary concern.
- DanielHB 2y ago
  In my job we have metrics, but they are not mandated to employees. There are no targets and stuff like that, they are just there if you want to know. Managers track it and try to optimize for it, but nobody bonuses or employment is on the line over them. Maybe some sales people have commission, but most of the customer acquisition is organic anyway. Funnily enough we have a weird situation right now, where we want to optimize the _cheaper_ plan of our product, meaning moving people from the more expensive plans to the cheaper one. It is a weird dynamic, but due to licensing our cheaper plan is actually far more profitable than the more expensive plans. Most clients wouldn't really miss anything from the more expensive plan. It is one of those rare situations where convincing people to switch to the cheaper plan aligns with what is best for the consumer.
  - 2y ago
jhghikvhu 2y ago
> It was so draining to run yourself as a mini PR/brand within one's team (/one's department) and market oneself (one's team) to your manager (/your company). Counter point. This is always inevitably a thing. They were only making the implicit explicit.
- aiizo 2y ago
  Yes but this being made into a formal process and having to regularly interact with that process is what's so draining. Not having to deal with this is one of the positives about working in a small startup versus a long-established large corporate.
  - deleted 2y ago
    [deleted]
- Lionga 2y ago
  In big teams you are right it almost is inevitable. It is not in small (say below 5 people) teams/ organisations. At least not in the ones I worked in.
- sunshowers 2y ago
  No, not to the extent that it was and is the case at FB. I think people who haven't worked at FB don't quite understand the degree to which PSC culture pervades the company. It's absolutely more intense than Google, Apple or Microsoft, though I'm not as sure about Amazon. Valve seems differently intense in a way that I think is a negative overall. A bit pithy, but: The goal of most enterprises is to build useful goods and services for its customers. The goal of Meta is to evaluate its employees.
robertlagrant 2y ago
> For example: let's say you're in the suicide prevention team at FB, and your OKR is 'number of suicides averted' That sounds like the sort of old-school KPIs that OKRs were meant to replace. I don't know if it's just impossible to measure anything, and you should just rely on a managers' word for how any team is doing, or if the people who did KPIs are now infecting OKRs.
- Izkata 2y ago
  They've always been closely related. The "KR" in OKR is "what is the new target for the KPI in order to reach O?"
theGnuMe 2y ago
The FB suicide risk detector OKR is an interesting example. It’d make a great business school / psychology case study. We measure something because we need something to measure even if it is divorced from reality.
QuadmasterXLII 2y ago
Safari’s idiotic AI has decided that every time I want to go to youtube, I actually want to go to a specific clip of peppa pig failing to learn how to whistle Google’s idiotic AI has decided that the clip of peppa pig failing to whistle depicts suicidal ideation, and so every time I visit the clip I get a big dramatic black rectangle and a therapyspeak question “Am I an adult who is prepared to view a depiction of suicide.” It took me embarassingly long to notice that this nessage, repeated constantly enough, was actually affecting my mood. Your scenario is generous in assuming that the suicide prevention FAANGers who coded up this situation are intelligently following bad incentives. I think its more likely that their intelligence is just found lacking, when unfairly compared to the galaxy brain needed to actually guess the consequences of our actions at this scale.
TheBigSalad 2y ago
At FB would the 'boots on the ground' devs be required to do OKRs? Or were they done at the team or manager level?
- fsociety 2y ago
  I did not do any OKRs on an infrastructure team. Qualitative results were just as valued, and we had the quantitative knobs to dial when needed thanks to excellent internal tooling and service maturity. I found it to be the ironic part of working at one of the most data-driven companies. We didn’t do OKRs in my org despite using data to drive decisions. I much prefer this to OKR hell.
- zeroonetwothree 2y ago
  It depends substantially based on which org you are in. But generally it is at the team level, so it would mostly be on the EM/TL/senior eng on the team.
loloquwowndueo 2y ago
> you've effectively produced a metric that is completely distinct from the thing you intended on measuring in the first place Basically Goodhart’s law https://en.m.wikipedia.org/wiki/Goodhart%27s_law https://en.m.wikipedia.org/wiki/Goodhart%27s_law
dfxm12 2y ago
You have invoked Goodhart's Law. The problem is, of course, that most managers are not good at their task of evaluating talent, proving the worth of their services, etc. and try to take the easiest way out of it. Sometimes this means outsourcing the job to you or picking a poor thing to measure.
roland35 2y ago
When I was at fb, I had the curse of being given an extremely vague goal as a new hire. Unfortunately, it was quite difficult to establish meaningful metrics AND hit them in a half! Like you said, I spent way more time on my self review than I wanted...
eweise 2y ago
I'm experiencing large company culture for the first time after being in small startups. What I object to the most, is the competition between engineers created from the goal setting and performance reviews. At startups, I had a domain that I could own. Now, at largco, everyone tries to take over my area in an effort to build their resumes. I used to view fellow engineers as teammates, now I see them as my competition.
- PaulHoule 2y ago
  Funny in a startup I value people being pinch hitters. Sure you should have your domain but if there is some exceptional event I want somebody to be able to cover for you or for you to do some job that wasn't even on the roster yesterday.
  - eweise 2y ago
    True. I guess what I mean is that other engineers try to take over the architecture and design. For instance, I had a fellow engineer without even talking with me, create a 2.0 vision doc for part of the system that I had designed and was mainly responsible for. This was clearly an attempt by them, to raise their status and influence.
    - dickersnoodle 2y ago
      I feel this pretty hard. You almost have to do your own 2.0 vision doc for things to make sure your name is on it in order to preempt the opportunists and influence pirates, but that can lead to you being seen as "hogging all the good work" and turning into the common adversary.
- nedrocks 2y ago
  I feel the same way. There’s a shift that happens at around 80 people where not everyone rows in the same direction. Incentives become different because not everyone “lives and dies” together or by the same metric. By the time you are at bigco status, this is so ingrained that work becomes repeated prisoner’s dilemma trials.
epolanski 2y ago
> Every single time an element of human activity is reduced to a metric, you lose something Also, metrics eventually become the goal and are gamified.
treetalker 2y ago
I think Laozi and Zhuangzi had something to say about this.
sunshowers 2y ago
One of the biggest culture shocks I had during my time at FB was when we were doing Mononoke, this completely greenfield project with a ton of unknowns (first big Rust project!), and we got a new skip level who was previously on the web performance team (super narrow and directed).
lumost 2y ago
The challenge is we don’t have an alternative, at a small company your performance boils down to “does the ceo want to fire you?”. The extent to which the ceo does not want to fire you depends on the reasonableness of that CEO, as well as how much the CEO cares. In an established small/medium business with flat growth, it’s entirely possible that no one cares to fire anyone, sits also possible the CEO expects everyone to work nights and weekends while being a top competitive coder.
theonething 2y ago
> I realize how marvellous it is to not have to worry about OKRs, performance reviews and all that malarkey Where are these malarkey-free companies? I've worked at several small companies and they all had all kinds of malarkey.

friendzis 2y ago

I have semi-humorously dropped a comment defining Goodhart's law. The problem you are describing is nothing else but Goodhart's law in action: A measure stops being a good measure, i.e. be a proxy for something, once there are objectives attached to it. In other words, attaching goals to a metric invalidates previous causal relationship. That's neither bad, not good. It's a property of goal setting. The problematic part is still treating the measure as if it had causal relationship to something when that relationship has already been invalidated. In your example, number of suicides ceases to be comparable between pre and post OKR timeframes, however if you look closely, this particular goal is based on a metric the underlying OKR targets invalidate. Yes, sometimes you get these weird tautologies where you have to change the whole framework/process to make something both targetable and measurable simultaneously, potentially losing comparability to past data.

qznc 2y ago

As a team within a larger company, your purpose is to contribute to the larger goals. How do you know if you are doing that? As this "suicide prevention team", how do you know if you are doing a good job? I agree with you that proxy metrics easily distract you into doing Y instead of X. My opinion is that you need to iterate on your metrics then. Not having metrics means it all depends on the gut feeling of executives. It surely is draining to be clear about your goals. I fear we cannot really be politically correct and sufficiently honest even. What is the real goal of having a suicide prevention team? It might be token effort after some incident, then the actual goal would be to as cheap as possible while still maintaining the illusion. It might be to prevent future PR disasters, then collection helpful evidence for lawyers should be part of the job. This touches hard ethical questions and these should become evident when discussing the purpose of a team.

paganel 2y ago
> How do you know if you are doing that? You don't, as an individual "unit", which is part of the problem, i.e. modern management's focus in trying to split teams/big companies down to its "elementary" unit, the employee. > Not having metrics means it all depends on the gut feeling of executives. And that's why you need good executives, executives who have good guts. You cannot automate your way into being successful, at the end of it all running a company is still pretty much a social endeavour, one that cannot be partitioned down to individual units, neither can its success or failure be explained by those individual units alone.
- 0xEF 2y ago
  It's exceptionally hard to find good executives due to nepotism and other problems with hiring them that result in the wrong people in charge of the wrong thing more frequently than not, though. So we need metrics., lest we get swept away in the Trickle-down Incompetence.
  - paganel 2y ago
    Yeah, I agree that it's a very difficult problem to solve and that getting the balance right between using metrics and having access to good "guts" is essential in a company's forward success, but that's the state of affairs we are in right now, for better or worse. I also think that the companies that matter getting bigger and bigger, with no actual failure on the horizon (such as bankruptcy) in case of strategically wrong decisions doesn't help things one bit, because in those cases management failure is in many cases rewarded as there's no immediate and adverse affect on the life of the company itself. We need some return to creative destruction, otherwise we'll be left re-arranging the chairs on the deck of the Titanic (I see this type of discussion on this particular subject as part of that metaphorical re-arrangement) until the proverbial iceberg will strike.

michaelt 2y ago

The times I've seen OKRs and metrics used effectively, they met two criteria. (1) it was always a metric closely aligned with what customers want and which drives revenue for the company; and (2) it didn't just impose a requirement to do work, it also provided political cover for not doing other, competing work. For example, if you're designing GPUs and/or GPU drivers? If your next-generation product has the aim of providing 25% more frames per second in "Baldur's Gate 3" and "Call of Duty 6" while maintaining the same quality - that would be a good objective for the team, as it's closely aligned with what your customers want. And if someone should come to you and tell you there's a lot of people streaming these days, and they think you should optimise gaming FPS and h264 compression at the same time? It's a sensible request but it's also a distraction; proper goal setting will let you say "great idea, but not this quarter". But there are a lot of fields of endeavour where it's not possible to meet these criteria - like the suicide prevention team from your example.

ricardobeat 2y ago
100% agree. There is a basic level of honesty required for this to work, which seems to have gone missing due to the "evolution" of corporate culture and the cash-rich environment which made actual productivity a secondary concern.
DanielHB 2y ago
In my job we have metrics, but they are not mandated to employees. There are no targets and stuff like that, they are just there if you want to know. Managers track it and try to optimize for it, but nobody bonuses or employment is on the line over them. Maybe some sales people have commission, but most of the customer acquisition is organic anyway. Funnily enough we have a weird situation right now, where we want to optimize the _cheaper_ plan of our product, meaning moving people from the more expensive plans to the cheaper one. It is a weird dynamic, but due to licensing our cheaper plan is actually far more profitable than the more expensive plans. Most clients wouldn't really miss anything from the more expensive plan. It is one of those rare situations where convincing people to switch to the cheaper plan aligns with what is best for the consumer.
- 2y ago

jhghikvhu 2y ago

> It was so draining to run yourself as a mini PR/brand within one's team (/one's department) and market oneself (one's team) to your manager (/your company). Counter point. This is always inevitably a thing. They were only making the implicit explicit.

aiizo 2y ago
Yes but this being made into a formal process and having to regularly interact with that process is what's so draining. Not having to deal with this is one of the positives about working in a small startup versus a long-established large corporate.
- deleted 2y ago
  [deleted]
Lionga 2y ago
In big teams you are right it almost is inevitable. It is not in small (say below 5 people) teams/ organisations. At least not in the ones I worked in.
sunshowers 2y ago
No, not to the extent that it was and is the case at FB. I think people who haven't worked at FB don't quite understand the degree to which PSC culture pervades the company. It's absolutely more intense than Google, Apple or Microsoft, though I'm not as sure about Amazon. Valve seems differently intense in a way that I think is a negative overall. A bit pithy, but: The goal of most enterprises is to build useful goods and services for its customers. The goal of Meta is to evaluate its employees.

robertlagrant 2y ago

> For example: let's say you're in the suicide prevention team at FB, and your OKR is 'number of suicides averted' That sounds like the sort of old-school KPIs that OKRs were meant to replace. I don't know if it's just impossible to measure anything, and you should just rely on a managers' word for how any team is doing, or if the people who did KPIs are now infecting OKRs.

Izkata 2y ago
They've always been closely related. The "KR" in OKR is "what is the new target for the KPI in order to reach O?"

theGnuMe 2y ago

The FB suicide risk detector OKR is an interesting example. It’d make a great business school / psychology case study. We measure something because we need something to measure even if it is divorced from reality.

QuadmasterXLII 2y ago

Safari’s idiotic AI has decided that every time I want to go to youtube, I actually want to go to a specific clip of peppa pig failing to learn how to whistle Google’s idiotic AI has decided that the clip of peppa pig failing to whistle depicts suicidal ideation, and so every time I visit the clip I get a big dramatic black rectangle and a therapyspeak question “Am I an adult who is prepared to view a depiction of suicide.” It took me embarassingly long to notice that this nessage, repeated constantly enough, was actually affecting my mood. Your scenario is generous in assuming that the suicide prevention FAANGers who coded up this situation are intelligently following bad incentives. I think its more likely that their intelligence is just found lacking, when unfairly compared to the galaxy brain needed to actually guess the consequences of our actions at this scale.

TheBigSalad 2y ago

At FB would the 'boots on the ground' devs be required to do OKRs? Or were they done at the team or manager level?

fsociety 2y ago
I did not do any OKRs on an infrastructure team. Qualitative results were just as valued, and we had the quantitative knobs to dial when needed thanks to excellent internal tooling and service maturity. I found it to be the ironic part of working at one of the most data-driven companies. We didn’t do OKRs in my org despite using data to drive decisions. I much prefer this to OKR hell.
zeroonetwothree 2y ago
It depends substantially based on which org you are in. But generally it is at the team level, so it would mostly be on the EM/TL/senior eng on the team.

loloquwowndueo 2y ago

> you've effectively produced a metric that is completely distinct from the thing you intended on measuring in the first place Basically Goodhart’s law https://en.m.wikipedia.org/wiki/Goodhart%27s_law https://en.m.wikipedia.org/wiki/Goodhart%27s_law

dfxm12 2y ago

You have invoked Goodhart's Law. The problem is, of course, that most managers are not good at their task of evaluating talent, proving the worth of their services, etc. and try to take the easiest way out of it. Sometimes this means outsourcing the job to you or picking a poor thing to measure.

roland35 2y ago

When I was at fb, I had the curse of being given an extremely vague goal as a new hire. Unfortunately, it was quite difficult to establish meaningful metrics AND hit them in a half! Like you said, I spent way more time on my self review than I wanted...

eweise 2y ago

I'm experiencing large company culture for the first time after being in small startups. What I object to the most, is the competition between engineers created from the goal setting and performance reviews. At startups, I had a domain that I could own. Now, at largco, everyone tries to take over my area in an effort to build their resumes. I used to view fellow engineers as teammates, now I see them as my competition.

PaulHoule 2y ago
Funny in a startup I value people being pinch hitters. Sure you should have your domain but if there is some exceptional event I want somebody to be able to cover for you or for you to do some job that wasn't even on the roster yesterday.
- eweise 2y ago
  True. I guess what I mean is that other engineers try to take over the architecture and design. For instance, I had a fellow engineer without even talking with me, create a 2.0 vision doc for part of the system that I had designed and was mainly responsible for. This was clearly an attempt by them, to raise their status and influence.
  - dickersnoodle 2y ago
    I feel this pretty hard. You almost have to do your own 2.0 vision doc for things to make sure your name is on it in order to preempt the opportunists and influence pirates, but that can lead to you being seen as "hogging all the good work" and turning into the common adversary.
nedrocks 2y ago
I feel the same way. There’s a shift that happens at around 80 people where not everyone rows in the same direction. Incentives become different because not everyone “lives and dies” together or by the same metric. By the time you are at bigco status, this is so ingrained that work becomes repeated prisoner’s dilemma trials.

epolanski 2y ago

> Every single time an element of human activity is reduced to a metric, you lose something Also, metrics eventually become the goal and are gamified.

treetalker 2y ago

I think Laozi and Zhuangzi had something to say about this.

sunshowers 2y ago

One of the biggest culture shocks I had during my time at FB was when we were doing Mononoke, this completely greenfield project with a ton of unknowns (first big Rust project!), and we got a new skip level who was previously on the web performance team (super narrow and directed).

lumost 2y ago

The challenge is we don’t have an alternative, at a small company your performance boils down to “does the ceo want to fire you?”. The extent to which the ceo does not want to fire you depends on the reasonableness of that CEO, as well as how much the CEO cares. In an established small/medium business with flat growth, it’s entirely possible that no one cares to fire anyone, sits also possible the CEO expects everyone to work nights and weekends while being a top competitive coder.

theonething 2y ago

> I realize how marvellous it is to not have to worry about OKRs, performance reviews and all that malarkey Where are these malarkey-free companies? I've worked at several small companies and they all had all kinds of malarkey.