5 ms·

It's possible to have teams that save millions of dollars a year and not be worth it to keep them. For the sake of argument, let's say a statistics team has 5

by apohn 4y ago

It's possible to have teams that save millions of dollars a year and not be worth it to keep them. For the sake of argument, let's say a statistics team has 5 people. Cost of Employee at FB, including insurance, office space, 401K match, salary, bonuses = 250K/year (probably very conservative). Cost of Data and Software Infrastructure to support them (including people to respond to Infrastructure support tickets), let's just be very conservative = 100K/year. Cost of People Management overhead to support them. Includes salary of at least one manager, not to mention the time of a program manager, project manager, product manager, or whomever else. Let's just say 500K/year. Total = 1.85 Million/Year. Let's say this team of 5 people comes up models that save the company $4M a year. I once had a VP tell me that to justify a Data Scientist on the team, they needed to have a savings of 10X what they cost the company to have that person on staff. I know this logic and math seems very weak and hazy. Mapping costs is a strange thing. But this is how some decision makers think, and this is how people get cut.

drc500free 4y ago
These employees are making a lot more than $250k just in base salary. Cost is probably closer to $1M each, all in. "A few million" in net cost savings isn't much for a team that probably costs $5M a year. It would definitely be better to find another internal home (assuming the team is portable without its mother team that got cut), but sometimes these decisions are made quickly without a lot of granularity. They aren't necessarily going to find one sub-team that saves only ~1x their cost in net profit and figure out how to transplant them to another org. He seems to have taken away the important lesson - if you're not primary you're in danger.
- vikingerik 4y ago
  How in the world are you getting from $250k salary to $1M total cost? Stuff like office space and equipment/services, health insurance, HR overhead are constants per person, they don't scale up with salary. Are you assuming that some big bonus or grant package is necessary?
  - drc500free 4y ago
    Yes, their total comp is $500k+. They are taking up a portion of the management time of someone whose total comp is approaching (or over) $1M. Software Engineer: https://www.levels.fyi/companies/facebook/salaries/software-engineer https://www.levels.fyi/companies/facebook/salaries/software-... Software Engineering Manager: https://www.levels.fyi/companies/facebook/salaries/software-engineering-manager https://www.levels.fyi/companies/facebook/salaries/software-...
  - johnfn 4y ago
    Facebook employees make a lot more than 250k. Someone with Eric lippert’s level of experience probably makes well over 600-700k in total compensation - just see levels.fyi!
    - athrowaway12 4y ago
      A big portion of the comp especially at higher levels is in stock grants... and Meta stock just dropped 75% in value this year. These grants are valued at the market price at time of hire (or refresh). So maybe pre-2022 the comp was 700k...
  - TheNewsIsHere 4y ago
    The trick with discussing any numbers like this are variables that none of us can know without more intimate knowledge of a firm. For example, my spouse works for an SV firm. His team is 100% WFH, 100% of the time. They have no permanently allocated office space in any of the company’s buildings anywhere in the world. However, they’re paying out bonuses twice a year, annual (PB)RSUs, (specifically for us) around almost $30k/yr in employer contributions to health insurance and our HSA combined, music streaming subscription, and so on. The benefits, the bonuses, the extras, they all add up and are all very company specific. I’m not saying you’re wrong by any stretch. But with the number of extra benefits, healthcare, and everything else that’s different from employer to employer, we are all just guessing.
- trenchgun 4y ago
  It was claimed to be millions of net savings per deployed model.
ericlippert 4y ago
The team was all mathematicians. We did the math. I helped one of our data scientists put a model into production that saved $15M a year from that model alone, and we had a dozen people like that. We were working on signal loss models that had potential to save billions. I genuinely do not understand the logic of cutting this team to save costs.
- hkon 4y ago
  > We were working on signal loss models that had potential to save billions What are signal loss models in this context?
  - ericlippert 4y ago
    "Signal loss" is the overarching term for all the factors that lead to the company being less able to make good inferences about users. Not just the obvious consideration of "how do we serve an ad that is relevant to the user?" but for any data-driven decision that affects a user's experience. The biggest recent cause of signal loss was Apple changing the rules for apps on their phone, but there are plenty of other causes. The idea of a signal loss model is to identify ways to work around signal loss and still do a good job of making a decision with the data you have, when some of the data you were relying upon disappears suddenly.
    - bombcar 4y ago
      Perhaps an example would be - we no longer have location data for users, but we do have time of activity, so we can presume that during daylight hours in the USA most of our activity is coming from there, things like that. But with more inputs and such.
- drc500free 4y ago
  Unfortunately, top-down mandates are imperfect and should be avoided as much as possible. Net profit matters to an operator who cares about today's profitability, but not at all to someone whose paradigm is "thinking in bets" and future payoffs. And the street has been rewarding people who ignore today's profits in favor of the narrative about tomorrow's growth. From afar, it looks like Meta's leadership is a bunch of future thinkers who got told to cut today's costs, and it's not a well-practiced muscle for them.
AlwaysRock 4y ago
Doesnt this paragraph indicate that they were making millions of dollars in savings OVER their cost of operating? "The PPL team in particular was at the point where we were regularly putting models into production that on net reduced costs by millions of dollars a year over the cost of the work"
kqr 4y ago
If the company as a whole has an ROI of 10×, then a position with a 4× ROI is actually reducing the marginal profit of the company.

by apohn 4y ago

drc500free 4y ago

These employees are making a lot more than $250k just in base salary. Cost is probably closer to $1M each, all in. "A few million" in net cost savings isn't much for a team that probably costs $5M a year. It would definitely be better to find another internal home (assuming the team is portable without its mother team that got cut), but sometimes these decisions are made quickly without a lot of granularity. They aren't necessarily going to find one sub-team that saves only ~1x their cost in net profit and figure out how to transplant them to another org. He seems to have taken away the important lesson - if you're not primary you're in danger.

vikingerik 4y ago
How in the world are you getting from $250k salary to $1M total cost? Stuff like office space and equipment/services, health insurance, HR overhead are constants per person, they don't scale up with salary. Are you assuming that some big bonus or grant package is necessary?
- drc500free 4y ago
  Yes, their total comp is $500k+. They are taking up a portion of the management time of someone whose total comp is approaching (or over) $1M. Software Engineer: https://www.levels.fyi/companies/facebook/salaries/software-engineer https://www.levels.fyi/companies/facebook/salaries/software-... Software Engineering Manager: https://www.levels.fyi/companies/facebook/salaries/software-engineering-manager https://www.levels.fyi/companies/facebook/salaries/software-...
- johnfn 4y ago
  Facebook employees make a lot more than 250k. Someone with Eric lippert’s level of experience probably makes well over 600-700k in total compensation - just see levels.fyi!
  - athrowaway12 4y ago
    A big portion of the comp especially at higher levels is in stock grants... and Meta stock just dropped 75% in value this year. These grants are valued at the market price at time of hire (or refresh). So maybe pre-2022 the comp was 700k...
- TheNewsIsHere 4y ago
  The trick with discussing any numbers like this are variables that none of us can know without more intimate knowledge of a firm. For example, my spouse works for an SV firm. His team is 100% WFH, 100% of the time. They have no permanently allocated office space in any of the company’s buildings anywhere in the world. However, they’re paying out bonuses twice a year, annual (PB)RSUs, (specifically for us) around almost $30k/yr in employer contributions to health insurance and our HSA combined, music streaming subscription, and so on. The benefits, the bonuses, the extras, they all add up and are all very company specific. I’m not saying you’re wrong by any stretch. But with the number of extra benefits, healthcare, and everything else that’s different from employer to employer, we are all just guessing.
trenchgun 4y ago
It was claimed to be millions of net savings per deployed model.

ericlippert 4y ago

The team was all mathematicians. We did the math. I helped one of our data scientists put a model into production that saved $15M a year from that model alone, and we had a dozen people like that. We were working on signal loss models that had potential to save billions. I genuinely do not understand the logic of cutting this team to save costs.

hkon 4y ago
> We were working on signal loss models that had potential to save billions What are signal loss models in this context?
- ericlippert 4y ago
  "Signal loss" is the overarching term for all the factors that lead to the company being less able to make good inferences about users. Not just the obvious consideration of "how do we serve an ad that is relevant to the user?" but for any data-driven decision that affects a user's experience. The biggest recent cause of signal loss was Apple changing the rules for apps on their phone, but there are plenty of other causes. The idea of a signal loss model is to identify ways to work around signal loss and still do a good job of making a decision with the data you have, when some of the data you were relying upon disappears suddenly.
  - bombcar 4y ago
    Perhaps an example would be - we no longer have location data for users, but we do have time of activity, so we can presume that during daylight hours in the USA most of our activity is coming from there, things like that. But with more inputs and such.
drc500free 4y ago
Unfortunately, top-down mandates are imperfect and should be avoided as much as possible. Net profit matters to an operator who cares about today's profitability, but not at all to someone whose paradigm is "thinking in bets" and future payoffs. And the street has been rewarding people who ignore today's profits in favor of the narrative about tomorrow's growth. From afar, it looks like Meta's leadership is a bunch of future thinkers who got told to cut today's costs, and it's not a well-practiced muscle for them.

AlwaysRock 4y ago

Doesnt this paragraph indicate that they were making millions of dollars in savings OVER their cost of operating? "The PPL team in particular was at the point where we were regularly putting models into production that on net reduced costs by millions of dollars a year over the cost of the work"

kqr 4y ago

If the company as a whole has an ROI of 10×, then a position with a 4× ROI is actually reducing the marginal profit of the company.