5 ms·

Many senior engineers who operate systems which spend 1M/month make 500k yearly 1M * 12 * 10% = 1.2M 500k * 12 * 2 * taxes >= 1.2M How many 10% wins do they

Many senior engineers who operate systems which spend 1M/month make 500k yearly 1M * 12 * 10% = 1.2M 500k * 12 * 2 * taxes >= 1.2M How many 10% wins do they have to achieve in what timeframe to have a good ROI on micro-optimization, considering that these seniors could be producing stuff that allows for multiple times their salary in revenue?

charleslmunger 5y ago
Improvements are one-time investments of engineering time, but ongoing savings in infra costs. Consider also that these savings also permit scaling to more customers or more work. Suppose you have two engineers that total to $1M of personnel cost. If they deliver a cost savings of 1% per month (compounding) the savings-per-month will have surpassed their pay per month; you will see your infra costs at 85% of what they were after 16 months, and on month 17 you will have made a net profit on those engineers, where the savings they got you has totalled more than what you've paid them. If your business is growing, you could project your infra costs to grow beyond $1M per month, and the numbers become even more favorable.
- abriosi 5y ago
  Unless a company is running an engineering team composed of monkeys, it will be hard to achieve 1% optimization per month, even without compounding. There are a lot of optimizations which look good on paper but in practice bring other unforeseen cons Anyway, a profitable software company will be looking at getting a big multiple of 10x the salary of their engineers
  - saagarjha 5y ago
    I work in perform engineering, although in a client rather than backend role. We usually do better than 1% a month. In fact we do this even as >1% of regressions make it into the build from the software development process.
  - charleslmunger 5y ago
    It depends what you're optimizing. Mature c++ codebase that is routinely profiled? Large legacy codebase with few tests? Mixed workloads? All hard. But there's also a lot of easy cases out there - even good engineers might code like monkeys if they're under time pressure and trying to grow the business. I've done a lot of performance work, and while of course no one individual could achieve a 1% savings per month on the entirety of FAANG compute loads, there is lots of code out there that nobody thought would run at scale or be on the critical path of something important, but now it is and it has gotten expensive.
CSMastermind 5y ago
As a rule of thumb, I devote about 0.5% of my engineering org to cost optimization. At about 200 people it makes sense to invest a single engineer. By the team you're approaching 800 it makes sense to have a team dedicates to it. Though cost opt generally involves way more than optimizing code. It's normally about auditing things, creating budget tooling, right sizing, broader architecture redesigns, etc.
- klodolph 5y ago
  I don't see how a rule of thumb could make sense, since the cost of operations can vary so much from company to company.
bilkow 5y ago
You shouldn't multiply by 12 if the 500k is yearly. Also what's the 2? 2 engineers? The correct math would be: 1M * 12 * 10% = 1.2M 500k * 2 * taxes >= 1M
- abriosi 5y ago
  Someone said "a couple engineers" hence 2. The 12 was a bad copy paste, thanks for pointing that out

charleslmunger 5y ago

Improvements are one-time investments of engineering time, but ongoing savings in infra costs. Consider also that these savings also permit scaling to more customers or more work. Suppose you have two engineers that total to $1M of personnel cost. If they deliver a cost savings of 1% per month (compounding) the savings-per-month will have surpassed their pay per month; you will see your infra costs at 85% of what they were after 16 months, and on month 17 you will have made a net profit on those engineers, where the savings they got you has totalled more than what you've paid them. If your business is growing, you could project your infra costs to grow beyond $1M per month, and the numbers become even more favorable.

abriosi 5y ago
Unless a company is running an engineering team composed of monkeys, it will be hard to achieve 1% optimization per month, even without compounding. There are a lot of optimizations which look good on paper but in practice bring other unforeseen cons Anyway, a profitable software company will be looking at getting a big multiple of 10x the salary of their engineers
- saagarjha 5y ago
  I work in perform engineering, although in a client rather than backend role. We usually do better than 1% a month. In fact we do this even as >1% of regressions make it into the build from the software development process.
- charleslmunger 5y ago
  It depends what you're optimizing. Mature c++ codebase that is routinely profiled? Large legacy codebase with few tests? Mixed workloads? All hard. But there's also a lot of easy cases out there - even good engineers might code like monkeys if they're under time pressure and trying to grow the business. I've done a lot of performance work, and while of course no one individual could achieve a 1% savings per month on the entirety of FAANG compute loads, there is lots of code out there that nobody thought would run at scale or be on the critical path of something important, but now it is and it has gotten expensive.

CSMastermind 5y ago

As a rule of thumb, I devote about 0.5% of my engineering org to cost optimization. At about 200 people it makes sense to invest a single engineer. By the team you're approaching 800 it makes sense to have a team dedicates to it. Though cost opt generally involves way more than optimizing code. It's normally about auditing things, creating budget tooling, right sizing, broader architecture redesigns, etc.

klodolph 5y ago
I don't see how a rule of thumb could make sense, since the cost of operations can vary so much from company to company.

bilkow 5y ago

You shouldn't multiply by 12 if the 500k is yearly. Also what's the 2? 2 engineers? The correct math would be: 1M * 12 * 10% = 1.2M 500k * 2 * taxes >= 1M

abriosi 5y ago
Someone said "a couple engineers" hence 2. The 12 was a bad copy paste, thanks for pointing that out