3 ms·

I was thinking about this recently. I work at a large company with untold millions in AWS spend. I'm 100% confident that I could shave off a few thousands (mayb

by paxys 3y ago

I was thinking about this recently. I work at a large company with untold millions in AWS spend. I'm 100% confident that I could shave off a few thousands (maybe even tens or hundreds of thousands) from the bill with a little bit of effort on my side. If I go up the management chain and ask if (1) I can make this an official project and put it on the roadmap or (2) I can do this on my own time and keep some % of the savings for myself as a reward, the answer would be a very clear "no" to both. So overall, as an end developer I really have no incentive to work harder and ensure lower operating costs for my company, and I'm sure most developers in the industry are in the same position as me.

jfim 3y ago
Part of it is that it creates an incentive to create wasteful systems, only to "optimize" them later to rack up a bonus. Even if it gets changed to only pay out for reducing spend incurred by other engineers, it's possible to collude in such a way to extract bonuses from the company. A better way to have aligned incentives for the company and the employees would be to allocate a bonus pool for the entire company, from which AWS expenses are taken out of, but that might be a bit unorthodox.
- avidiax 3y ago
  > allocate a bonus pool for the entire company, from which AWS expenses are taken out of Also a perverse incentive. If we use ec2.small, the customer's query will take 3x longer but be half the price. Let's turn off the nightly security audits. We can live with quarterly backups, right? What do we need all these logs for, anyway? We could hack something that works together in 2 weeks, but if we spend 3 months, it could be really efficient, let's do that...
cduzz 3y ago
How are they going to be sure you're not just farming cobras[1]? [1]https://en.wikipedia.org/wiki/Perverse_incentive https://en.wikipedia.org/wiki/Perverse_incentive
nick_ 3y ago
This is one of many insights that hint at why biz-facing cloud architecture is so popular, wasteful, and profitable. The incentives are designed to form an enormous cash siphon. From aggressively marketing toward fearful & liable (or maybe just tech-cost-illiterate) upper-management to the silencing effect that the low-rung experts experience when sounding the alarm.
PH95VuimJjqBqy 3y ago
some of us care enough about our craft to do it without the backpatting or the extra money.
devin 3y ago
Unfortunately it seems like businesses only wake up and asks for cost reduction on the infrastructure spend side once the problem is out of control. At that point, the level of operations effort to get it under control feels more like a "big rewrite" than a collection of small tweaks.
- killingtime74 3y ago
  It's opportunity cost. If it's not a problem then you can just make more money with your time.
  - PreachSoup 3y ago
    That's why in our company we have 2 type of engineering effort, core projects impact and improvement. Not all time is spent on the impact. A balancing act is needed
  - Draiken 3y ago
    Nah, that'd imply they think about this objectively. Most companies simply don't. Most time this "opportunity cost" is then spent on useless hacky features that are never used and forgotten right after release (redesign anyone?). Of course there are exceptions to all of these, but IME the majority of companies are either focusing on pennies or ignoring it completely. Not sure why you don't see balanced approaches more often. Maybe this will change with less VC money flying around.
mlsu 3y ago
Maybe there should be a sort of "anti-saas-sales" role: you get commission on whatever costs you're able to justify as superfluous. After all, the person at AWS makes commission selling you the stuff.
- jddj 3y ago
  Funny downvotes. Although not structured like this value engineering is common in other areas.
mullingitover 3y ago
If your company has a spending commitment with AWS in order to get a few percent savings, and it's just barely hitting the contracted amount, it may not be worth the effort to pursue any cost savings. Suppose your company has committed to hit 5M in spend, and they're just barely inching over the line at 5.01M. You might spend a bunch of time and labor expense knocking it down to 4M of usage and not really move the bottom line at all. > (2) I can do this on my own time and keep some % of the savings for myself as a reward This is a textbook case of perverse incentives.
- deepsun 3y ago
  Yep, that's how AWS sales make money. Theoretically, your company could reduce their commitment to 4M next year, but the AWS sales would start negotiating hard against it, like "you will not get the same discount with less commitment".
  - kthejoker2 3y ago
    Not only will you not get the discount you won't get the commit at all, AWS only reups commits with incremental growth. You can spend less; you just don't get commit pricing.
- mk89 3y ago
  I agree with this comment. Ironically, I was asked by a manager some time ago if we can imagine using some (more) resources from AWS to reach the next spending commitment. If you're just below the threshold, it's probably inconvenient. EDIT: To give some context: you can only do this, of course, when you know that you're not really wasting resources, otherwise you end up with just burning money to save little to no money :)
- caeril 3y ago
  Spending commitment on AWS? Are you referring to ec2 reserved instances? Even if you have one year reservations on your instances, starting service migrations/deprecations now would pay off quickly enough. Your commitment expires in 6 months, on an average basis.
killingtime74 3y ago
Not a commission but don't you have performance reviews? Promotion boards? Surely it can count for a bit, if the effort is as little as you say.
tayo42 3y ago
Companies make and spend so much money that this doesn't matter. Thousands, 10s of thousands, 100s is pointless in comparison the potential of building features. A developer costs about half million a year, (salary, bonus, rsu, taxes, benefits) If they are paying you to saves 10s to maybe hundreds the company is losing money on you so they won't do this. If your at a public company, look at your company's quarterly reports and see what it would take to many any kind of impact on net income.

by paxys 3y ago

jfim 3y ago

Part of it is that it creates an incentive to create wasteful systems, only to "optimize" them later to rack up a bonus. Even if it gets changed to only pay out for reducing spend incurred by other engineers, it's possible to collude in such a way to extract bonuses from the company. A better way to have aligned incentives for the company and the employees would be to allocate a bonus pool for the entire company, from which AWS expenses are taken out of, but that might be a bit unorthodox.

avidiax 3y ago
> allocate a bonus pool for the entire company, from which AWS expenses are taken out of Also a perverse incentive. If we use ec2.small, the customer's query will take 3x longer but be half the price. Let's turn off the nightly security audits. We can live with quarterly backups, right? What do we need all these logs for, anyway? We could hack something that works together in 2 weeks, but if we spend 3 months, it could be really efficient, let's do that...

cduzz 3y ago

How are they going to be sure you're not just farming cobras[1]? [1]https://en.wikipedia.org/wiki/Perverse_incentive https://en.wikipedia.org/wiki/Perverse_incentive

nick_ 3y ago

This is one of many insights that hint at why biz-facing cloud architecture is so popular, wasteful, and profitable. The incentives are designed to form an enormous cash siphon. From aggressively marketing toward fearful & liable (or maybe just tech-cost-illiterate) upper-management to the silencing effect that the low-rung experts experience when sounding the alarm.

PH95VuimJjqBqy 3y ago

some of us care enough about our craft to do it without the backpatting or the extra money.

devin 3y ago

Unfortunately it seems like businesses only wake up and asks for cost reduction on the infrastructure spend side once the problem is out of control. At that point, the level of operations effort to get it under control feels more like a "big rewrite" than a collection of small tweaks.

killingtime74 3y ago
It's opportunity cost. If it's not a problem then you can just make more money with your time.
- PreachSoup 3y ago
  That's why in our company we have 2 type of engineering effort, core projects impact and improvement. Not all time is spent on the impact. A balancing act is needed
- Draiken 3y ago
  Nah, that'd imply they think about this objectively. Most companies simply don't. Most time this "opportunity cost" is then spent on useless hacky features that are never used and forgotten right after release (redesign anyone?). Of course there are exceptions to all of these, but IME the majority of companies are either focusing on pennies or ignoring it completely. Not sure why you don't see balanced approaches more often. Maybe this will change with less VC money flying around.

mlsu 3y ago

Maybe there should be a sort of "anti-saas-sales" role: you get commission on whatever costs you're able to justify as superfluous. After all, the person at AWS makes commission selling you the stuff.

jddj 3y ago
Funny downvotes. Although not structured like this value engineering is common in other areas.

mullingitover 3y ago

If your company has a spending commitment with AWS in order to get a few percent savings, and it's just barely hitting the contracted amount, it may not be worth the effort to pursue any cost savings. Suppose your company has committed to hit 5M in spend, and they're just barely inching over the line at 5.01M. You might spend a bunch of time and labor expense knocking it down to 4M of usage and not really move the bottom line at all. > (2) I can do this on my own time and keep some % of the savings for myself as a reward This is a textbook case of perverse incentives.

deepsun 3y ago
Yep, that's how AWS sales make money. Theoretically, your company could reduce their commitment to 4M next year, but the AWS sales would start negotiating hard against it, like "you will not get the same discount with less commitment".
- kthejoker2 3y ago
  Not only will you not get the discount you won't get the commit at all, AWS only reups commits with incremental growth. You can spend less; you just don't get commit pricing.
mk89 3y ago
I agree with this comment. Ironically, I was asked by a manager some time ago if we can imagine using some (more) resources from AWS to reach the next spending commitment. If you're just below the threshold, it's probably inconvenient. EDIT: To give some context: you can only do this, of course, when you know that you're not really wasting resources, otherwise you end up with just burning money to save little to no money :)
caeril 3y ago
Spending commitment on AWS? Are you referring to ec2 reserved instances? Even if you have one year reservations on your instances, starting service migrations/deprecations now would pay off quickly enough. Your commitment expires in 6 months, on an average basis.

killingtime74 3y ago

Not a commission but don't you have performance reviews? Promotion boards? Surely it can count for a bit, if the effort is as little as you say.

tayo42 3y ago

Companies make and spend so much money that this doesn't matter. Thousands, 10s of thousands, 100s is pointless in comparison the potential of building features. A developer costs about half million a year, (salary, bonus, rsu, taxes, benefits) If they are paying you to saves 10s to maybe hundreds the company is losing money on you so they won't do this. If your at a public company, look at your company's quarterly reports and see what it would take to many any kind of impact on net income.