5 ms·

It'd be great if we could start measuring how much $ it costs to run our services vs how much developer salary $ it costs to maintain and build them. If Rust o

It'd be great if we could start measuring how much $ it costs to run our services vs how much developer salary $ it costs to maintain and build them. If Rust or something really saves money because it means we don't need 50 extra web workers for the same TPS then I don't think it's "premature optimization" - unless the dev salary is too much! As it stands we're all just saying stuff with no data or evidence to back it up.

Karrot_Kream 5y ago
It's not going to change any time soon, IMHO. Programmers have too big of a culture of individuality, blogging, and intellectual daydreaming to actually try to come to consensus on terms of art. On top of which businesses are loathe to share data because it's tied very closely to their competitive advantage. Programming culture is more about memetic and shared ideas and uses tradition to select winning strategies. Until this changes, it'll continue to be discordant voices talking over each other. Ideology [1] is a great talk about this. [1]: https://www.destroyallsoftware.com/talks/ideology https://www.destroyallsoftware.com/talks/ideology
- Jensson 5y ago
  > Programming culture is more about memetic and shared ideas and uses tradition to select winning strategies. Sounds exactly like business management culture. Makes sense though, developer productivity is first and foremost a business management issue so discussions about it ought to follow the same path as business management. And business management love their anecdotes, great person citations and jargon.
wongarsu 5y ago
With silicon valley salaries maybe the trade-off is more between Developer time vs. Devops time. Those 50 extra web workers might not matter much on their own, but someone has to set them up, orchestrate them and deal with all weird interactions that arise. And of course the further you get from silicon valley the more the hardware/service costs matter. A developer in Sofia easily earns an order of magnitude less than one in San Francisco, which shifts the whole optimization tradeoff a lot.
- Jensson 5y ago
  Silicon valley companies tend to have more users and therefore more servers, so saying they mostly don't need to care about server costs should be wrong. I worked at a small team at Google that had many thousands of servers, those servers cost way more to run than the engineers working on them. Compare that to a small company I worked at where one server with an extra as fallback was more than enough, there performance didn't matter at all even though salaries were much smaller.
mynameisash 5y ago
One of my anecdata: My team does a ton of large-scale processing (most of which is ETL), and we have TONS of servers at our disposal. We effectively throw hardware at our compute, and even then, jobs can take 5-10 hours to run. No big deal - just run the job before you call it a day and it's done when you start tomorrow morning. Or when you have daily jobs, you schedule it and it's just done whenever it's done. I can't tell you the number of times I've looked at a job someone wrote, made a 5 LOC change in 2 minutes, and cut processing from 8 hours to 5 hours. I literally had one that went from 5 hours to 5 minutes. These extremely simple investments save a lot of money, but it's a problem that's spread across hundreds of different jobs, and it takes so much effort to convince people that we should invest in it. A daily job that costs $1500/mo to run (based on amortized hardware costs) needs only be optimized once, and those savings are yours forever. But I've had the conversation that is effectively: my time is more costly than a measly $1500 job, so it's okay if the job is expensive.
- resonious 5y ago
  This is a pretty interesting example. It sounds like even with the numbers, it's a pretty hard sell to focus on performance..

Karrot_Kream 5y ago

It's not going to change any time soon, IMHO. Programmers have too big of a culture of individuality, blogging, and intellectual daydreaming to actually try to come to consensus on terms of art. On top of which businesses are loathe to share data because it's tied very closely to their competitive advantage. Programming culture is more about memetic and shared ideas and uses tradition to select winning strategies. Until this changes, it'll continue to be discordant voices talking over each other. Ideology [1] is a great talk about this. [1]: https://www.destroyallsoftware.com/talks/ideology https://www.destroyallsoftware.com/talks/ideology

Jensson 5y ago
> Programming culture is more about memetic and shared ideas and uses tradition to select winning strategies. Sounds exactly like business management culture. Makes sense though, developer productivity is first and foremost a business management issue so discussions about it ought to follow the same path as business management. And business management love their anecdotes, great person citations and jargon.

wongarsu 5y ago

With silicon valley salaries maybe the trade-off is more between Developer time vs. Devops time. Those 50 extra web workers might not matter much on their own, but someone has to set them up, orchestrate them and deal with all weird interactions that arise. And of course the further you get from silicon valley the more the hardware/service costs matter. A developer in Sofia easily earns an order of magnitude less than one in San Francisco, which shifts the whole optimization tradeoff a lot.

Jensson 5y ago
Silicon valley companies tend to have more users and therefore more servers, so saying they mostly don't need to care about server costs should be wrong. I worked at a small team at Google that had many thousands of servers, those servers cost way more to run than the engineers working on them. Compare that to a small company I worked at where one server with an extra as fallback was more than enough, there performance didn't matter at all even though salaries were much smaller.

mynameisash 5y ago

One of my anecdata: My team does a ton of large-scale processing (most of which is ETL), and we have TONS of servers at our disposal. We effectively throw hardware at our compute, and even then, jobs can take 5-10 hours to run. No big deal - just run the job before you call it a day and it's done when you start tomorrow morning. Or when you have daily jobs, you schedule it and it's just done whenever it's done. I can't tell you the number of times I've looked at a job someone wrote, made a 5 LOC change in 2 minutes, and cut processing from 8 hours to 5 hours. I literally had one that went from 5 hours to 5 minutes. These extremely simple investments save a lot of money, but it's a problem that's spread across hundreds of different jobs, and it takes so much effort to convince people that we should invest in it. A daily job that costs $1500/mo to run (based on amortized hardware costs) needs only be optimized once, and those savings are yours forever. But I've had the conversation that is effectively: my time is more costly than a measly $1500 job, so it's okay if the job is expensive.

resonious 5y ago
This is a pretty interesting example. It sounds like even with the numbers, it's a pretty hard sell to focus on performance..