5 ms·

I'm very surprised they don't even mention the efficiency of the developer in this context - C might be the greener language once the system is developed, but i

by rcbdev 3y ago

I'm very surprised they don't even mention the efficiency of the developer in this context - C might be the greener language once the system is developed, but it's somewhat common sense that a higher-level language should significantly lessen development time. Since developers needs to run an entire OS incl. employer/administrative overhead for the time they're developing, the greenest thing one could do for software is improving DevEx. (at all levels - language, infrastructure, tooling) I'm talking about the majority of software being developed here - if you're FAANG or run a system with huge loads then the trade-off might be worth it.

morelisp 3y ago
Development time makes up a nearly immeasurable fraction of power resources outside of the tiniest teams.
- rcbdev 3y ago
  Imagine you have a team migrating the federal tax system of a smaller country - you can choose to develop the project in C and have X amount of devs work on it for 3-4 years 40-50h a week. The resulting code base will be huge and probably hard to maintain. Or you could choose to hire devs in a higher level language, maybe X/2 or X/1.5 amount of devs - saving you the environmental cost of running possibly hundreds of workstations every single day for years at a time and doing administration for more employees. The resulting code base will also be smaller and (hopefully) easier to maintain. I don't think you could call this immeasurable at all. (This is based on a real example)
  - vidarh 3y ago
    Not just the workstations. You really need to factor in the other carbon output of those developers for the time spent on the project too, not just the tools. And you then need to consider to what extent any performance saving will be efficiently captured as reduced power use. But, yes, fully agree. One system I worked on recently involved about 20 developer years of effort, and about 40 core years of computation... I'm pretty sure the developers machines combined spent far more energy than the production systems, for a Ruby deployment, before factoring in any other energy use relating to difference in effort.
  - flohofwoe 3y ago
    IME development resources will always expand to the available budget and time anyway, the programming language or programmer skills really don't matter much. Also, whether C is actually more or less "productive" than other languages for specific tasks isn't all that clear either. For the things I pick C for (for instance cross-platform libraries sitting between OS APIs and user code, and home computer emulators), it is also the most productive option (in the sense that higher level programming languages wouldn't make me more productive, because all that's needed for this type of stuff is functions, structs, loops, conditionals and a handful of math operators - and all those things are in C). High level features like automatic memory management don't make much sense when there's hardly any heap memory to be managed, and a rich stdlib also isn't needed when all you do is number crunching and bit twiddling.
vidarh 3y ago
And you need to keep this in account when considering efficiencies in general. My first production Ruby project increased our CPU usage in userspace tenfold for the service where it replaced a C version. At a point where it processed millions of messages a day, that added up to 10% of one core. The carbon cost of the development effort almost certainly outweighed the cost of running the service - either version - by a large factor, and hence any savings on dev effort might easily (might, as I didn't keep precise records) have outweighed production costs by a significant factor too. The Ruby version was less than 1/10th the size despite more features. So I agree with you. Many systems have plenty of services like that. Some systems are very clearly not like that and are worth optimizing, as you say.
- benj111 3y ago
  Is looking at current running costs the only metric? Computer co2 emissions are dominated by manufacture, not use. At some point the extra bloat means you have to upgrade your computer / buy another one.
  - vidarh 3y ago
    The point is that for a lot of services that "at some point" never come, and/or favors avoiding "extra bloat" in the development team rather than production servers. By all means, assess the costs and benefits - I've done projects where I've optimized heavily for performance because it means we could shave tiny amounts of power use off embedded systems we hoped to deploy in large quantities; I'd certainly not deploy Ruby there. But people seem to have a tendency to overestimate the scale they will need, often dramatically, and often because they have no idea the size of their total addressable market or the amount of data available in a given space (e.g. I've seen developers worry about scaling things to large clusters in niches where actually talking it through with a business analyst would've made them realize that a total saturation of the product to 100% of the maximally possible worldwide market for their service would see it all fit on their laptop) Unless you actually sit down and work out what your realistic scaling scenarios look like, you don't know whether a tradeoff of time vs. performance will ever pay off. Even when you do need a scale where putting significant effort into performance matters, there are many scenarios where starting "inefficient" is still more efficient, because it lets you focus more effort on the parts of the system that are proven in real-world settings to actually carry a cost that matters, and the most cost-effective solutions might well be different than what you expect. E.g. one other aspect of the system that included the Ruby component I mentioned actually hit scaling challenges. Instead of investing time in rewriting it, we cut the computational cost to fractions of a percent by just caching the main pages for a few seconds (I want to say 10, but not sure the exact number). The total energy use for generating those pages once that cache was in place (reconfiguring the reverse proxy that was already in place) was so low that rewriting that part in a more efficient language would equally have saved at most fractions of a core. Instead of spending extra dev time on that, we got to spend the extra dev resources where it did matter, such as e.g. the backend search engine indexing and query engine, where processing costs certainly would have killed us if it'd been written in a slow language. Basically, know your costs. Your fully loaded, real costs, including the actual dev costs by service and by feature. Developers in particular tend to have absolutely no clue about the fully loaded costs of the choices they make, and it often shows in the choices they make. EDIT: I'd like to add that this is often not the fault of developers. It took years, and very specific circumstances (co-founding startups, that exposed me much more to the money side) before I started seriously thinking about the costs of what I built, and sitting down and actually calculating "worst case" scale and likely scale, and estimating costs of going from one to the other if we engineered for the smaller scale. It's not a habit most developers have, because it's not something most developers get asked, even though they ought to at the very least be involved (a lot of the time I've seen costing done by architects, or even worse directors or VPs with no exposure to the hands on work, it bears little resemblance to reality).
flohofwoe 3y ago
> I'm very surprised they don't even mention the efficiency of the developer in this context Because in terms of energy use it doesn't matter. Take any somewhat successful video game for example. It might take years and hundreds of developers to create, but all the energy used during that period is most likely just a fraction of running the game hundreds of thousands of times on the first day after release.
- rcbdev 3y ago
  This is a very specific example since video games genuinely require loads of resources to run at the end user side. Most software projects aren't client-side behemoths like that.
  - flohofwoe 3y ago
    Ok, then web browsers :) Run by many more people than video games, but (not quite as) heavy when it comes to resource usage.
    - rcbdev 3y ago
      How many people do you know who work on browsers though?
      - flohofwoe 3y ago
        A handful, but there are definitely more people who *use* browsers each day than there are people developing browsers. That's the whole point of the argument ;)
        rcbdev 3y ago
        As I said, if you are FAANG or similar - which browsers are - the trade-off might be worth it, yes. But only a handful of software projects out there are browsers or video games. That's the whole point of the argument. ;)
offices 3y ago
The development time is almost inconsequential. Software is written once-ish, read a few times, and run many, many times.
- Gud 3y ago
  You also have to factor in how urgently you need the software.
PH95VuimJjqBqy 3y ago
I don't agree with this, developers will just develop something else with the extra time, their footprint is a constant.

by rcbdev 3y ago

morelisp 3y ago

Development time makes up a nearly immeasurable fraction of power resources outside of the tiniest teams.

rcbdev 3y ago
Imagine you have a team migrating the federal tax system of a smaller country - you can choose to develop the project in C and have X amount of devs work on it for 3-4 years 40-50h a week. The resulting code base will be huge and probably hard to maintain. Or you could choose to hire devs in a higher level language, maybe X/2 or X/1.5 amount of devs - saving you the environmental cost of running possibly hundreds of workstations every single day for years at a time and doing administration for more employees. The resulting code base will also be smaller and (hopefully) easier to maintain. I don't think you could call this immeasurable at all. (This is based on a real example)
- vidarh 3y ago
  Not just the workstations. You really need to factor in the other carbon output of those developers for the time spent on the project too, not just the tools. And you then need to consider to what extent any performance saving will be efficiently captured as reduced power use. But, yes, fully agree. One system I worked on recently involved about 20 developer years of effort, and about 40 core years of computation... I'm pretty sure the developers machines combined spent far more energy than the production systems, for a Ruby deployment, before factoring in any other energy use relating to difference in effort.
- flohofwoe 3y ago
  IME development resources will always expand to the available budget and time anyway, the programming language or programmer skills really don't matter much. Also, whether C is actually more or less "productive" than other languages for specific tasks isn't all that clear either. For the things I pick C for (for instance cross-platform libraries sitting between OS APIs and user code, and home computer emulators), it is also the most productive option (in the sense that higher level programming languages wouldn't make me more productive, because all that's needed for this type of stuff is functions, structs, loops, conditionals and a handful of math operators - and all those things are in C). High level features like automatic memory management don't make much sense when there's hardly any heap memory to be managed, and a rich stdlib also isn't needed when all you do is number crunching and bit twiddling.

vidarh 3y ago

And you need to keep this in account when considering efficiencies in general. My first production Ruby project increased our CPU usage in userspace tenfold for the service where it replaced a C version. At a point where it processed millions of messages a day, that added up to 10% of one core. The carbon cost of the development effort almost certainly outweighed the cost of running the service - either version - by a large factor, and hence any savings on dev effort might easily (might, as I didn't keep precise records) have outweighed production costs by a significant factor too. The Ruby version was less than 1/10th the size despite more features. So I agree with you. Many systems have plenty of services like that. Some systems are very clearly not like that and are worth optimizing, as you say.

benj111 3y ago
Is looking at current running costs the only metric? Computer co2 emissions are dominated by manufacture, not use. At some point the extra bloat means you have to upgrade your computer / buy another one.
- vidarh 3y ago
  The point is that for a lot of services that "at some point" never come, and/or favors avoiding "extra bloat" in the development team rather than production servers. By all means, assess the costs and benefits - I've done projects where I've optimized heavily for performance because it means we could shave tiny amounts of power use off embedded systems we hoped to deploy in large quantities; I'd certainly not deploy Ruby there. But people seem to have a tendency to overestimate the scale they will need, often dramatically, and often because they have no idea the size of their total addressable market or the amount of data available in a given space (e.g. I've seen developers worry about scaling things to large clusters in niches where actually talking it through with a business analyst would've made them realize that a total saturation of the product to 100% of the maximally possible worldwide market for their service would see it all fit on their laptop) Unless you actually sit down and work out what your realistic scaling scenarios look like, you don't know whether a tradeoff of time vs. performance will ever pay off. Even when you do need a scale where putting significant effort into performance matters, there are many scenarios where starting "inefficient" is still more efficient, because it lets you focus more effort on the parts of the system that are proven in real-world settings to actually carry a cost that matters, and the most cost-effective solutions might well be different than what you expect. E.g. one other aspect of the system that included the Ruby component I mentioned actually hit scaling challenges. Instead of investing time in rewriting it, we cut the computational cost to fractions of a percent by just caching the main pages for a few seconds (I want to say 10, but not sure the exact number). The total energy use for generating those pages once that cache was in place (reconfiguring the reverse proxy that was already in place) was so low that rewriting that part in a more efficient language would equally have saved at most fractions of a core. Instead of spending extra dev time on that, we got to spend the extra dev resources where it did matter, such as e.g. the backend search engine indexing and query engine, where processing costs certainly would have killed us if it'd been written in a slow language. Basically, know your costs. Your fully loaded, real costs, including the actual dev costs by service and by feature. Developers in particular tend to have absolutely no clue about the fully loaded costs of the choices they make, and it often shows in the choices they make. EDIT: I'd like to add that this is often not the fault of developers. It took years, and very specific circumstances (co-founding startups, that exposed me much more to the money side) before I started seriously thinking about the costs of what I built, and sitting down and actually calculating "worst case" scale and likely scale, and estimating costs of going from one to the other if we engineered for the smaller scale. It's not a habit most developers have, because it's not something most developers get asked, even though they ought to at the very least be involved (a lot of the time I've seen costing done by architects, or even worse directors or VPs with no exposure to the hands on work, it bears little resemblance to reality).

flohofwoe 3y ago

> I'm very surprised they don't even mention the efficiency of the developer in this context Because in terms of energy use it doesn't matter. Take any somewhat successful video game for example. It might take years and hundreds of developers to create, but all the energy used during that period is most likely just a fraction of running the game hundreds of thousands of times on the first day after release.

rcbdev 3y ago
This is a very specific example since video games genuinely require loads of resources to run at the end user side. Most software projects aren't client-side behemoths like that.
- flohofwoe 3y ago
  Ok, then web browsers :) Run by many more people than video games, but (not quite as) heavy when it comes to resource usage.
  - rcbdev 3y ago
    How many people do you know who work on browsers though?
    - flohofwoe 3y ago
      A handful, but there are definitely more people who *use* browsers each day than there are people developing browsers. That's the whole point of the argument ;)
      - rcbdev 3y ago
        As I said, if you are FAANG or similar - which browsers are - the trade-off might be worth it, yes. But only a handful of software projects out there are browsers or video games. That's the whole point of the argument. ;)

offices 3y ago

The development time is almost inconsequential. Software is written once-ish, read a few times, and run many, many times.

Gud 3y ago
You also have to factor in how urgently you need the software.

PH95VuimJjqBqy 3y ago

I don't agree with this, developers will just develop something else with the extra time, their footprint is a constant.