13 ms·
Energy Efficiency across Programming Languages [pdf]
- tines 6y agoI did some research on this topic in university, and our consistent result for CPU-based programs was: if it finishes faster, it uses less energy, and vice versa. So it's no surprise to see that VM-based programs use more energy; they're slower.
- gameswithgo 6y agoPeople like to argue this point because there are rare exceptions when it isn't true, but yes, generally the faster the things finishes, the less energy you use. It is a fine rule of thumb to use in absence of direct power draw.
- corysama 6y agoIn SDK docs for the GameBoy Advance there was a note along the lines of "Even if your game is so simple that it does not require the full speed of the (16.78 MHz) machine, please put effort into optimizing it anyway so that it can spend more time sleeping in the low-power PAUSE instruction. Your players will notice the difference in battery drain and they will tell their friends."
- cozzyd 6y agoMemory usage matters too! Languages lacking shared memory multiprocessing will use more energy even if they're equally fast.
- tines 6y agoThat's a good point. DRAM only sucks power when you flip the bits right?
- cozzyd 6y agoWell it has to be constantly refreshed (I think?) if there's something important in it. But more I was thinking that you need more memory to support less-efficient memory usage in the first place, or, if the amount of memory is fixed, less of it would be used for things like the disk cache.
- gojomo 6y agoOh heavens, busybodies are gonna ban Python and Ruby under the guise of fighting climate change.
- gameswithgo 6y agoI've been fighting to ban use of interpreted code in production because I hate slow stuff, and like money. So you can use those motivations instead if you like. If you love Python and Ruby, make compilation a first class method of running them.
- Gibbon1 6y agoWhen I looked at the results seems Swift and C# are pretty good general purpose languages. Interesting to see how bad Typescript compares to JavaScript.
- saagarjha 6y agoSadly those two are fairly tied to a specific platform (note: yes, I know they are technically cross-platform).
- gameswithgo 6y agoc# is extremely cross platform now
- Gibbon1 6y agoI think the current big deal is GUI support. Seems like they are carefully attacking the problem. If they succeed react is going to be for a world of hurt.
- gameswithgo 6y agothe performance ceiling of c# has gone up a lot since that data was collected too.
- 6y ago
- cphajduk 6y agoInteresting to see Rust beat out C++ very slightly.
- petermcneeley 6y agoGiven that C++ is (mostly) as superset of C its not simply interesting its odd. Did they not simply give the C program to C++? Are they claiming that the C++ compiler compiles C programs to slower lower perf binary.
- rsanders 6y agoThat's a good point. But then you're not benchmarking C++ as a distinct language. So what would sufficiently distinguish a C++ program from a C program? Let's assume it's not just minor incompatibilities introduced to prevent compilation by a C compiler. There must have used some definition that is not explicit in the paper, but you can see in this code sample that the author used various C++ standard data types (std::string, std::array), iterators, classes, concurrency (std::thread). I'm no judge of C++ style, but perhaps it's "C++ as a C++ developer circa 1997 would have written it". https://github.com/greensoftwarelab/Energy-Languages/blob/master/C%2B%2B/fasta/fasta.gpp-5.c%2B%2B https://github.com/greensoftwarelab/Energy-Languages/blob/ma...
- iTokio 6y agoIt’s probable that they are mostly measuring implementations efficiency or GCC vs LLVM for these languages.
- dochtman 6y agoGiven that it's from 2017, I'm guessing today's results might be quite different.
- moonchild 6y agoIf you look at some of the individual benchmarks, they show that c, c++, rust, ada, and fortran are all over each other. I expect the difference in this case is either due to differences between llvm and gcc; differences in the standard library implementation; or because rust requires strict aliasing by default.
- StillBored 6y agoSo, the Steve Jobs rumor about only allowing compiled programs on the original iphone was right? There is about a 4x energy increase going to a VM'ed language and about a 19x going to a fully interpreted one over using a natively compiled language. So, the energy efficiency is actually worse than the perf loss in general.
- gameswithgo 6y agoDoesn't just save battery but leads to more responsive interface, which people noticed.
- fauigerzigerk 6y ago>There is about a 4x energy increase going to a VM'ed language That's not what this data shows. Java is at 1.98x, ahead of Swift (2.79x), Pascal (2.14x) and Fortran (2.52x).
- StillBored 6y agoLook in the summary section where they provide overall perf per class. As you point out some languages are better than others within the class. But your comparing the best in one class with some of the worst in others. That is disingenuous, I might buy comparing the best in each class, but the results are much the same as the overall.
- fauigerzigerk 6y agoI am comparing with the best in class of non-VM languages (C). That's what all the multiples mean. What Java shows (and a comparison of averages per class wouldn't show) is that there is not necessarily a 4x decrease in efficiency as a result of using a VM. It depends on the implementation. And there are quite a few other VM languages that are doing far better than 4x. Swift clearly demonstrates that native AOT compilation is no guarantee for efficiency. Swift may well have become faster since this study was run (same goes for other languages), but using reference counting for garbage collection will make it very hard to catch up to the best.
- leothekim 6y agoThis appears to have been published in 2017?
- tastyminerals2 6y agoLua memory consumption is surprisingly very high.
- drmeister 6y agoHuh - I somehow submitted the same message twice. Hacker News doesn't let me delete it. So I'll edit it down - see the version above about Common Lisp and our implementation of it called Clasp.
- nynx 6y agoYou're developing clasp for use in your matter compiler project, correct?
- drmeister 6y agoI did - and we compiled our first batch of molecules using an application implemented in Cando (Clasp + computational chemistry code) yesterday. I'm absolutely serious about this.
- nynx 6y agoAmazing. Have the goals of the project changed over time, or is it still to generate molecules that fit a certain shape/function based on a number of known building blocks?
- drmeister 6y agoThe goals have not changed. The pace of development (in the chemistry) has accelerated by orders of magnitude in the last year. The software is advancing as well. We are still looking for good developers who want to work with us.
- transfire 6y agoNo Forth :(
- drmeister 6y agoLisp (Common Lisp) beats all the other dynamic languages by a considerable margin. This is why I am developing Clasp - a Common Lisp implementation based on LLVM that interoperates with C++/C (https://github.com/clasp-developers/clasp.git https://github.com/clasp-developers/clasp.git) for scientific programming. With Clasp, we get the best of multiple worlds. We get a dynamic language (Common Lisp) with automatic memory management and enormous expressive power that can directly use powerful C and C++ libraries. All three of these languages are "long-lived" languages in that code that was written 10 and 20 years ago still works. Performance is really important to me and I have written a lot of code over the past four decades. I won't develop meaningful code in any language that falls below Racket in table 4 because these language implementations are too inefficient. I furthermore want to keep using my code over the years and decades and so I won't develop meaningful code in any language where someone else can break my code by changing the standard. My program "leap" was written 27 years ago in C and it is still being used daily by thousands of computational chemists. But it's really hard to improve leap because the code is brittle largely because of malloc/free-style memory management (brrr). For a compiled, high-performance, standard language with proven lasting power and performance - Common Lisp is the best choice.
- agumonkey 6y agoI'm often surprised how sbcl is 'benchmark' competitive with other native langs (c, ada, pascal)
- drmeister 6y agoSBCL is an amazing implementation and it has an amazing compiler. I would argue that it is one of the best compilers around. Javascript compilers in browsers are pretty impressive - but given the far fewer resources that SBCL development has had, SBCL is remarkable. It's a fast compiler that generates fast code. I would be using it (and do for some applications) if I didn't also need my own large C++ computational chemistry code written over decades.
- agumonkey 6y ago
- stephc_int13 6y agoMarginal differences should be ignored in this kind of benchmark. It is usually well accepted that faster execution leads to lower power usage, as long as the CPU is operating in a reasonable thermal envelope. Nothing new here, except that we can have a better grasp of the different orders of magnitude.
- lliamander 6y agoThat Erlang is relatively power inefficient doesn't surprise me. I wonder how much of that is due to the "busy wait" it uses to reduce the latency in message processing.
- devmunchies 6y agonot really a fair comparison since erlang is a fully fledged operating system (joke). in general functional languages do worse due to the abstraction, and then VM languages do worse still, and then dynamically typed languages are less efficient than statically typed. Erlang is all the above. F# fits the first 2 (functional lang on a VM) and has a pretty bad energy rating despite being a fast language.
- lliamander 6y ago> not really a fair comparison since erlang is a fully fledged operating system (joke). Oh, Robert Virding told me once the plan was to make it essentially an OS. So, it's only a joke in the "Ha ha but seriously" sense. Lisp also fits all those criteria and is quite efficient, but it developed under different design constraints.
- dnautics 6y agoThe workloads here are numerical so erlang has to do a bunch of unboxing and boxing. "Programming languages shootout" is generally not a useful metric to judge anything on the erlang VM.
- tastyminerals2 6y agoI wonder how these rankings would have changed for C++ and Rust if they included compile times :)
- Jetroid 6y agoI wouldn't consider compile time to be a useful metric here, as you will generally compile once (for the final release candidate(s)) and execute many times.
- tastyminerals2 6y agoTrue. But maybe during development it could.
- atroxone 6y agoWhy do you consider typescript and javascript different languages?
- krapp 6y agoCan you put a program written in typescript between <script> tags and have it run natively in the browser?
- deleted 6y ago[deleted]
- 7thaccount 6y agoNot OP and don't use JS, but generally speaking, a lot of these languages that transpile one language into another (or compile one language to another for those that insist transpile is the wrong word), it comes up with a less efficient result than if I had just used the original language. Or at least a different program. So if I use Type Script that gets converted to JavaScript, the end result will be different than if I had done it like that to begin with (not necessarily slower either). Also...I guess there are situations where using something (Ex: C) could lead to faster code than me using Assembly by myself if the compiler is smarter than me (GCC knows a lot more about hardware than I do).
- k__ 6y agoI once read that coding in TypeScript leads to more monomorphic code, which can be better optimized by the JS VM.
- connor4312 6y ago> So if I use Type Script that gets converted to JavaScript, the end result will be different than if I had done it like that to begin with This isn't true for Typescript. TypeScript is a superset of JavaScript, and running JavaScript through the TypeScript compiler will produce identical output code[1]. If you add type annotations to that JavaScript to make it fully pass the strictest type checking settings, that will still be true provided you didn't otherwise rearrange or modify your program. This makes me doubt their methodology on TypeScript at least, or wonder if they're running a tool like `ts-node` which compiles and runs at the same time, thus counting compile time in their execution time and energy. 1. As long as you're targeting the same language version as the original code was written for. For instance the compiler will downlevel async/await into slower async generators if you're targeting a version of JavaScript which doesn't support it yet.
- zachruss92 6y agoSomething I've been thinking a lot about lately is environmental friendliness in software, given that data centers contribute 3% of global greenhouse emissions (same amount as the entire airline indistry). I'm thinking along the lines of using interpreted languages less server side because of efficiency, but also relying on JS less client side and using WASM where it makes sense. This has stemmed from me leaning Go last year and being moved by actually how much faster it is than Node for my use cases (API development and data processing). Where I am curious to see the total impact is how we can take advantage of economies of scale to save money and increase efficiency. I'm thinking along the lines of scale to zero, event driven architectures. Google Cloud, for example, claims they operate their data centers with industry leading efficiency while also being powered by renewable energy. At scale, do services like Cloud Run or Cloud Functions actually make a difference?
- open-source-ux 6y agoWhen I look at other industries I see efforts to reduce energy consumption. A lot of programmers, in contrast, couldn't care less. The most popular languages are the least energy efficient and most resource intensive. But they make life easier for developers, or as programmers love to say 'more productive'. When performance is an issue in running programs, a common response is: hardware is cheap, just add another energy-guzzling server or use a more powerful computer. This attitude is embarrassing when you consider that in every other industry there is a push for reduced resource usage and lower energy consumption. The programming field is the exception. On the other hand, when it's programmers who are on the receiving end of slow, resource intensive apps, they'll complain loudly. This industry is rife with hypocrisy.
- user5994461 6y agoThere is absolutely an objective to reduce the use of compute resources. Especially in the current era of AWS/google cloud where you pay $50 per core per month. Companies care very much about resource usage past some amount of machines. They don't think in terms of power consumption or environmental footprint though, but in real dollars (the costs of hardware/cloud) that is highly correlated. Kubernetes is the latest trend in spite of being an overcomplicated mess, precisely because it's an overcomplicated mess that can deploy and pack services more efficiently onto (less) resources.
- beached_whale 6y agoI would love to see the one where they do CRUD'y things or do data transformations which seem to occupy a large part of compute power.
- wmf 6y agoI'd predict that it would look very similar to the TechEmpower benchmarks.
- beached_whale 6y agoThat's pretty neat. Somewhat reinforces my experience in how things will perform.
- dleslie 6y agoI wonder what the carbon impact the use of those inefficient dynamic languages has had, in both desktop and backend environments? I imagine it's substantial and worth considering.
- jeffbee 6y agoI bet it is completely insignificant in the great scheme. The average American, for example, uses the equivalent of 4 kW of motor fuels continuously, not even counting electricity consumption. It doesn't amount to a hill of beans if the Dropbox sync client uses an extra joule here and there because it's written in Python instead of C++.
- deleted 6y ago[deleted]
- jcelerier 6y ago> The average American, for example, uses the equivalent of 4 kW of motor fuels continuously how is that possible ? I have a normal house with a few appliances turned on, plus three fairly powerful computers and some musical gear and I'm at 530VA right now according to my home's electricity meter
- jeffbee 6y agoBut how do you move yourself from place to place? Note that it's not necessary for you to personally burn the three gallons of fuel per day. It's an average.
- jcelerier 6y agoI drive on average 8k kilometers per year. I doubt that this is amounting to all the rest, is it ? a quick computation gives me, given that my car drinks roughly 7 liters of diesel / 100km if i'm not careful: - 560 liters of diesel / year - a liter of diesel is apparently ~10.74 kwh -> 6014 kwh total - 6014 / 365 -> 16 kwh a day ? I don't see how this is making me any closer to 4000 kwh per day.
- lliamander 6y agoThis is cool. Now, can we get a comparison of these results vs. LOC? I feel like almost any assessment of programming languages should have a table weighting the results based on how many lines of code it took to get that result.
- gok 6y agopreviously https://news.ycombinator.com/item?id=15249289 https://news.ycombinator.com/item?id=15249289
- dnautics 6y agoThese are computationally-heavy workloads. Do most HN programmers really do work in those domains? Is most computational work done today even in those domains (possibly, due to the amount of streaming videos, but also most of us are not coding video streamers)? Maybe a more interesting to test workload would be parsing medium-large random JSONs issued by concurrently-connecting entities. And also comparing the same setup under a low-workload scenario with a high workload scenario, possibly also comparing orchestration engines (e.g. kubernetes autoscaling). I'd also be curious to probe "worst case" scenarios. Can you cause kubernetes to thrash spinning up and killing containers really badly, and how much of an effect does that have on energy consumption?
- egsmi 6y agoI was really interested in how they measured power because there is a ton of nuance there. They used the metric reported by a tool that limits average power via a programmable power limiter in hardware which an interesting way to do it. Totally valid but I really wish they provided more detail here. For example, did all workloads run at limit all the time? Presumably they did. Limit based throttling is a form of hysteretic control so the penalty part will be critical. How often and when the limit is hit will be critical too.
- m0zg 6y agoI'm surprised that Rust is ahead of C++ in their ranking, and by quite a bit. I tend to use C++ as "C with STL and smart pointers" basically (a-la Google), I don't see why it'd be any slower or less energy efficient than C.
- ncmncm 6y agoAgreed. Bad code is slower, news at 11.
- cellularmitosis 6y agoSuch a shame they didn't include LuaJIT.
- cryptoquick 6y agoIt's worth noting that since this was published several years ago, and Rust has come a long way since then, it might very well top most of these benchmarks nowadays.
- papaf 6y agoIsn't Rust dependent on LLVM for its optimizations? Also, do you imagine that C++ and Fortran compiler developers have stopped optimizing?
- HellzStormer 6y agoThere is still a lot of work that has to be done before the ball is given to LLVM. If that gets optimized, it can improve the results. Also, I would expect a more recent language to have a lot more low-hanging fruits than much older and highly used languages. The more you optimize, the harder it gets to optimize more.
- ncmncm 6y agoAnything that seems to demonstrate C++ as slower than C is implicitly busted. You could compile the C code with the C++ compiler and get the same speed. I'm looking at their table 4, with C:1.0, C++:1.56. This throws the whole paper into doubt. Comparing crappy code in one language with good code in another reveals little of substance.
- cozzyd 6y agoYou have to include the compile time for C++ and the servers for cppreference.com.
- higerordermap 6y agoThis is funny but you can install OS documentaion for libc++ on linux systems, and access them through man pages. As for compile time, the stuff is hard. There are some caching compilers, and build systems like bazel. A good build configuration can improve compile times.
- higerordermap 6y agoIdiomatic C++ has lot of performance pitfalls.
- Zelizz 6y agoC is not a true subset of C++ (though several compilers can handle both).
- ncmncm 6y agoThe difference between C and the C subset of C++ is negligible, for any practical purpose.
- dhab 6y ago> In order to properly compare the languages, we needed to collect the energy consumed by a single execution of a specific solution. With this, Java ranking on top 5 is quite impressive. Considering that JIT optimisations wouldn't have really kicked in. My hypothesis is that if the Java program was allowed to run a few more times, and then compared, it would rank higher. And, along the lines, couldn't the other compiled languages and vm-based (common lisp, racket) be JIT optimised?
- afrojack123 6y agoWhich real life use case would Go be better than Java? Go uses less memory but is slower than Java. Which uses cases use a lot of memory?
- higerordermap 6y agoCloud.
- aib 6y agoAs if there weren't enough reasons to learn Rust already, there's a new one: We owe it to the planet.
- zelphirkalt 6y agoIt would be interesting to have a similar research done for distributed systems as well. Then one would have to choose programming language and library or framework for distributed systems development, if the language or the runtime einvironment does not offer support for it out of the box.
- lebuffon 6y agoFunny (old) energy efficiency story that used to be published on- line but I can't find it. It's about the first handheld scanner for a large shipping company. The hardware was engineered and nailed down and a team was contracted to write the software. They got about 1/2 way completed and said the box didn't have enough ROM to handle all the features in the spec. The company contracted Forth Inc. to try and salvage the project and that was possible because they used their proprietary Forth VM and factored the heck out the code so they could reuse as much as possible and got it all to fit. (Common Forth trick) 10 Years later, a new device was commissioned and management was smarter! They made sure their was a lot more memory on board and a new contracted team finished the job. In the field however the batteries would not last an entire shift... Forth Inc was called again. They made use of their cooperative tasking system to put the machine to sleep at every reasonable opportunity and wake it from user input. Maybe it ain't the language that matters as much as the skill and imagination of the designers and coders. Just sayin'