13 ms·

Energy Efficiency Across Programming Languages

188 points by foob 9y ago

flavio81 9y ago
According to their normalized "global" results, something interesting i see: 1. Pascal, surprisingly, the most memory efficient of all. I should take a look at the implementation they used. 2. Rust a good alternative to C which leads in "energy efficiency" and speed. 3. Common Lisp most energy-efficient and fastest and smallest memory footprint of all the dynamic programming languages in the list -- like Python, Ruby, Lua, Perl, and even Racket (which fares pretty well) 4. PHP, JRuby, Ruby, Typescript, Perl, Python, being massively slow than the fastest languages, for example Ruby being 59 times slower than Rust or C. I agree that these languages (in item 4) are "acceptably fast" for many applications, but we can't say they are "close to the speed" of the fastest languages, even more if we consider the fastest dynamic+interactive languages like Lisp (Python being 21 times slower than Lisp.) 5. I wonder which implementation of Lua they used. Lua can be pretty fast, one of the fastest dynamic languages out there.
- goatlover 9y ago
  I have noticed a significant speedup using LuaJit, similar to PyPy compared to normal Python. I'm guessing if LuaJit & PyPy were used, the performance for Lua & Python would be similar to JS in these results.
- mamcx 9y ago
  > 1. Pascal Pascal is a "hidden" gem in the area of languages. Sadly not enough pus for it, but imagine if it have the push that other languages have...
  - agumonkey 9y ago
    Now I need to finish my lisp.pas
    - mamcx 9y ago
      You will like this: https://github.com/kanaka/mal https://github.com/kanaka/mal (Have implementation in pascal!)
      - agumonkey 9y ago
        I try to not look at it to not steal ideas :)
  - open-source-ux 9y ago
    "Pascal is a "hidden" gem in the area of languages" I agree. Programmers won't look at it because they perceive it to be old and out-of-date. But the language hasn't stood still. It's a fast, low-memory language. FreePascal with the Lazarus IDE is one of the best cross-platform development toolkits for building native desktop apps. Sadly a lot of programmers can never see beyond the verbose (but readable) syntax.
    - Kamgunz 9y ago
      Maybe somebody needs to create a more palatable language, that'll transpile (I know, I know, compile) into pascal?
      - mamcx 9y ago
        This guys made a pascal-like language: http://www.elementscompiler.com/elements/oxygene/ http://www.elementscompiler.com/elements/oxygene/ Exist various dialects of pascal (Modula, Oberon, etc) so is not something so far in the wind. But if you lose too much of the syntax you remove part of the charm.
      - fasquoika 9y ago
        Palatable how? Pascal's syntax is not especially different from that of most scripting languages
        andrewflnr 9y ago
        Every time I look at the syntax, it feels really verbose. IIRC some dialects require all caps or initial caps on keywords; at any rate that's how most examples are shown, and it's rather off-putting. Superficial, I know, but I'm probably not the only one.
        fasquoika 9y ago
        I don't know about other implementations but FreePascal is completely case insensitive. Identifiers are actually treated the same if the only difference is case. I don't particularly like this, but I've never seen it actually be an issue
  - CoreXtreme 9y ago
    Ruby also has gem.
  - psyc 9y ago
    When I started programming in the early 90's, I remember Pascal being a serious contender, with C vs Pascal compilers being a pretty evenly matched contest.
  - mac01021 9y ago
    But why should it's memory utilization be so much lower? Pascal and C give the programmer basically the same control over memory layout, no?
    - subwayclub 9y ago
      Basically, C's semantics assume too little - everything is a primitive type or a pointer dressed up to look like something else. Pascal has a stronger notion of types and this allows for additional automatic optimization where they're relevant - memory layout just happens to be one such area.
      - addHocker 9y ago
        I miss the ability to pair structures as "antagonistic" - aka if one exists the other is not, thus allowing for unions of various datatypes who guarantee that if they exist- the shared memory they reside in is theirs and theirs alone.
    - dmytrish 9y ago
      Just a wild guess: Pascal may use tightly packed structures without alignment by default?
  - nine_k 9y ago
    Much of Pascal (or, rather, Modula-2 and Oberon) is revived inside Go.
    - continuational 9y ago
      What parts of Pascal do you recognize in Go?
      - pjmlp 9y ago
        The id type way instead of type id. The way methods are attached to types appeared first in Oberon-2. Type assertions are similar to Oberon. The unsafe package idea is similar to SYSTEM in Oberon or Pascal. The stronger type semantics forcing explicit casts.
    - dom96 9y ago
      And even more so is revived in Nim.
- agumonkey 9y ago
  Imaging if the world ran on SBCL instead :)
  - equalunique 9y ago
    SBCL is one lisp that I intend to learn more about. It's apparently the lisp of choice for XMaxima, which is a program that I used a lot in high school.
- aaron-lebo 9y ago
  Where are you getting that Pascal is the most memory efficient (edit: duh, under the "memory efficiency" table, mistakingly read this as "efficient")? It's firmly in the middle of the pack in every result, in their global result table it's slower and less energy efficient than Java (!). It's only advantage is that it's smaller than anything else. Don't mean to assume you are wrong, but what are you referring to?
  - shock 9y ago
    In the `Normalized Global Results` table, Pascal is used as the reference (for memory use) and everything else is reported relative to it.
- copx 9y ago
  >5. I wonder which implementation of Lua they used. Lua can be pretty fast, one of the fastest dynamic languages out there. I looked at the binary trees benchmark and while the Python version used multiprocessing to parallelize the code, the Lua version was not parallelized at all.. .. not that this is entirely unrealistic. Python comes with MP in the standard library while I am not aware of any "industrial strength" solution for Lua, whether MP or thread-based.
  - igouy 9y ago
    For an interested Lua programmer, there's a kind-of workable approach -- http://benchmarksgame.alioth.debian.org/u64q/program.php?test=mandelbrot&lang=lua&id=6 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
  - wruza 9y ago
    Idk if it is industrial strong, but there is lanes and probably few more listed at lua-users. Do you have any experience to share on these? The choice of 5.3 is pretty strange, as Lua is known to split in two at 5.2, because LuaJIT was fixed to 5.1 variant with compat backports and is to my opinion an "industrial" Lua standard. 5.x are actually three different (but not absolutely superior to each other) languages with shared 5-like foundation and app-level code compatibility, so speaking about Lua in general is pointless in a sense. http://lualanes.github.io/lanes/ http://lualanes.github.io/lanes/ http://lua-users.org/wiki/MultiTasking http://lua-users.org/wiki/MultiTasking
  - etiene 9y ago
    The industry standard in Lua is to have multiple Lua states doing stuff. A different approach, but it is there
- shock 9y ago
  > 5. I wonder which implementation of Lua they used. There is a table with the versions they used¹, but they don't mention the implementation – in can be inferred for some: e.g. for Pascal they used the Free Pascal Compiler. ① - https://sites.google.com/view/energy-efficiency-languages/setup https://sites.google.com/view/energy-efficiency-languages/se...
  - vvanders 9y ago
    Ouch, they used Lua 5.3 LuaJIT is at least an order of magnitude(10x) faster.
    - leggomylibro 9y ago
      To the point where a lot of popular libraries (like Love2D) use LuaJIT by default. IIRC, Lapis (Lua-based web framework) can be easily configured for it as well. I'd be interested in seeing how it fared; Lua really is a nice language.
      - etiene 9y ago
        Lapis afaik also uses LuaJIT by default because it's built for openresty which does use LuaJIT
- ashark 9y ago
  > 4. PHP, JRuby, Ruby, Typescript, Perl, Python, being massively slow than the fastest languages, for example Ruby being 59 times slower than Rust or C. Take a look at some of the TypeScript code and compare it to the Javascript versions. At least some are way different, i.e. the TypeScript uses "modern" JS features and the JS version... looks about as much like C as it can.
  - jsd1982 9y ago
    Typescript is not its own language in terms of having a distinct runtime from Javascript so it has no performance difference and does not make sense to appear in this comparison. I can see comparing different target versions of Javascript against one another (es2015 vs es2017 for instance), but it all depends on the VM chosen as well.
    - ashark 9y ago
      Exactly. The difference at least some of these tests is measuring between the two is idiomatic modern JS (promise, map, careful variable scoping) with fast & loose old-school c-like JavaScript (for loops, global vars galore). Typescript has little to do with it.
      - sitkack 9y ago
        Typescript can use only of these, so it does make sense to have it on the list. Clearly it doesn't fall right next to JavaScript in the results. It is a benchmark of language implementations not runtimes. This is why JRuby, C# and F# are on the list, one uses the JVM and the latter two both use the CLR. Different languages.
- avodonosov 9y ago
  And Forth is missing.
- etiene 9y ago
  > 5. I wonder which implementation of Lua they used. Lua can be pretty fast, one of the fastest dynamic languages out there. Apparently 5.3, but I am very skeptical of these results. Also the way the data is organised is very weird. One should not compare a JIT compiled VM (JavaScript) with interpreters (Lua, Python, Ruby...). An honest approach would have used LuaJIT, PyPy etc. or separated the languages into "families" better. Not to mention the choice of algorithms they used. Some of the algorithms in the benchmark game have many flaws. As it was already stated in this thread, some of the Python ones were multithreaded but the ones in Lua weren't. Why? In a different benchmark I've seen algorithms using C wrappers for Python, but not in Lua. While I understand wrapping C and multithreading reflect the industry use in Python, so they do in Lua. Benchmarks are useless if you are not consistent in the implementations of the things you are comparing with. It disgusts me, to be honest, when I see benchmarks comparing JIT compilers with vanilla Lua just to say "hey we're faster", as LuaJIT does not even exist. It looks like pure dishonesty. In the case of this paper in particular, as they do not represent any language, it strikes me as lack of research / understanding.
  - igouy 9y ago
    > … but the ones in Lua weren't. Why? No mystery: someone has made the effort to contribute all those multi-process Python programs; no one has made the effort to contribute all those multi-process Lua programs.
    - etiene 9y ago
      Oh yes, that is true. But the person making the benchmarks should be aware of that and either look for better matching algorithms or implement them themselves.
      - igouy 9y ago
        Why should a comparison be restricted just because the best Lua programs-contributed fail to use multi core? (Tail wagging the dog). If someone who didn't know Lua wrote those programs, then you'd complain that the programs weren't written by an expert. If Lua wasn't included then I think you'd complain about that too.
problemsmith 9y ago
It'd be nice if these werent images of the tables ...:-\
- knolan 9y ago
  They're just screen capped from the actual tables in the paper PDF.
boznz 9y ago
Considering most CPU cores (Arm/X86/etc) are optimized for C/GCC its an unsurprising result
- nur0n 9y ago
  Can someone elaborate. I always thought that the compiler was optimized for the architecture, not the other way around. Why would an architecture be optimized for a specific compiler?
mtm 9y ago
A real shame that Forth wasn't included in the tested languages. Chuck Moore has been an advocate for more energy efficient computation for a while now.
- fnord123 9y ago
  The tests come from the benchmark game. If you want to see Forth, implement the tests in Forth: http://benchmarksgame.alioth.debian.org/ http://benchmarksgame.alioth.debian.org/
  - mncharity 9y ago
    > The tests come from the benchmark game. If you want to see Forth, implement the tests in Forth: But perhaps archive your code elsewhere. Github? Lots of language communities participated in past versions of the game. Like gforth and bigforth on shootout.alioth.debian.org circa 2008. Now it seems little remains but a few archive.org snapshots without source. Though perhaps all of that code was archived somewhere else that I'm not quickly finding?
    - igouy 9y ago
      It seems like you are entirely ignorant of the facts. Sitting in the same old-repo, a GForth binary-trees program from 2005 -- https://alioth.debian.org/scm/viewvc.php/shootout/bench/binarytrees/binarytrees.gforth?view=markup&revision=1.2&root=shootout https://alioth.debian.org/scm/viewvc.php/shootout/bench/bina... edit Now it seems little remains but for you to lift-that-shade and tell-everyone how amazing it is that someone took the trouble to archive those obscure old programs.
    - deleted 9y ago
      [deleted]
  - mncharity 9y ago
    @igouy points out that while the old shootout site is gone, its language files are still available[1]. There are about 70 benchmarks, in about 70 languages. I remember them as interesting multilingual browsing. Unlike the old shootout, adding languages directly to benchmarksgame.alioth.debian.org is not an option. "Because I know it will take more time than I choose. Been there; done that."[2] :) Instead, the code is available[3], and communities are invited to document their own benchmarksgame comparisons[2], such as Nim''s[4]. The OP used different code[5]. A brief glance at the current benchmarks, suggests there is some overlap with the old ones. Or at least an overlap of names - the requirements may have changed. So it might be possible to get started fairly easily? Perhaps even to do several languages... [1] https://alioth.debian.org/scm/viewvc.php/shootout/bench/?root=shootout https://alioth.debian.org/scm/viewvc.php/shootout/bench/?roo... [2] https://benchmarksgame.alioth.debian.org/play.html https://benchmarksgame.alioth.debian.org/play.html [3] https://github.com/Byron/benchmarksgame-cvs-mirror https://github.com/Byron/benchmarksgame-cvs-mirror [4] https://github.com/def-/nim-benchmarksgame https://github.com/def-/nim-benchmarksgame [5] https://github.com/greensoftwarelab/Energy-Languages https://github.com/greensoftwarelab/Energy-Languages
kcarnold 9y ago
I see they monitor a complete process lifetime, not just the active workload. A quick grep of the paper didn't turn up anything discussing this. And it would seem to hurt dynamic languages and runtime-JIT languages a lot. Perhaps the active workload takes long enough that the transient is washed out. Anyone have insights?
- fnord123 9y ago
  If you want to tease the startup out, then you can run the game with a null or extremely small input set and then discount the time it takes to run a small/null workload from the actual run. But really, if they hurt, then they get hurt. There's only so many knobs you can turn and so many runs you can run before you have to assemble a table and submit your paper. They put all the code online so you can try it yourself.
- igouy 9y ago
  > Perhaps the active workload takes long enough that the transient is washed out. That does seem to be the case with some-of the benchmarks game programs -- http://benchmarksgame.alioth.debian.org/sometimes-people-just-make-up-stuff.html#jvm-startup-time http://benchmarksgame.alioth.debian.org/sometimes-people-jus...
goatlover 9y ago
No Julia?
ram_rar 9y ago
I am curious to know, if there is any explanation for why java performs better than Go.
- jcelerier 9y ago
  20 years of optimizing tends to do this
- glibgil 9y ago
  Go was invented in the 1970s and then placed on the self for 40 years
  - jsd1982 9y ago
    I personally enjoy Go but your comment gave me a chuckle since it's very true. Have your upvote, friend, for whatever they're worth on HN.
- ademarre 9y ago
  I can't speak to the Java comparison, but the site says they used Go 1.6.3. The SSA back end introduced in Go 1.7 (amd64) came with a nice performance boost. https://blog.golang.org/go1.7 https://blog.golang.org/go1.7
- topspin 9y ago
  Two decades of optimization probably account for that. From the paper: Moreover, the top 5 languages that need less energy and time to execute the solutions are: C (57J, 2019ms), Rust (59J, 2103ms), C++ (77J, 3155ms), Ada (98J, 3740ms), and Java (114J, 3821ms); of these, only Java is not compiled. That is an admirable accomplishment. JRE 1.8 has the most efficient "managed" language runtime tested. The PHP result is interesting; they used 7.x which is roughly twice as efficient as older (5.x) versions; PHP 7 was primarily a (successful) effort to improve performance. If you extrapolate the PHP result to the old version (multiply by 2) it's nearly as bad as Perl and among the slowest/most costly. That certainly corroborates Facebook's several PHP re-implementations.
  - 9y ago
mapletree 9y ago
Nim compiles to C
cwzwarich 9y ago
I can't believe they used the Computer Language Benchmarks Game. Those benchmarks don't reflect real-world workloads at all, and the contest has fairly arbitrary rules about implementations and widely differing implementation quality between languages. This should have been rejected by peer review.
- igouy 9y ago
  > Those benchmarks don't reflect real-world workloads at all… That's a very definite claim, for which you provide no supporting evidence ;-)
  - oconnor663 9y ago
    Maybe we could get away with a couple smaller claims: - It would be weird to expect programs specifically optimized for a performance benchmark, to also be optimal for energy usage or memory. Maybe less weird for energy, if runtime is the biggest factor in how much energy gets used. But I'd expect there to be huge tradeoffs between runtime and memory usage, once we start really optimizing for memory. - Running the Benchmarks Game has meant doing a lot of work to figure out what the relationship between "optimal" and "representative" is, and what the rules of the game should be as a result. Even if we take it for granted that we've answered those questions in a way that gives us useful benchmarks for performance, it would be weird to expect the answers to be exactly the same in benchmarks for energy or memory.
    - jackmott 9y ago
      performance usually == energy efficiency on modern cpus. race to shutdown.
      - igouy 9y ago
        Does your comment agree or disagree with what they measure?
        9y ago
BenoitEssiambre 9y ago
Why is TypeScript so much less efficient than JavaScript?
- kerpele 9y ago
  It seems to only be significantly different with fannkuch-redux. Looking at the CLBG page (http://benchmarksgame.alioth.debian.org/u32/performance.php?test=fannkuchredux&sort=elapsed http://benchmarksgame.alioth.debian.org/u32/performance.php?...) for that algorithm there doesn't seem to be very big difference at all. I cannot explain the difference and if this is indeed an error by the authors of the paper it casts a fairly big shadow on the reliability of the rest of the data.
- jackmott 9y ago
  less optimal implementation is all
- ashark 9y ago
  Guess: its output has more lines of JS than the vanilla JS versions, and since a lot of the variation between scripting languages in these things often come down to how fast they can stop running themselves and start running some underlying C, those extra lines hurt it. This may also explain why JS is really fast in a couple of the examples. Those script likely do almost nothing but call some built-in function that does all its work in C (or C++, or whatever). [EDIT] OK, if I'm reading these[1][2] files correctly, it has more to do with the the TS version being written in "modern" "idiomatic" "beautiful" JS with promises and crap, and the vanilla JS one being written in "ugly" "bad" old-school 90's-lookin' C-ish Javascript. [1] https://github.com/greensoftwarelab/Energy-Languages/blob/master/JavaScript/regex-redux/regexredux.node-2.node https://github.com/greensoftwarelab/Energy-Languages/blob/ma... [2] https://github.com/greensoftwarelab/Energy-Languages/blob/master/TypeScript/regex-redux/regexredux.typescript-3.typescript https://github.com/greensoftwarelab/Energy-Languages/blob/ma...
stewbrew 9y ago
So go (a garbage collected language) is about as memory efficient as pascal, the most memory efficient language. That comes surprising.
- brlewis 9y ago
  Not really. "Manual" memory allocation is usually malloc/free, which leaves a lot of gaps.
  - bbatha 9y ago
    Go's garbage collector isn't compacting.
    - brlewis 9y ago
      My point wasn't that garbage collection is more efficient than people think, but that "manual" memory allocation is less efficient than people think.
      - pjmlp 9y ago
        To the point that people used GC enabled systems programming languages to write full OSes[0], but as Joe Duffy puts on his last keynote, winning prejudice is an uphill battle. [0]- Done at Xerox PARC, DEC/Olivetti, ETHZ, Erlangen, MSR
trhway 9y ago
For a bunch of years i was writing in Java and driving V8. Now i'm back to native (C++) and am driving Prius these days. I guess that it is my eco-consciousness that is making me shudder in disgust every time i look at Python... pretty much the same way like when looking at Hummers. While of course i love Perl, an M1 Abrams which gets things done despite anything :)
danieltillett 9y ago
On of the things with the C set is they didn't use Intel's icc. On our main application (which involved a lot of heavy duty number crunching) it speeds up our application by 100% over the best we can get out gcc.
- fnord123 9y ago
  I've seen 20% speedups with icc over gcc; but not a doubling. Is this including mkl over some other numeric library as well?
  - danieltillett 9y ago
    No nothing extra. The code does vectorise well. But even if the result is 20%, it would make quite a bit of difference to the results presented in this paper.
winter_blue 9y ago
Note: The PDF of the paper has the a lot more detailed listing of the results (with graphs, explanation, etc) than the results web page: http://greenlab.di.uminho.pt/wp-content/uploads/2017/09/paperSLE.pdf http://greenlab.di.uminho.pt/wp-content/uploads/2017/09/pape...
- pjmlp 9y ago
  Only while printing it, I noticed my university name there! :)
br1n0 9y ago
brief: C is the winner of all the races ( time, energy and memory)
- milcron 9y ago
  Pascal won on memory.
matt_wulfeck 9y ago
Higher-level scripting languages. Fast for programmer's time, slow for run times.
- pjmlp 9y ago
  Not really, if you look into Lisp results.
Apocryphon 9y ago
One wonders how Kotlin (or other popular JVM languages) would compare. And also Objective-C or Smalltalk.
komone 9y ago
So the moral of this story is that if you need to fix a bottleneck in the real world, you mainly use erlang then drop into C (port/nif) where necessary?
phkahler 9y ago
Rust is fairly competitive with C, but it's memory use was 50 percent higher. I bet if they focus on getting the memory usage down it will perform better too.
- steveklabnik 9y ago
  Rust uses jemalloc by default; I wonder what it would look like if you swapped the allocator.
int_19h 9y ago
I'm kinda surprised to see Go score so low on some of these, given that it's AOT-compiled to native code, and that its memory model and safety guarantees are optimization-friendly. I'm also kinda surprised that OCaml scored that high, considering how high-level it is.
- tyoverby 9y ago
  OCaml has an incredible optimizer, and many of the abstractions that one would use in OCaml are transparent to the optimizer. You could say the opposite about Go.
  - int_19h 9y ago
    Kinda makes me wonder why OCaml didn't become more popular for Unix systems programming, the way Go seems to be becoming now. Looking at the two languages, it seems like pretty much everything that Go has to offer in that department, OCaml has as well - and if it also optimizes better, why is this even a contest?
flukus 9y ago
I'd love to see some scatter plots of CPU-time and energu-time, I expected a fairly linear correlation but there looks to be quite a lot of outliers, some but not all are explained with parallelism. Memory seems all over the place too.
nur0n 9y ago
I have just started looking into Ada and it is cool to see up there with C. Does anyone know a good resource for writing fast Ada?
gwenzek 9y ago
I'm suprised at the Java-C# comparaison. When are this numbers from? The following blogspot explains how the author improved C# implementations after seeing defavorable C# results. https://anthonylloyd.github.io/blog/2017/08/15/dotnetcore-performance https://anthonylloyd.github.io/blog/2017/08/15/dotnetcore-pe...
htmlcolor 9y ago
This is the problem that I am looking to learn. May more people share more. I will follow up on this topic http://abcdeya.com http://abcdeya.com
equalunique 9y ago
Haskell did not shine here. Ocaml did surprisingly well.

188 points by foob 9y ago

According to their normalized "global" results, something interesting i see: 1. Pascal, surprisingly, the most memory efficient of all. I should take a look at the implementation they used. 2. Rust a good alternative to C which leads in "energy efficiency" and speed. 3. Common Lisp most energy-efficient and fastest and smallest memory footprint of all the dynamic programming languages in the list -- like Python, Ruby, Lua, Perl, and even Racket (which fares pretty well) 4. PHP, JRuby, Ruby, Typescript, Perl, Python, being massively slow than the fastest languages, for example Ruby being 59 times slower than Rust or C. I agree that these languages (in item 4) are "acceptably fast" for many applications, but we can't say they are "close to the speed" of the fastest languages, even more if we consider the fastest dynamic+interactive languages like Lisp (Python being 21 times slower than Lisp.) 5. I wonder which implementation of Lua they used. Lua can be pretty fast, one of the fastest dynamic languages out there.

goatlover 9y ago
I have noticed a significant speedup using LuaJit, similar to PyPy compared to normal Python. I'm guessing if LuaJit & PyPy were used, the performance for Lua & Python would be similar to JS in these results.
mamcx 9y ago
> 1. Pascal Pascal is a "hidden" gem in the area of languages. Sadly not enough pus for it, but imagine if it have the push that other languages have...
- agumonkey 9y ago
  Now I need to finish my lisp.pas
  - mamcx 9y ago
    You will like this: https://github.com/kanaka/mal https://github.com/kanaka/mal (Have implementation in pascal!)
    - agumonkey 9y ago
      I try to not look at it to not steal ideas :)
- open-source-ux 9y ago
  "Pascal is a "hidden" gem in the area of languages" I agree. Programmers won't look at it because they perceive it to be old and out-of-date. But the language hasn't stood still. It's a fast, low-memory language. FreePascal with the Lazarus IDE is one of the best cross-platform development toolkits for building native desktop apps. Sadly a lot of programmers can never see beyond the verbose (but readable) syntax.
  - Kamgunz 9y ago
    Maybe somebody needs to create a more palatable language, that'll transpile (I know, I know, compile) into pascal?
    - mamcx 9y ago
      This guys made a pascal-like language: http://www.elementscompiler.com/elements/oxygene/ http://www.elementscompiler.com/elements/oxygene/ Exist various dialects of pascal (Modula, Oberon, etc) so is not something so far in the wind. But if you lose too much of the syntax you remove part of the charm.
    - fasquoika 9y ago
      Palatable how? Pascal's syntax is not especially different from that of most scripting languages
      - andrewflnr 9y ago
        Every time I look at the syntax, it feels really verbose. IIRC some dialects require all caps or initial caps on keywords; at any rate that's how most examples are shown, and it's rather off-putting. Superficial, I know, but I'm probably not the only one.
        fasquoika 9y ago
        I don't know about other implementations but FreePascal is completely case insensitive. Identifiers are actually treated the same if the only difference is case. I don't particularly like this, but I've never seen it actually be an issue
- CoreXtreme 9y ago
  Ruby also has gem.
- psyc 9y ago
  When I started programming in the early 90's, I remember Pascal being a serious contender, with C vs Pascal compilers being a pretty evenly matched contest.
- mac01021 9y ago
  But why should it's memory utilization be so much lower? Pascal and C give the programmer basically the same control over memory layout, no?
  - subwayclub 9y ago
    Basically, C's semantics assume too little - everything is a primitive type or a pointer dressed up to look like something else. Pascal has a stronger notion of types and this allows for additional automatic optimization where they're relevant - memory layout just happens to be one such area.
    - addHocker 9y ago
      I miss the ability to pair structures as "antagonistic" - aka if one exists the other is not, thus allowing for unions of various datatypes who guarantee that if they exist- the shared memory they reside in is theirs and theirs alone.
  - dmytrish 9y ago
    Just a wild guess: Pascal may use tightly packed structures without alignment by default?
- nine_k 9y ago
  Much of Pascal (or, rather, Modula-2 and Oberon) is revived inside Go.
  - continuational 9y ago
    What parts of Pascal do you recognize in Go?
    - pjmlp 9y ago
      The id type way instead of type id. The way methods are attached to types appeared first in Oberon-2. Type assertions are similar to Oberon. The unsafe package idea is similar to SYSTEM in Oberon or Pascal. The stronger type semantics forcing explicit casts.
  - dom96 9y ago
    And even more so is revived in Nim.
agumonkey 9y ago
Imaging if the world ran on SBCL instead :)
- equalunique 9y ago
  SBCL is one lisp that I intend to learn more about. It's apparently the lisp of choice for XMaxima, which is a program that I used a lot in high school.
aaron-lebo 9y ago
Where are you getting that Pascal is the most memory efficient (edit: duh, under the "memory efficiency" table, mistakingly read this as "efficient")? It's firmly in the middle of the pack in every result, in their global result table it's slower and less energy efficient than Java (!). It's only advantage is that it's smaller than anything else. Don't mean to assume you are wrong, but what are you referring to?
- shock 9y ago
  In the `Normalized Global Results` table, Pascal is used as the reference (for memory use) and everything else is reported relative to it.
copx 9y ago
>5. I wonder which implementation of Lua they used. Lua can be pretty fast, one of the fastest dynamic languages out there. I looked at the binary trees benchmark and while the Python version used multiprocessing to parallelize the code, the Lua version was not parallelized at all.. .. not that this is entirely unrealistic. Python comes with MP in the standard library while I am not aware of any "industrial strength" solution for Lua, whether MP or thread-based.
- igouy 9y ago
  For an interested Lua programmer, there's a kind-of workable approach -- http://benchmarksgame.alioth.debian.org/u64q/program.php?test=mandelbrot&lang=lua&id=6 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
- wruza 9y ago
  Idk if it is industrial strong, but there is lanes and probably few more listed at lua-users. Do you have any experience to share on these? The choice of 5.3 is pretty strange, as Lua is known to split in two at 5.2, because LuaJIT was fixed to 5.1 variant with compat backports and is to my opinion an "industrial" Lua standard. 5.x are actually three different (but not absolutely superior to each other) languages with shared 5-like foundation and app-level code compatibility, so speaking about Lua in general is pointless in a sense. http://lualanes.github.io/lanes/ http://lualanes.github.io/lanes/ http://lua-users.org/wiki/MultiTasking http://lua-users.org/wiki/MultiTasking
- etiene 9y ago
  The industry standard in Lua is to have multiple Lua states doing stuff. A different approach, but it is there
shock 9y ago
> 5. I wonder which implementation of Lua they used. There is a table with the versions they used¹, but they don't mention the implementation – in can be inferred for some: e.g. for Pascal they used the Free Pascal Compiler. ① - https://sites.google.com/view/energy-efficiency-languages/setup https://sites.google.com/view/energy-efficiency-languages/se...
- vvanders 9y ago
  Ouch, they used Lua 5.3 LuaJIT is at least an order of magnitude(10x) faster.
  - leggomylibro 9y ago
    To the point where a lot of popular libraries (like Love2D) use LuaJIT by default. IIRC, Lapis (Lua-based web framework) can be easily configured for it as well. I'd be interested in seeing how it fared; Lua really is a nice language.
    - etiene 9y ago
      Lapis afaik also uses LuaJIT by default because it's built for openresty which does use LuaJIT
ashark 9y ago
> 4. PHP, JRuby, Ruby, Typescript, Perl, Python, being massively slow than the fastest languages, for example Ruby being 59 times slower than Rust or C. Take a look at some of the TypeScript code and compare it to the Javascript versions. At least some are way different, i.e. the TypeScript uses "modern" JS features and the JS version... looks about as much like C as it can.
- jsd1982 9y ago
  Typescript is not its own language in terms of having a distinct runtime from Javascript so it has no performance difference and does not make sense to appear in this comparison. I can see comparing different target versions of Javascript against one another (es2015 vs es2017 for instance), but it all depends on the VM chosen as well.
  - ashark 9y ago
    Exactly. The difference at least some of these tests is measuring between the two is idiomatic modern JS (promise, map, careful variable scoping) with fast & loose old-school c-like JavaScript (for loops, global vars galore). Typescript has little to do with it.
    - sitkack 9y ago
      Typescript can use only of these, so it does make sense to have it on the list. Clearly it doesn't fall right next to JavaScript in the results. It is a benchmark of language implementations not runtimes. This is why JRuby, C# and F# are on the list, one uses the JVM and the latter two both use the CLR. Different languages.
avodonosov 9y ago
And Forth is missing.
etiene 9y ago
> 5. I wonder which implementation of Lua they used. Lua can be pretty fast, one of the fastest dynamic languages out there. Apparently 5.3, but I am very skeptical of these results. Also the way the data is organised is very weird. One should not compare a JIT compiled VM (JavaScript) with interpreters (Lua, Python, Ruby...). An honest approach would have used LuaJIT, PyPy etc. or separated the languages into "families" better. Not to mention the choice of algorithms they used. Some of the algorithms in the benchmark game have many flaws. As it was already stated in this thread, some of the Python ones were multithreaded but the ones in Lua weren't. Why? In a different benchmark I've seen algorithms using C wrappers for Python, but not in Lua. While I understand wrapping C and multithreading reflect the industry use in Python, so they do in Lua. Benchmarks are useless if you are not consistent in the implementations of the things you are comparing with. It disgusts me, to be honest, when I see benchmarks comparing JIT compilers with vanilla Lua just to say "hey we're faster", as LuaJIT does not even exist. It looks like pure dishonesty. In the case of this paper in particular, as they do not represent any language, it strikes me as lack of research / understanding.
- igouy 9y ago
  > … but the ones in Lua weren't. Why? No mystery: someone has made the effort to contribute all those multi-process Python programs; no one has made the effort to contribute all those multi-process Lua programs.
  - etiene 9y ago
    Oh yes, that is true. But the person making the benchmarks should be aware of that and either look for better matching algorithms or implement them themselves.
    - igouy 9y ago
      Why should a comparison be restricted just because the best Lua programs-contributed fail to use multi core? (Tail wagging the dog). If someone who didn't know Lua wrote those programs, then you'd complain that the programs weren't written by an expert. If Lua wasn't included then I think you'd complain about that too.

problemsmith 9y ago

It'd be nice if these werent images of the tables ...:-\

knolan 9y ago
They're just screen capped from the actual tables in the paper PDF.

boznz 9y ago

Considering most CPU cores (Arm/X86/etc) are optimized for C/GCC its an unsurprising result

nur0n 9y ago
Can someone elaborate. I always thought that the compiler was optimized for the architecture, not the other way around. Why would an architecture be optimized for a specific compiler?

mtm 9y ago

A real shame that Forth wasn't included in the tested languages. Chuck Moore has been an advocate for more energy efficient computation for a while now.

fnord123 9y ago
The tests come from the benchmark game. If you want to see Forth, implement the tests in Forth: http://benchmarksgame.alioth.debian.org/ http://benchmarksgame.alioth.debian.org/
- mncharity 9y ago
  > The tests come from the benchmark game. If you want to see Forth, implement the tests in Forth: But perhaps archive your code elsewhere. Github? Lots of language communities participated in past versions of the game. Like gforth and bigforth on shootout.alioth.debian.org circa 2008. Now it seems little remains but a few archive.org snapshots without source. Though perhaps all of that code was archived somewhere else that I'm not quickly finding?
  - igouy 9y ago
    It seems like you are entirely ignorant of the facts. Sitting in the same old-repo, a GForth binary-trees program from 2005 -- https://alioth.debian.org/scm/viewvc.php/shootout/bench/binarytrees/binarytrees.gforth?view=markup&revision=1.2&root=shootout https://alioth.debian.org/scm/viewvc.php/shootout/bench/bina... edit Now it seems little remains but for you to lift-that-shade and tell-everyone how amazing it is that someone took the trouble to archive those obscure old programs.
  - deleted 9y ago
    [deleted]
- mncharity 9y ago
  @igouy points out that while the old shootout site is gone, its language files are still available[1]. There are about 70 benchmarks, in about 70 languages. I remember them as interesting multilingual browsing. Unlike the old shootout, adding languages directly to benchmarksgame.alioth.debian.org is not an option. "Because I know it will take more time than I choose. Been there; done that."[2] :) Instead, the code is available[3], and communities are invited to document their own benchmarksgame comparisons[2], such as Nim''s[4]. The OP used different code[5]. A brief glance at the current benchmarks, suggests there is some overlap with the old ones. Or at least an overlap of names - the requirements may have changed. So it might be possible to get started fairly easily? Perhaps even to do several languages... [1] https://alioth.debian.org/scm/viewvc.php/shootout/bench/?root=shootout https://alioth.debian.org/scm/viewvc.php/shootout/bench/?roo... [2] https://benchmarksgame.alioth.debian.org/play.html https://benchmarksgame.alioth.debian.org/play.html [3] https://github.com/Byron/benchmarksgame-cvs-mirror https://github.com/Byron/benchmarksgame-cvs-mirror [4] https://github.com/def-/nim-benchmarksgame https://github.com/def-/nim-benchmarksgame [5] https://github.com/greensoftwarelab/Energy-Languages https://github.com/greensoftwarelab/Energy-Languages

kcarnold 9y ago

I see they monitor a complete process lifetime, not just the active workload. A quick grep of the paper didn't turn up anything discussing this. And it would seem to hurt dynamic languages and runtime-JIT languages a lot. Perhaps the active workload takes long enough that the transient is washed out. Anyone have insights?

fnord123 9y ago
If you want to tease the startup out, then you can run the game with a null or extremely small input set and then discount the time it takes to run a small/null workload from the actual run. But really, if they hurt, then they get hurt. There's only so many knobs you can turn and so many runs you can run before you have to assemble a table and submit your paper. They put all the code online so you can try it yourself.
igouy 9y ago
> Perhaps the active workload takes long enough that the transient is washed out. That does seem to be the case with some-of the benchmarks game programs -- http://benchmarksgame.alioth.debian.org/sometimes-people-just-make-up-stuff.html#jvm-startup-time http://benchmarksgame.alioth.debian.org/sometimes-people-jus...

goatlover 9y ago

No Julia?

ram_rar 9y ago

I am curious to know, if there is any explanation for why java performs better than Go.

jcelerier 9y ago
20 years of optimizing tends to do this
glibgil 9y ago
Go was invented in the 1970s and then placed on the self for 40 years
- jsd1982 9y ago
  I personally enjoy Go but your comment gave me a chuckle since it's very true. Have your upvote, friend, for whatever they're worth on HN.
ademarre 9y ago
I can't speak to the Java comparison, but the site says they used Go 1.6.3. The SSA back end introduced in Go 1.7 (amd64) came with a nice performance boost. https://blog.golang.org/go1.7 https://blog.golang.org/go1.7
topspin 9y ago
Two decades of optimization probably account for that. From the paper: Moreover, the top 5 languages that need less energy and time to execute the solutions are: C (57J, 2019ms), Rust (59J, 2103ms), C++ (77J, 3155ms), Ada (98J, 3740ms), and Java (114J, 3821ms); of these, only Java is not compiled. That is an admirable accomplishment. JRE 1.8 has the most efficient "managed" language runtime tested. The PHP result is interesting; they used 7.x which is roughly twice as efficient as older (5.x) versions; PHP 7 was primarily a (successful) effort to improve performance. If you extrapolate the PHP result to the old version (multiply by 2) it's nearly as bad as Perl and among the slowest/most costly. That certainly corroborates Facebook's several PHP re-implementations.
- 9y ago

mapletree 9y ago

Nim compiles to C

cwzwarich 9y ago

I can't believe they used the Computer Language Benchmarks Game. Those benchmarks don't reflect real-world workloads at all, and the contest has fairly arbitrary rules about implementations and widely differing implementation quality between languages. This should have been rejected by peer review.

igouy 9y ago
> Those benchmarks don't reflect real-world workloads at all… That's a very definite claim, for which you provide no supporting evidence ;-)
- oconnor663 9y ago
  Maybe we could get away with a couple smaller claims: - It would be weird to expect programs specifically optimized for a performance benchmark, to also be optimal for energy usage or memory. Maybe less weird for energy, if runtime is the biggest factor in how much energy gets used. But I'd expect there to be huge tradeoffs between runtime and memory usage, once we start really optimizing for memory. - Running the Benchmarks Game has meant doing a lot of work to figure out what the relationship between "optimal" and "representative" is, and what the rules of the game should be as a result. Even if we take it for granted that we've answered those questions in a way that gives us useful benchmarks for performance, it would be weird to expect the answers to be exactly the same in benchmarks for energy or memory.
  - jackmott 9y ago
    performance usually == energy efficiency on modern cpus. race to shutdown.
    - igouy 9y ago
      Does your comment agree or disagree with what they measure?
      - 9y ago

BenoitEssiambre 9y ago

Why is TypeScript so much less efficient than JavaScript?

kerpele 9y ago
It seems to only be significantly different with fannkuch-redux. Looking at the CLBG page (http://benchmarksgame.alioth.debian.org/u32/performance.php?test=fannkuchredux&sort=elapsed http://benchmarksgame.alioth.debian.org/u32/performance.php?...) for that algorithm there doesn't seem to be very big difference at all. I cannot explain the difference and if this is indeed an error by the authors of the paper it casts a fairly big shadow on the reliability of the rest of the data.
jackmott 9y ago
less optimal implementation is all
ashark 9y ago
Guess: its output has more lines of JS than the vanilla JS versions, and since a lot of the variation between scripting languages in these things often come down to how fast they can stop running themselves and start running some underlying C, those extra lines hurt it. This may also explain why JS is really fast in a couple of the examples. Those script likely do almost nothing but call some built-in function that does all its work in C (or C++, or whatever). [EDIT] OK, if I'm reading these[1][2] files correctly, it has more to do with the the TS version being written in "modern" "idiomatic" "beautiful" JS with promises and crap, and the vanilla JS one being written in "ugly" "bad" old-school 90's-lookin' C-ish Javascript. [1] https://github.com/greensoftwarelab/Energy-Languages/blob/master/JavaScript/regex-redux/regexredux.node-2.node https://github.com/greensoftwarelab/Energy-Languages/blob/ma... [2] https://github.com/greensoftwarelab/Energy-Languages/blob/master/TypeScript/regex-redux/regexredux.typescript-3.typescript https://github.com/greensoftwarelab/Energy-Languages/blob/ma...

stewbrew 9y ago

So go (a garbage collected language) is about as memory efficient as pascal, the most memory efficient language. That comes surprising.

brlewis 9y ago
Not really. "Manual" memory allocation is usually malloc/free, which leaves a lot of gaps.
- bbatha 9y ago
  Go's garbage collector isn't compacting.
  - brlewis 9y ago
    My point wasn't that garbage collection is more efficient than people think, but that "manual" memory allocation is less efficient than people think.
    - pjmlp 9y ago
      To the point that people used GC enabled systems programming languages to write full OSes[0], but as Joe Duffy puts on his last keynote, winning prejudice is an uphill battle. [0]- Done at Xerox PARC, DEC/Olivetti, ETHZ, Erlangen, MSR

trhway 9y ago

For a bunch of years i was writing in Java and driving V8. Now i'm back to native (C++) and am driving Prius these days. I guess that it is my eco-consciousness that is making me shudder in disgust every time i look at Python... pretty much the same way like when looking at Hummers. While of course i love Perl, an M1 Abrams which gets things done despite anything :)

danieltillett 9y ago

On of the things with the C set is they didn't use Intel's icc. On our main application (which involved a lot of heavy duty number crunching) it speeds up our application by 100% over the best we can get out gcc.

fnord123 9y ago
I've seen 20% speedups with icc over gcc; but not a doubling. Is this including mkl over some other numeric library as well?
- danieltillett 9y ago
  No nothing extra. The code does vectorise well. But even if the result is 20%, it would make quite a bit of difference to the results presented in this paper.

winter_blue 9y ago

Note: The PDF of the paper has the a lot more detailed listing of the results (with graphs, explanation, etc) than the results web page: http://greenlab.di.uminho.pt/wp-content/uploads/2017/09/paperSLE.pdf http://greenlab.di.uminho.pt/wp-content/uploads/2017/09/pape...

pjmlp 9y ago
Only while printing it, I noticed my university name there! :)

br1n0 9y ago

brief: C is the winner of all the races ( time, energy and memory)

milcron 9y ago
Pascal won on memory.

matt_wulfeck 9y ago

Higher-level scripting languages. Fast for programmer's time, slow for run times.

pjmlp 9y ago
Not really, if you look into Lisp results.

Apocryphon 9y ago

One wonders how Kotlin (or other popular JVM languages) would compare. And also Objective-C or Smalltalk.

komone 9y ago

So the moral of this story is that if you need to fix a bottleneck in the real world, you mainly use erlang then drop into C (port/nif) where necessary?

phkahler 9y ago

Rust is fairly competitive with C, but it's memory use was 50 percent higher. I bet if they focus on getting the memory usage down it will perform better too.

steveklabnik 9y ago
Rust uses jemalloc by default; I wonder what it would look like if you swapped the allocator.

int_19h 9y ago

I'm kinda surprised to see Go score so low on some of these, given that it's AOT-compiled to native code, and that its memory model and safety guarantees are optimization-friendly. I'm also kinda surprised that OCaml scored that high, considering how high-level it is.

tyoverby 9y ago
OCaml has an incredible optimizer, and many of the abstractions that one would use in OCaml are transparent to the optimizer. You could say the opposite about Go.
- int_19h 9y ago
  Kinda makes me wonder why OCaml didn't become more popular for Unix systems programming, the way Go seems to be becoming now. Looking at the two languages, it seems like pretty much everything that Go has to offer in that department, OCaml has as well - and if it also optimizes better, why is this even a contest?

flukus 9y ago

I'd love to see some scatter plots of CPU-time and energu-time, I expected a fairly linear correlation but there looks to be quite a lot of outliers, some but not all are explained with parallelism. Memory seems all over the place too.

nur0n 9y ago

I have just started looking into Ada and it is cool to see up there with C. Does anyone know a good resource for writing fast Ada?

gwenzek 9y ago

I'm suprised at the Java-C# comparaison. When are this numbers from? The following blogspot explains how the author improved C# implementations after seeing defavorable C# results. https://anthonylloyd.github.io/blog/2017/08/15/dotnetcore-performance https://anthonylloyd.github.io/blog/2017/08/15/dotnetcore-pe...

htmlcolor 9y ago

This is the problem that I am looking to learn. May more people share more. I will follow up on this topic http://abcdeya.com http://abcdeya.com

equalunique 9y ago

Haskell did not shine here. Ocaml did surprisingly well.