10 ms·

> conflates languages and their implementations A more charitable reading might accept that language names may be used as shorthand for particular language imp

by igouy 3y ago

> conflates languages and their implementations A more charitable reading might accept that language names may be used as shorthand for particular language implementations. In this case: https://sites.google.com/view/energy-efficiency-languages/setup https://sites.google.com/view/energy-efficiency-languages/se... ~ > representative of the broader set of software written in most languages To your knowledge, did such a collection of programs — actually shown to meet that criterion — exist? ~ > Typescript and JavaScript are very different in their analysis When we emphasize outliers with arithmetic means in "Table 4. Normalized global results for Energy, Time, and Memory". With medians: JS 7.25 times slower than C TS 7.8 times slower than C ~ > all JavaScript is valid TypeScript Except `--alwaysStrict` and `--use_strict` So a JavaScript program may have failed as a TypeScript program, and a different program which worked as TypeScript may have been measured. ~ > not reproducible The authors provided a repo, including test program source code, that is still available 5 years later. page 3, footnote 1 "The measuring framework and the complete set of results are publicly available at https://sites.google.com/view/energy-efficiency-languages https://sites.google.com/view/energy-efficiency-languages"

steveklabnik 3y ago
I am not denigrating the benchmark game in this comment, I am saying that the paper does not convincingly make the argument for its thesis. I know you are proud of your work. You yourself encourage people to understand exactly what the benchmark game is and is not. Suggesting that it is representative of all programs is something that you yourself literally have in the FAQ: https://benchmarksgame-team.pages.debian.net/benchmarksgame/why-measure-toy-benchmark-programs.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > We are profoundly uninterested in claims that these measurements, of a few tiny programs, somehow define the relative performance of programming languages aka "Which programming language is fastest." Just because I do not think that using the Benchmark Game is a good idea to demonstrate their thesis does not think that I do not think the Benchmark Game is bad. Additionally, > The authors provided a repo, including test program source code, that is still available 5 years later. That link gives "We are sorry, but you do not have access to this service".
- steveklabnik 3y ago
  Trawling through the wayback machine, I did find that the older pages link to https://github.com/greensoftwarelab/Energy-Languages https://github.com/greensoftwarelab/Energy-Languages, which does seem to provide the contents of the specific programs used and the benchmarking software. Excellent.
- igouy 3y ago
  > … not denigrating the benchmark game … Agreed. My response was to your comments about the "Energy Efficiency across Programming Languages" conference paper. You found the JS/TS "very confusing": I suggested a simple cause. ~ > Suggesting that it is representative of all programs is something that you yourself literally … Huh? How have you read "profoundly uninterested" to mean "Suggesting that it is representative …" ? ~ > That link gives … I really did just click-on (Microsoft Edge) the link odyssey7 provided, click-on the "footnote 1" link in the paperSLE.pdf, click-on the "[1] Measuring Framework & Benchmarks" link, without any difficulties.

by igouy 3y ago

steveklabnik 3y ago

I am not denigrating the benchmark game in this comment, I am saying that the paper does not convincingly make the argument for its thesis. I know you are proud of your work. You yourself encourage people to understand exactly what the benchmark game is and is not. Suggesting that it is representative of all programs is something that you yourself literally have in the FAQ: https://benchmarksgame-team.pages.debian.net/benchmarksgame/why-measure-toy-benchmark-programs.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > We are profoundly uninterested in claims that these measurements, of a few tiny programs, somehow define the relative performance of programming languages aka "Which programming language is fastest." Just because I do not think that using the Benchmark Game is a good idea to demonstrate their thesis does not think that I do not think the Benchmark Game is bad. Additionally, > The authors provided a repo, including test program source code, that is still available 5 years later. That link gives "We are sorry, but you do not have access to this service".

steveklabnik 3y ago
Trawling through the wayback machine, I did find that the older pages link to https://github.com/greensoftwarelab/Energy-Languages https://github.com/greensoftwarelab/Energy-Languages, which does seem to provide the contents of the specific programs used and the benchmarking software. Excellent.
igouy 3y ago
> … not denigrating the benchmark game … Agreed. My response was to your comments about the "Energy Efficiency across Programming Languages" conference paper. You found the JS/TS "very confusing": I suggested a simple cause. ~ > Suggesting that it is representative of all programs is something that you yourself literally … Huh? How have you read "profoundly uninterested" to mean "Suggesting that it is representative …" ? ~ > That link gives … I really did just click-on (Microsoft Edge) the link odyssey7 provided, click-on the "footnote 1" link in the paperSLE.pdf, click-on the "[1] Measuring Framework & Benchmarks" link, without any difficulties.