4 ms·

How can you trust a list that places JavaScript an order of magnitude lower than TypeScript? Seems fundamentally broken.

andsoitis 3y ago
> How can you trust a list that places JavaScript an order of magnitude lower than TypeScript? Seems fundamentally broken. The list places JavaScript at a score of 4.45 and TypeScript worse at 21.5. For reference, C, the most energy efficient it 1.0
- BackBlast 3y ago
  > The list places JavaScript at a score of 4.45 and TypeScript worse at 21.5. For reference, C, the most energy efficient it 1.0 The point is that TypeScript is JavaScript with typing syntax added on top. There is a transpile step into JS. That's how TypeScript works. The runtimes are exactly the same. Unless we're also measuring the build step? Which seems silly. The difference should be near zero. And it's not. They clearly do not understand exactly what they are measuring.
  - andsoitis 3y ago
    One explanation is that the typical JS output by the TS compiler is more verbose (so more code to load, parse, and run). On the other hand, one can imagine that the TS compiler’s catching classes of errors at compile time means fewer runtime exceptions. I think the takeaway, however is the theme of a spectrum of energy efficiency, with compiled languages being more efficient than those that are not. But I still maintain that the overall efficiency of a system is more a function of other factors.
    - BackBlast 3y ago
      My take away is that this list is very unreliable.
  - soegaard 3y ago
    I don't know what the usual speed difference of TypeScript programs vs JavaScript programs. But the argument that TypeScript generates a JavaScript, so it must have similar speed doesn't hold in general. If the compiler in question is a whole-program compiler, it can make optimizations that a normal person couldn't do. As an anecdote in [1] a raytracing program were implemented in both Scheme and C. The Stalin compiler (a whole-program Scheme compiler) produced an executable 45% faster than the one produced by g++. The Stalin compiler produced the excecuable by compiling Scheme to C, and then used a C compiler to produce the final executable. The price of a whole-program compiler? Well, the compile time are huge. [1] Scroll to Siskind's comment. https://groups.google.com/g/comp.lang.scheme/c/NJxRsdMNKz4 https://groups.google.com/g/comp.lang.scheme/c/NJxRsdMNKz4 For those curious about the actual programs and results: https://web.archive.org/web/20071011073406/http://www.ffconsultancy.com/languages/ray_tracer/index.html https://web.archive.org/web/20071011073406/http://www.ffcons...
    - deleted 3y ago
      [deleted]
  - igouy 3y ago
    > with typing syntax Programs that didn't type check were excluded. https://github.com/greensoftwarelab/Energy-Languages/issues/3#issuecomment https://github.com/greensoftwarelab/Energy-Languages/issues/...
- igouy 3y ago
  The authors chose to summarize their data using the mean, and that highlights the variance. Using the Geometric Mean or the Median with the time measurements from that table would have highlighted the middle value, like this: JS 7.25 TS 7.80 https://benchmarksgame-team.pages.debian.net/benchmarksgame/sometimes-people-just-make-up-stuff.html#averages https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
badsectoracula 3y ago
I didn't read the original paper[0] but it is based on the Debian Benchmark Game[1] as it was in 2017 and the game relies on user submitted code for each language separately, so most likely when the researchers checked the results, the TypeScript tests didn't have as optimized tests as the JavaScript tests - which makes sense as the latter is way more popular than the former, so there are less people to bother with it (and in fact the benchmarks nowadays do not include TypeScript at all). You can confirm it via archive.org too: the TypeScript[2] page shows both less implementations and the implementations that are shown are often slower than those in the JavaScript[3] page. As for if that makes sense, well, IMO using the benchmark game for judging how good a language is at optimizations is flawed in the first place as not only there is a bias towards the more popular languages but also a lot of the top entries use approaches that in practice you wouldn't find in real projects. [0] https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sleFinal.pdf https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sle... [1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [2] http://web.archive.org/web/20170425064751/https://benchmarksgame.alioth.debian.org/u64q/typescript.html http://web.archive.org/web/20170425064751/https://benchmarks... [3] http://web.archive.org/web/20170425114504/https://benchmarksgame.alioth.debian.org/u64q/javascript.html http://web.archive.org/web/20170425114504/https://benchmarks...
- IshKebab 3y ago
  I fixed the Typescript implementation 3 years ago when this paper was debunked back then. Didn't submit it though because they make it difficult. https://news.ycombinator.com/item?id=24826453 https://news.ycombinator.com/item?id=24826453 (the whole thread is worth a read)
  - igouy 3y ago
    And-yet your program was accepted and included, and then with a later TypeScript update it stopped working. spectralnorm.typescript-6.ts(115,29): error TS2794: Expected 1 arguments, but got 0. Did you forget to include 'void' in your type argument to 'Promise'? Something like that is probably what stopped the authors of "Energy Efficiency across Programming Languages". https://sites.google.com/view/energy-efficiency-languages https://sites.google.com/view/energy-efficiency-languages
igouy 3y ago
> How can you trust a list that places JavaScript an order of magnitude lower than TypeScript? Simple: start with the same data, make the same calculations, see the same results. When we make assumptions about how measurements were made and analyzed, our assumptions may be wrong.

andsoitis 3y ago

> How can you trust a list that places JavaScript an order of magnitude lower than TypeScript? Seems fundamentally broken. The list places JavaScript at a score of 4.45 and TypeScript worse at 21.5. For reference, C, the most energy efficient it 1.0

BackBlast 3y ago
> The list places JavaScript at a score of 4.45 and TypeScript worse at 21.5. For reference, C, the most energy efficient it 1.0 The point is that TypeScript is JavaScript with typing syntax added on top. There is a transpile step into JS. That's how TypeScript works. The runtimes are exactly the same. Unless we're also measuring the build step? Which seems silly. The difference should be near zero. And it's not. They clearly do not understand exactly what they are measuring.
- andsoitis 3y ago
  One explanation is that the typical JS output by the TS compiler is more verbose (so more code to load, parse, and run). On the other hand, one can imagine that the TS compiler’s catching classes of errors at compile time means fewer runtime exceptions. I think the takeaway, however is the theme of a spectrum of energy efficiency, with compiled languages being more efficient than those that are not. But I still maintain that the overall efficiency of a system is more a function of other factors.
  - BackBlast 3y ago
    My take away is that this list is very unreliable.
- soegaard 3y ago
  I don't know what the usual speed difference of TypeScript programs vs JavaScript programs. But the argument that TypeScript generates a JavaScript, so it must have similar speed doesn't hold in general. If the compiler in question is a whole-program compiler, it can make optimizations that a normal person couldn't do. As an anecdote in [1] a raytracing program were implemented in both Scheme and C. The Stalin compiler (a whole-program Scheme compiler) produced an executable 45% faster than the one produced by g++. The Stalin compiler produced the excecuable by compiling Scheme to C, and then used a C compiler to produce the final executable. The price of a whole-program compiler? Well, the compile time are huge. [1] Scroll to Siskind's comment. https://groups.google.com/g/comp.lang.scheme/c/NJxRsdMNKz4 https://groups.google.com/g/comp.lang.scheme/c/NJxRsdMNKz4 For those curious about the actual programs and results: https://web.archive.org/web/20071011073406/http://www.ffconsultancy.com/languages/ray_tracer/index.html https://web.archive.org/web/20071011073406/http://www.ffcons...
  - deleted 3y ago
    [deleted]
- igouy 3y ago
  > with typing syntax Programs that didn't type check were excluded. https://github.com/greensoftwarelab/Energy-Languages/issues/3#issuecomment https://github.com/greensoftwarelab/Energy-Languages/issues/...
igouy 3y ago
The authors chose to summarize their data using the mean, and that highlights the variance. Using the Geometric Mean or the Median with the time measurements from that table would have highlighted the middle value, like this: JS 7.25 TS 7.80 https://benchmarksgame-team.pages.debian.net/benchmarksgame/sometimes-people-just-make-up-stuff.html#averages https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

badsectoracula 3y ago

I didn't read the original paper[0] but it is based on the Debian Benchmark Game[1] as it was in 2017 and the game relies on user submitted code for each language separately, so most likely when the researchers checked the results, the TypeScript tests didn't have as optimized tests as the JavaScript tests - which makes sense as the latter is way more popular than the former, so there are less people to bother with it (and in fact the benchmarks nowadays do not include TypeScript at all). You can confirm it via archive.org too: the TypeScript[2] page shows both less implementations and the implementations that are shown are often slower than those in the JavaScript[3] page. As for if that makes sense, well, IMO using the benchmark game for judging how good a language is at optimizations is flawed in the first place as not only there is a bias towards the more popular languages but also a lot of the top entries use approaches that in practice you wouldn't find in real projects. [0] https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sleFinal.pdf https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sle... [1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [2] http://web.archive.org/web/20170425064751/https://benchmarksgame.alioth.debian.org/u64q/typescript.html http://web.archive.org/web/20170425064751/https://benchmarks... [3] http://web.archive.org/web/20170425114504/https://benchmarksgame.alioth.debian.org/u64q/javascript.html http://web.archive.org/web/20170425114504/https://benchmarks...

IshKebab 3y ago
I fixed the Typescript implementation 3 years ago when this paper was debunked back then. Didn't submit it though because they make it difficult. https://news.ycombinator.com/item?id=24826453 https://news.ycombinator.com/item?id=24826453 (the whole thread is worth a read)
- igouy 3y ago
  And-yet your program was accepted and included, and then with a later TypeScript update it stopped working. spectralnorm.typescript-6.ts(115,29): error TS2794: Expected 1 arguments, but got 0. Did you forget to include 'void' in your type argument to 'Promise'? Something like that is probably what stopped the authors of "Energy Efficiency across Programming Languages". https://sites.google.com/view/energy-efficiency-languages https://sites.google.com/view/energy-efficiency-languages

igouy 3y ago

> How can you trust a list that places JavaScript an order of magnitude lower than TypeScript? Simple: start with the same data, make the same calculations, see the same results. When we make assumptions about how measurements were made and analyzed, our assumptions may be wrong.