4 ms·
This criticism is flawed. The fact that the code is imperfect or sub optimal actually makes their conclusions strong and more useful for real world comparisons.
by devveppev 9y ago
This criticism is flawed. The fact that the code is imperfect or sub optimal actually makes their conclusions strong and more useful for real world comparisons.
The code guidelines in their benchmarks represent real programming practices in those languages. They promote "idiomatic" code because it is meant to represent the typical quality of code in the real world. Obviously many of the examples could be made more performant, but in doing so you would made the code less representative of the real world. Example: the typescript code outputs wanky ES6 features that are slower than their old plain js counterparts (classes, arrow functions, let, etc). You could abuse the ts code until its output is identical to js, but you would have a pointless benchmark now.
What would the benchmark honestly represent if the code looked nothing like code in the real world? The theoretical speed of the language just doesn't matter.
In fact, JS engines have been optimizing their performance around numerical benchmarks for decades. The benchmark problems (nbody, etc) are actually highly unrepresentative of real javascript performance because real world javascript is touching strings and awful DOM apis and messing with dictionaries all day.
Your 'real-world workload' should be stuff like text editor operations, a domain where JS's alleged 6x slowdown compared to C has not been remotely approached by current editors.
- Veedrac 9y ago> The code guidelines in their benchmarks represent real programming practices in those languages. They promote "idiomatic" code because it is meant to represent the typical quality of code in the real world. This could not be further from my observations when I tried contributing. These results are neither controlled nor the result of idiomatic programs.
- devveppev 9y agoI'm talking about stuff like putting your code in classes, initializing with constructors, or using your language's standard library instead of writing your own stuff. Perhaps idiomatic was the wrong word. I mean that the code isn't supposed to be fighting the language. What were your observations when you were contributing? Which problems were you working on? Any chance you still have the code?
- Veedrac 9y ago> I mean that the code isn't supposed to be fighting the language. That's the issue: those programs that did best were those that fought the language the most, and those that pushed the closest to the edge of the rules. You can't, in general, look at two programs and assume they approach the problem the same way.