4 ms·
> Most obviously, if your study claims that C++ uses 34% more energy, 56% more time, and 14% more memory than C, it’s time to reexamine your assumptions. Approx
by rectang 5y ago
> Most obviously, if your study claims that C++ uses 34% more energy, 56% more time, and 14% more memory than C, it’s time to reexamine your assumptions. Approximately every C program is a valid C++ program, so C++ can’t lose, especially not that badly!
Benchmarking software is so easy to do wrong. It's so easy to manipulate the results if you are so inclined, and even when you're trying to be evenhanded it's so easy to disadvantage one of the systems under test.
Different approaches are called for with different tools, and the classic error we see over and over again in benchmarking studies is the inappropriate application of idioms which suit one of the competitors across the others.
- gpderetta 5y agoNever trust a benchmark you didn't falsify yourself. Benchmarks can be useful as long as you can actually understand the reasons for the performance differences to be able to evaluate the pro and cons of the benchmarked systems.
- PeterCorless 5y ago> "Never trust a benchmark you didn't falsify yourself." Stealing this.
- gpderetta 5y agoFor what is worth, I stole it myself ;)
- PeterCorless 5y agoI am wondering if James Kanze is the source: As my friend James Kanze put it, "never trust a benchmark you didn't falsify yourself." https://stackoverflow.com/questions/5326269/is-c-sharp-really-slower-than-say-c https://stackoverflow.com/questions/5326269/is-c-sharp-reall...
- gpderetta 5y agoIndeed he is, from comp.lang.c++!
- tacitusarc 5y agoAgenda-driven benchmarks and decision-making benchmarks are dramatically different things.
- igouy 5y agoWe can do better than "every C program is a valid C++ program, so C++ can’t lose" defensiveness. Look at the data tables published with that 2017 paper: https://sites.google.com/view/energy-efficiency-languages/re https://sites.google.com/view/energy-efficiency-languages/re... There's an order of magnitude difference between the times of the selected C and C++ programs, for one thing — regex-redux. That is enough to explain all of the difference between the C and C++ time measurements shown in "Table 4. Normalized global results". (Seems like an outlier which could have been excluded.)