5 ms·
It's subjective because it can't be used as a source of truth. Of course "I measured X and my results were Y using methodology Z" can be a statement of fact but
by fnord123 5y ago
It's subjective because it can't be used as a source of truth. Of course "I measured X and my results were Y using methodology Z" can be a statement of fact but X and Z are where the subjectivity lie.
For example, benchmark game allows for warmups and so does awfy. This favors jits because it allows them to warm up when they would otherwise be slower. This might give the mistaken impression that java is a great choice for command line tools due to the performance characteristics.
In contrast, most benchmarks I've seen don't use profiler guided optimizations for C or C++. Hence the subjectivity.
And the claim of only wanting idiomatic code in awfy. This is, of course, subjective as well.
- kaba0 5y agoLast I checked benchmark games didn’t care about warmups.
- fnord123 5y agoYou could be correct. The point doesn't rely on what a specific benchmark collection in particular does but that there is an open discussion on what is appropriate in the context of what people using the results find important.
- deleted 5y ago[deleted]