4 ms·
I often wonder if there should be a suite of tests that deliberately dont use well-tuned algorithms, or optimal ones, as the basis for benchmarks. In the real
by kjellsbells 3y ago
I often wonder if there should be a suite of tests that deliberately dont use well-tuned algorithms, or optimal ones, as the basis for benchmarks.
In the real world, people dont choose optimal paths in code very often. Or what was once optimal becomes suboptimal because the context changes.
Supposing that a benchmark chose to do, say, bubblesort on an insanely large list. Would that lead to more insight around how the compiler would act in the real world? Or are the pitfalls so well understood that they are universally optimized away by the compiler's earliest stages or processing?
If I were selling, say, Azul or some other alternative JDK, I might make a point of noting that it does a superior job of dealing with the code as it exists today, rather than as one might wish it to be.
- codr7 3y agoI like to use naive recursive fibonacci to bootstrap performance testing of my interpreters. It's a pretty lousy algorithm, but it does a good job of stress testing instruction dispatch and function calls.
- hocuspocus 3y agoAnecdotally I've seen much bigger gains on my team's Spark jobs that use complex business logic (i.e. our own hand rolled algorithms in idiomatic Scala) compared to plain ETL jobs that use the vanilla Spark API, which is now very mature and optimized. GraalVM does a really impressive job with mediocre code :-)