3 ms·
In the context of a superoptimizer benchmarks dont make sense really. You don't know what it's going to find.
by xxpor 6y ago
In the context of a superoptimizer benchmarks dont make sense really. You don't know what it's going to find.
- ianhanschen 6y agoFor this sort of benchmarking you don’t want to know what it should find, that would be a functional test and not a benchmark. Compiling and profiling reasonably complex apps against baseline would be enough.
- JJJollyjim 6y agoI'd also like to see motivating examples of specific things it does find (that LLVM doesn't), even if that's not a representative benchmark
- xxpor 6y agoI think I get what you're saying, I think people would read too much into them though. The results are very dependent on the type of program you have. I wouldn't be surprised if there were 10s of percentage point differences in benchmarks between various examples.
- HelloNurse 6y agoThat's the point. Benchmarks would show what code is already squeezed dry by basic LLVM tools and what code could benefit from superoptimization, which (as other comments point out) is the only reasonable basis for deciding to give Souper a try.