3 ms·
When 95% of the runtime is spent in the benchmarking infrastructure rather than the benchmarked code, interpreting even the intra-implementation results is hard
by jsnell 17y ago
When 95% of the runtime is spent in the benchmarking infrastructure rather than the benchmarked code, interpreting even the intra-implementation results is hard. Here's the results I get (benchmark, original, symbol-function lifted out of time-test). Times in milliseconds:
CASE-10 240 12
CASE-100 244 52
CASE-1000 632 432
NCASE-10 212 20
NCASE-100 220 24
SWITCH-10 276 80
SWITCH-100 272 76
SWITCH-1000 268 72
For example SWITCH looks a lot more reasonable for a small number of cases with the original benchmark than it does when the benchmarking overhead is reduced.
(Edit: try to reformat the table)
- gjm11 17y agoI'm assuming those are SBCL numbers, given who's posting them :-). So, it seems to me that these figures confirm that CASE gets expensive for large numbers of cases whereas SWITCH doesn't. (You're missing the NCASE-1000 entry. If it's also quite small then you're also confirming that NCASE doesn't get expensive for large numbers of cases.) Which I thought was the most interesting finding here. You've also confirmed that SWITCH is a bit more expensive than CASE and NCASE except when the former blows up. However. Geoff Wozniak's results seem to show (1) CASE-100 being more expensive than CASE-10, where you find it costing the same; and (2) CASE-1000 being more expensive than CASE-10 by a much larger factor than you've found. The curious thing here is that a constant extra bogus overhead from an expensive SYMBOL-FUNCTION implementation ought (I'd have thought) to bias his results in the opposite direction to that. Are you and he using different versions of SBCL, or something?