3 ms·
Shouldn't there be max along with min and avg? Std dev is higher in many cases. Could there be an issue where max (worst case) has worse performance in Unlade
by tl 17y ago
Shouldn't there be max along with min and avg? Std dev is higher in many cases. Could there be an issue where max (worst case) has worse performance in Unladen Swallow?
- kragen 17y agoI don't know if this is the case here, but often in benchmarking, "min" is the real answer of how fast something is going, and any time on top of that is some kind of overhead that you weren't trying to measure — some other process was taking up CPU time, you had to page in part of libc, crap like that. "avg" serves to tell you whether that overhead was reasonably small, or whether you need to run your tests again (on a quieter machine, say) to get a smaller "min". This depends on the assumption that what you're measuring is actually a deterministic process. There are cases where it's not; as an example, the Self papers have all kinds of moaning about the measurement imprecision caused by physically-mapped direct-mapped caches without OS page coloring, but with more than one page of cache, on old SPARCs. For any particular logical→physical mapping, your measurements would be reproducible, but when it changed (if you got paged out and back in, or if you restarted the virtual machine in a new process) you would get different results.
- imbaczek 17y agomax is useless in benchmarking; there's no upper limit on how long any computation can take.
- kragen 17y agoIf that were true, real-time software would be impossible. (Unless you're talking about the risk of your hardware malfunctioning, I guess.) Practically speaking, max can be useful.