4 ms·
long story short. Average is a poor measure of non linear phenomenon like the measure of a performance for a task . Histogram are better to see the repartition
by SFJulie 10y ago
long story short. Average is a poor measure of non linear phenomenon like the measure of a performance for a task .
Histogram are better to see the repartitions of values, and can hint you on a n-modal behaviour. (a «mode» when everything goes well, and a degraded one for instance which can hint on a potential transition of behavior).
I guess the next step is to say this is even more important to know the evolution over time and announce that the equivalent of spectrogram (3D mapped on 2D plus colour) is a good idea to known towards which mode you are tending the more. [EDIT: which can also more significantly graphed as spectrum analysis, colour encoding the amplitude of measure for a given frequency, and frequency being 1/Period thus making sure that slow processes may appear between 0 and average]
A shortcut is to hire of of the many jobless guys with a master in science that actually have experience in labs.
Beware that for a reason that is beyond my reach, people with background in math, management, literature have problems with scientific concepts that has been discovered after 1800 unless it is packaged like magic.
- static_noise 10y agoReplace the average (mean) with the median and you will have better results in almost all cases. When the data is good the result ist virtually identical. When there are outliers, the median just ignores them unless there lots of them. When the data is that bad, that the median doesn't work anymore you need to up your statistics quite a few notches.
- SFJulie 10y agoWell the median of measure that are only aggregated by average (RRD, carbon ...) is ... biased. Hence the reason I made my own backend for making very high precision measurement using a percentile filter (10%, 50%, 90% and average for the fun). It was fun to watch linux «monotonic» clock not being monotonic.
- omginternets 10y agoIn psychophysics we either use the median for reaction times (Gamma distribution) or we normalize with something like the logarithm. I think some of the techniques might be applicable to system performance analysis, so I submit the following introductory publication, in case any are interested [0]. [0] http://opensiuc.lib.siu.edu/cgi/viewcontent.cgi?article=1077&context=tpr http://opensiuc.lib.siu.edu/cgi/viewcontent.cgi?article=1077...
- gnufx 10y agoReplacing a single number with a different one hardly helps. The distribution matters, specifically the outliers in performance measurements.
- gnufx 10y agoI'd like to agree about scientific experience, but it's tragic how much trouble many professional scientists, or ex-scientists, seem to have with anything outside their specialism, like quantitative analysis of their computational techniques. When did you last see error bars on a benchmark, even?