3 ms·

Thanks, but I disagree with the approach of only showing storage benchmarks with disabled caches. Production workloads will encounter variance between the provi

Thanks, but I disagree with the approach of only showing storage benchmarks with disabled caches. Production workloads will encounter variance between the providers thanks to different caches and behaviors of handling direct I/O. I'd include direct I/O results _with_ cached results, so that I wasn't misleading my customers. I know what I'm suggesting is not the current norm for cloud evaluations. And I believe the current norm is wrong. The more important question is how the benchmarks were analyzed -- what other tools were run to confirm that they measured what they were supposed to?

jread 12y ago
Good point - user experience may include cached and non-cached I/O so it would be beneficial to include both in this type of analysis. The benchmarks binaries, configurations and runtime settings were generally consistent for instance types of the same size across services, but we didn't verify efficacy of the benchmarks as they ran.