4 ms·
It'd be awesome if they provided a comparison of other tools on the same hardware like, "on an AWS M4.xlarge instance we were able to achieve 160m/ops/sec when
by haney 8y ago
It'd be awesome if they provided a comparison of other tools on the same hardware like, "on an AWS M4.xlarge instance we were able to achieve 160m/ops/sec when the dataset fit into memory where as redis only did X"
Lacking that I agree it's a pretty meaningless stat.
- mgraczyk 8y agoCheck out the linked paper for a detailed performance comparison.
- ddorian43 8y agoDon't do benchmarks on shared-host.
- ebikelaw 8y agoIf that’s your production environment then that’s where you should run them.
- dymk 8y agoThat's not why you don't run them on a shared host. It's because every other tenant on the machine is going to make run of the same benchmark unpredictable, and it will likely vary greatly through the day. Even taking multiple runs of each benchmark isn't sufficient, because you don't know the usage patterns of other tenants.
- pvg 8y agoYou're making it sound like performance on such hosts in unknowable which isn't really accurate. 'Multiple runs of each benchmark' is vague enough to be potentially insufficient in just about any environment to boot.
- andrewstuart 8y agoNo benchmark means anything on an EC2 shared instance (or probably any other cloud instance) because you don't know what else is running on the machine.
- haney 8y agoWhat about running the benchmark multiple times on instances of the same type? I get that it would be noisy but lots of workloads run on shared instances so it’s a useful measuring stick in that way.
- deleted 8y ago[deleted]