5 ms·
I was initially surprised at their benchmark results, until I read more closely and noticed that they were benchmarking on a cloud VM. I wish people wouldn't do
by jfindley 6y ago
I was initially surprised at their benchmark results, until I read more closely and noticed that they were benchmarking on a cloud VM. I wish people wouldn't do this for this form of test - it adds so many variables it's hard to take their benchmark too seriously.
- eyalitki 6y agoWe executed the benchmark suite multiple times, and results were consistent. It was way more stable than the same execution on a PC with multiple programs. Anyway, the maintainers of glibc performed the same checks and confirmed our measurements.
- kees99 6y agoThumbs up with reaching out to upstream to mainline it. Did you talk to musl team as well?
- eyalitki 6y agoNot yet. Working with each open-source demanded a lot of effort, so we were hoping that integrating it into a few dominant libraries will attract the attention of the rest. We will happily assist them if they need/want our help.
- boomer_joe 6y agoThe musl mailing list is very active, and communication with the maintainers is generally effortless. It would be interesting to see how this plays out. They are also in the process of rewriting their malloc implementation. Thank you for your work!
- rurban 6y agoAnd musl is just working in merging its new mallocng.
- shard972 6y agoAm i reading correctly that he tested on a single vCPU? Isn't that half a physical core?
- arielb1 6y agoNot that surprising: CPUs can perform scalar in parallel with computation, so I would expect a few scalar ops to be free in any code that is not scalar-bound.
- deleted 6y ago[deleted]
- the_duke 6y agoIn my experience, cloud VMs are often a more consistent benchmarking platform than your personal machine. A desktop environment and consumer-grade hardware add a lot of noise. Ideally you'd always benchmark on completely idle dedicated servers, but those are rarely available.
- devit 6y agoYou can just run it on your local machine and dedicate some of the cores to it with a tool like "cset shield". If you _really_ want to avoid any contention, then build the benchmark as a Linux kernel module and call stop_machine(), which will prevent anything else including interrupt handlers from running.