3 ms·

that benchmark is terrible (see the github for details). for k-nucleotides, the test that showed the biggest delta (c# vs java), i was able to improve the java

by nqzero 8y ago

that benchmark is terrible (see the github for details). for k-nucleotides, the test that showed the biggest delta (c# vs java), i was able to improve the java perf by 35%. but the test harness is so poorly done that i didn't bother running it against the other impls. the reality is that good java developers just don't care about this sort of thing (which may say something unfortunate about me) and the benchmark is bad https://github.com/nqzero/k-nucleotide https://github.com/nqzero/k-nucleotide

igouy 8y ago
> able to improve the java perf by 35% How much more memory did you use? > i didn't bother running it https://salsa.debian.org/benchmarksgame-team/benchmarksgame/blob/master/CONTRIBUTING.md https://salsa.debian.org/benchmarksgame-team/benchmarksgame/...
igouy 8y ago
What a terrible attempt at criticism — `nqzero` says it's terrible so it must be terrible! `nqzero` says "the test harness is difficult to configure and use" — but does not show any example of what is supposed to be difficult about using the bencher script. `nqzero` says "the tests are not representative of common programming tasks" — but does not show any example common programming tasks and does not show the tests are not representative. `nqzero` says "there's no attempt to account for JIT warmup, and many of the tasks are too short to ever warm up" — but admits that "it is plausible" warmup costs are amortised and comparison will show miniscule difference. `nqzero` says "maintainers are opinionated in terms of what code they'll allow, effectively choosing the winners" — but (again) does not show any example. `nqzero` says "doesn't appear to allow for jvm options to be included" — but seems not to have looked. `nqzero` says "the test cpu is from 2007 and is not necessarily representative of current cpus" — That at-least is true!

by nqzero 8y ago

igouy 8y ago

> able to improve the java perf by 35% How much more memory did you use? > i didn't bother running it https://salsa.debian.org/benchmarksgame-team/benchmarksgame/blob/master/CONTRIBUTING.md https://salsa.debian.org/benchmarksgame-team/benchmarksgame/...

igouy 8y ago

What a terrible attempt at criticism — `nqzero` says it's terrible so it must be terrible! `nqzero` says "the test harness is difficult to configure and use" — but does not show any example of what is supposed to be difficult about using the bencher script. `nqzero` says "the tests are not representative of common programming tasks" — but does not show any example common programming tasks and does not show the tests are not representative. `nqzero` says "there's no attempt to account for JIT warmup, and many of the tasks are too short to ever warm up" — but admits that "it is plausible" warmup costs are amortised and comparison will show miniscule difference. `nqzero` says "maintainers are opinionated in terms of what code they'll allow, effectively choosing the winners" — but (again) does not show any example. `nqzero` says "doesn't appear to allow for jvm options to be included" — but seems not to have looked. `nqzero` says "the test cpu is from 2007 and is not necessarily representative of current cpus" — That at-least is true!