3 ms·
> You seem to have found the benchmarks game "original programs" in the project tarball. > People measure their own programs with bencher, not the benchmarks g
by spion 14y ago
> You seem to have found the benchmarks game "original programs" in the project tarball.
> People measure their own programs with bencher, not the benchmarks game programs -- it's not dependent on the benchmarks game programs.
It is not, but if you sincerely want to make it easy for people to reproduce all or parts of the benchmark, you should at least document where they can find all the settings of the original.
>>I had to remove everything in tmp/* because...<<
#5.3 #5.4
Fair enough, I missed those two points in the README. A weaker point still remains though: its not very user friendly or obvious, like say a --force flag (or something in the spirit of make's --always-make).
These things may not seem important, but I think they are when distributing software with the intent of someone else running it.
>>testrange from your CSV files, as the ones used on the website aren't distributed anywhere<<
nanobench/makefiles/u64q.ini
nanobench/makefiles/u64.ini
nanobench/makefiles/u32q.ini
nanobench/makefiles/u32.ini
Ah in nanobench. Well that was definitely not easy or obvious, was it? My fault for not using find on the directory.
And how about that suggestion of splitting the configuration file to two separate files: program configuration and environment configuration? It would make it easier to run the benchmarks on the same set of programs with the same parameters, except in a differently configured environment. (the set of language implementations and their build and run commands would be a part of the environment configuration)
> You're comments are useful because they show what someone might become confused about.
That was exactly my point - to prove that your "easily" claim is a stretch. The README does not cover everything, especially not the parts needed for someone to bench the programs found on the website. Which is understandable since its a README for the bencher program.
Perhaps a separate README for the entire archive (documenting where the original configurations and programs are) will fix most of these confusion issues. More descriptive directory names would also help (e.g. "game-programs" and "game-configurations")
And if I seem so negative, its because I used to love the benchmarks game and felt that every change you've made lately significantly subtracted from the game's fun factor of it while not significantly adding anything.
For example, that recent removal of the (not very useful, but still extremely fun!) combined language comparison table. Why remove things? Its sufficient to simply warn the user. If they ignore the warning and take those numbers seriously, that is their own fault. Why should they be ruining the fun for everyone else?
Oh well...
- igouy 14y ago>>to prove that your "easily" claim is a stretch<< Yes, you were trying to prove that -- you were finding fault instead of finding how to make things work. Other people found how to make things work, because that's what they were trying to do. >>a separate README for the entire archive<< Already done. >>Its sufficient to simply warn the user.<< Was it sufficient to list Willingness to read the README as a requirement? Apparently not.
- spion 14y ago>>to prove that your "easily" claim is a stretch<< > Yes, you were trying to prove that -- you were finding fault instead of finding how to make things work. > Other people found how to make things work, because that's what they were trying to do. Not really. The first time I downloaded the zip (about a year ago) I sincerely tried to make it work and did read the readme. Spent about 1 hour then gave up. This time I simply retraced those exact same steps with the intent of showing the current deficiencies. >>Its sufficient to simply warn the user.<< > Was it sufficient to list > Willingness to read the README > as a requirement? Apparently not. Yes it is. Its my fault for not reading the README completely. (but to be fair, all directories of the distribution were not quite covered) You don't have to remove the complete zip archive then add a quiz to the website testing if the user read the README before allowing them to download the complete archive. If that doesn't work either, what then? Will you completely remove the download link to the archive, ruining it for those willing to read the README? The way I see it, you've done your part, the rest is up to the user. This is where I strongly disagree with your approach - and this is why I want to make a new benchmarks game.
- igouy 14y ago> I downloaded the zip (about a year ago) Do you mean the project tarball or do you mean the bencher zip? > I sincerely tried to make it work Did you ask for help? > This time I simply retraced those exact same steps This time you've told everyone such-and-such isn't in the README when it is; such-and-such aren't written anywhere when they are; such-and-such aren't distributed anywhere when they are. None of that stuff changed in the last year. >>and this is why I want to make a new benchmarks game<< So get the tarball tomorrow, read the README, find out what doesn't work for you and fix-it (if you're missing python-gtop install it), make measurements and publish them. Easy.