3 ms·
Nice article, and thanks for coming by! What are your thought on how to treat performance benchmarks, or really any claims that one algorithm is "better" than
by nkurz 13y ago
Nice article, and thanks for coming by!
What are your thought on how to treat performance benchmarks, or really any claims that one algorithm is "better" than another? Since these are often extremely hardware specific, I've been wondering if instead of a VM it makes sense to offer a full bootable image.
Are there virtual image formats that can be either run within a VM or copied to a USB stick and booted?
- turingfan 13y agoI've thought about this quite a lot because people often ask me. My main comment is that for one algorithm to be judged against another, the more different environments it is tested in the better, and we can get a deeper understanding of its performance profile. If it's always better then that's clear. If it is sometimes better and sometimes worse, either it's not really better, or there's some dependency on the hardware or other environment: but that is an interesting result in itself. However there is always a place for very hardware specific claims. E.g. "For this chip/motherboard combination this flag is better", and for that we might always have a problem. Interesting thought about the either VM or booting. Somebody suggested to me making live images (as in live dvds) which would serve this purpose.