3 ms·
Hi, this is Ian Gent, author of the Recomputation Manifesto. Many thanks for all the comments. If anybody wants to get in touch to work in any way on recomput
by turingfan 13y ago
Hi, this is Ian Gent, author of the Recomputation Manifesto.
Many thanks for all the comments.
If anybody wants to get in touch to work in any way on recomputation, please do! You can find me very easily on google.
Special thanks to @lifebeyondfife, I worked out who you are and you were a pleasure to supervise too. Hope all is going well.
- nkurz 13y agoNice article, and thanks for coming by! What are your thought on how to treat performance benchmarks, or really any claims that one algorithm is "better" than another? Since these are often extremely hardware specific, I've been wondering if instead of a VM it makes sense to offer a full bootable image. Are there virtual image formats that can be either run within a VM or copied to a USB stick and booted?
- turingfan 13y agoI've thought about this quite a lot because people often ask me. My main comment is that for one algorithm to be judged against another, the more different environments it is tested in the better, and we can get a deeper understanding of its performance profile. If it's always better then that's clear. If it is sometimes better and sometimes worse, either it's not really better, or there's some dependency on the hardware or other environment: but that is an interesting result in itself. However there is always a place for very hardware specific claims. E.g. "For this chip/motherboard combination this flag is better", and for that we might always have a problem. Interesting thought about the either VM or booting. Somebody suggested to me making live images (as in live dvds) which would serve this purpose.
- cmdkeen 13y agoHi Ian, Without being too cheeky - what's happened to TAILOR? The Minion project you work on is a classic example of this problem - an important tool to actually encode problems for the solver has disappeared. In this case it seems it is literally a case of a disappearing post doc and her site disappearing...
- turingfan 13y agoThis is not being cheeky at all. A very fair point. In general terms one reason I am keen on recomputation is because I can see how bad we have been about things in the past, and I want us to do better. To answer your concrete question: Tailor has been replaced by SavileRow which serves the same purpose and which is being maintained. http://savilerow.cs.st-andrews.ac.uk/ http://savilerow.cs.st-andrews.ac.uk/
- cmdkeen 13y agoHuzzah - cracking name as well. I really am going to have to get round to digging out my dissertation work and having a play.
- edwintorok 13y agoI don't know if you're aware of this initiative regarding reproducible research in networking: https://reproducingnetworkresearch.wordpress.com/about/ https://reproducingnetworkresearch.wordpress.com/about/ It seems to be about providing full code to reproduce experiments using MiniNet.
- Nimi 13y agoIIUC, there's a project aiming to solve some of the problems with code dependencies, packaging the data with the code etc: http://www.pgbovine.net/cde.html http://www.pgbovine.net/cde.html (the author hangs occasionally here on HN, but thought I'll plug this anyway, in case he misses this post)
- turingfan 13y agoYes CDE looks very good though I haven't investigated in depth. The main limitation I think is that you need to have a recentish linux kernel installed. But if somebody distributes a CDE package then it will probably be easy to run it.
- Nimi 13y agoIf I may suggest, you should contact the author - he wrote he takes pride in having real-world users, and adapting the tool to their needs.