6 ms·
I'm as annoyed by this as the next person, but to be fair opening the code isn't necessary for reproduction, it just makes it easier. If their paper is properl
by dagw 10y ago
I'm as annoyed by this as the next person, but to be fair opening the code isn't necessary for reproduction, it just makes it easier.
If their paper is properly written there should be enough information there to recreate what they've done. If there isn't that and they refuse to provide that information when asked then you have every right to complain.
- mattkrause 10y agoIf… There is a really interesting case study in the Collins Parser. Starting around 1999, Michael Collins published some really exciting results on parsing (interpreting the grammatical structure of nature language). However, people could not replicate them--a "clean room" implementation of Collins' models didn't work nearly as well as the paper claimed it should. Dan Bickel identified a number of apparently trivial implementation decisions that, when taken together accounted for Collins' models improved performance. There's a nice tech report describing this process here: http://repository.upenn.edu/cgi/viewcontent.cgi?article=1026&context=cis_reports http://repository.upenn.edu/cgi/viewcontent.cgi?article=1026...
- YeGoblynQueenne 10y agoKind of amazing that a single guy put in the effort to rev-eng such a complex implementation. Brilliant paper that, thanks :)