3 ms·

I see some of the comments here talk about the code being useless it terms of using it the same way the author of the paper has. Personally for a lot of the pap

by robryan 16y ago

I see some of the comments here talk about the code being useless it terms of using it the same way the author of the paper has. Personally for a lot of the papers I have read in NLP based stuff I'd feel more comfortable just having some source to look through to give a better idea of how it was done. Some papers are quiet abstract about their method and aren't really helpful past giving you basic ideas without some hard code to let you fully understand what they have done. In contrasting it to maths it would see stupid for a maths paper to describe a new maths proof without putting it forward in mathematical notation in the paper.

alextp 16y ago
I'm not sure I agree. In most NLP papers I've read the real meat of the paper is not in the code, but in a couple of equations that, with a lot of mostly-mechanical wrapping around, implementing, debugging, and plugging into well-understood parts should make sense and be a contribution in and of themselves. And for these other things, it might be good to see code a couple of times, but mostly at first, and to get up to speed in an area. I think a problem is that, if you force all papers to include source code, due to the fact that science builds upon itself, you'd find that the average paper length for an area should climb a bit (going back down when a paradigm shift happens, because then the tricks stop working, but climbing all over again), and most of it would just be repeats of what's already there. Comparing to maths, it's like asking every paper proving a theorem to prove their lemmas, even very basic ones: sure, it'd help a novice understand what's going on, but it'll hinder progress more often than it'd help it. There's a place for introductory writing and a place for stand-in-the-shoulders-of-giants writing, and papers are mostly of the last sort. For example, a couple of decades ago every paper that used a naive bayes classifier would derive the equations, describe feature selection, weighting, etc; today, most just say "I use a naive bayes classifier for this, that, and that" and move on. Likewise for SVMs---you don't want to see the full code for most papers that use that, since it's a mess of kernel caches and dual variables that mean nothing whatsoever to the problem at hand (but the algorithm won't work without it).
- robryan 16y ago
  I guess it depends on the level your at, it comes back to an article a while ago that to truly understand all the new research coming out you have to be actively contributing research of your own and interacting with others in your field. Your right about the classifier, not needing a mention now. My problem starting out of my honours thesis is that I would read something like that then have to go research the thing they have just mentioned in passing because their core audience knows all about it already. So I guess it has an aspect of knowing the best starting point for what you want to research as well.

by robryan 16y ago

alextp 16y ago

I'm not sure I agree. In most NLP papers I've read the real meat of the paper is not in the code, but in a couple of equations that, with a lot of mostly-mechanical wrapping around, implementing, debugging, and plugging into well-understood parts should make sense and be a contribution in and of themselves. And for these other things, it might be good to see code a couple of times, but mostly at first, and to get up to speed in an area. I think a problem is that, if you force all papers to include source code, due to the fact that science builds upon itself, you'd find that the average paper length for an area should climb a bit (going back down when a paradigm shift happens, because then the tricks stop working, but climbing all over again), and most of it would just be repeats of what's already there. Comparing to maths, it's like asking every paper proving a theorem to prove their lemmas, even very basic ones: sure, it'd help a novice understand what's going on, but it'll hinder progress more often than it'd help it. There's a place for introductory writing and a place for stand-in-the-shoulders-of-giants writing, and papers are mostly of the last sort. For example, a couple of decades ago every paper that used a naive bayes classifier would derive the equations, describe feature selection, weighting, etc; today, most just say "I use a naive bayes classifier for this, that, and that" and move on. Likewise for SVMs---you don't want to see the full code for most papers that use that, since it's a mess of kernel caches and dual variables that mean nothing whatsoever to the problem at hand (but the algorithm won't work without it).

robryan 16y ago
I guess it depends on the level your at, it comes back to an article a while ago that to truly understand all the new research coming out you have to be actively contributing research of your own and interacting with others in your field. Your right about the classifier, not needing a mention now. My problem starting out of my honours thesis is that I would read something like that then have to go research the thing they have just mentioned in passing because their core audience knows all about it already. So I guess it has an aspect of knowing the best starting point for what you want to research as well.