4 ms·

I hate this genre of post that basically follows the line: "I went to <established project> and attempted to educate them. When they didn't listen I went and b

by ben336 11y ago

I hate this genre of post that basically follows the line: "I went to <established project> and attempted to educate them. When they didn't listen I went and built something better. Now its clear they should have listened to me, and you should all abandon their software" Almost always the scope of the new project is much smaller, different or much less mature than the project being bashed. Open source projects are not required to make changes to please any arbitrary user that wants to make changes, even if it's to bring technical improevements. In NLTK's case, they have a whole book written around their project. Presumably significant changes to project structure and function would mean heavy documentation/writing work, and might not fit the goals of their project. Bashing them as a result just shows a complete lack of understanding of how/why people write and maintain software.

Smerity 11y ago
I disagree strongly. This is the same difference as Linux vs Minix. Minix didn't want anything added as it was said to be for educational purposes even though that didn't reflect the use cases. The author points out that whilst the stated aim of NLTK is for education, it's used for far more than that in industry and academia. You'll see it used in papers, you'll see it as the basis of real world projects, etc. This presents a problem if the aims of the project are different to the how the project is used. The biggest flag for me is, as pointed out in the blog post, when the project doesn't even know how the part of speech (POS) model was trained[1]. That means a lack of reproducibility[2]. Given POS tags are the first level of almost any NLP task, this is strongly troubling. [1]: "Where did the NLTK pos_tag model come from?" https://github.com/nltk/nltk/issues/1063 https://github.com/nltk/nltk/issues/1063 [2]: The POS tags from NLTK are used for many papers and research - see https://scholar.google.com/scholar?as_ylo=2015&q=nltk&hl=en&as_sdt=0,5 https://scholar.google.com/scholar?as_ylo=2015&q=nltk&hl=en&...
- wisty 11y ago
  NLTK in research is probably mostly used as glue, its corpus interface, and its standard wrappers to common libraries. Everyone using it for research will do something like "I used data from NLTK, pushed it through my custom parser, and here's how it compares to the wrapped parsers that NLTK also interfaces with". That's why the maintainer said, basically, "nope, we only implement the standard algorithms". Most of the researchers want to get standard data, and compare their new algorithm to the standard algorithms that every other researcher uses.
  - syllogism 11y ago
    My blog post does explain the standard algorithms! Just, the ones that are standard now. That reply actually made no sense. I guess the maintainer thought that my post described novel research. It didn't. There's now a ticket to implement the dynamic oracle, as I recommended: https://github.com/nltk/nltk/issues/905 https://github.com/nltk/nltk/issues/905
syllogism 11y ago
Surely there's a place for technical criticism? If you think I've over-stepped that and made comments which reflect personally on the NLTK maintainers I'll apologise and revise. That wasn't my intention. For a long time we've been in a situation where everyone experienced in NLP knows, but nobody says, that you should not use NLTK. That's not a healthy situation. Having a book as baggage explains but does not excuse how out-of-date and low-quality NLTK's software is. The bottom-line is that in 2015 you can't go to NLTK and: a) Learn how modern NLP is done; b) Access a convenient toolkit of reliable, basic NLP components. That's the mission statement, right? Well I think they don't achieve that, and that they do a disservice by pretending they do.
- danso 11y ago
  I have to disagree with the GP and agree with you...NLTK is a deservedly popular library and an attack on it is going to seem like an attack on its many hard-working volunteers...but good on you for putting the time into building an open source project that follows your vision of an alternative...that's far from being just a griper, and it's often the way that software in a field improves. I'm definitely going to try out spacy.
svckr 11y ago
OP did not "go to <established project> and attemt to educate them". He wrote spaCy after looking at NLTK and noticed a difference in philosophy that could not be overcome by talking it over. The NLTK project has grown a community that values choice and, well, history I guess? The author knows that and does not want to be part of it and that is totally fine. He seems to understand the project goals, he just doesn't agree with them, so started from scratch and wrote a post about it, presenting a different way to do things. All of this is to say, I agree with some points you made but IMO this post is not part of the genre you mentioned.

by ben336 11y ago

Smerity 11y ago

I disagree strongly. This is the same difference as Linux vs Minix. Minix didn't want anything added as it was said to be for educational purposes even though that didn't reflect the use cases. The author points out that whilst the stated aim of NLTK is for education, it's used for far more than that in industry and academia. You'll see it used in papers, you'll see it as the basis of real world projects, etc. This presents a problem if the aims of the project are different to the how the project is used. The biggest flag for me is, as pointed out in the blog post, when the project doesn't even know how the part of speech (POS) model was trained[1]. That means a lack of reproducibility[2]. Given POS tags are the first level of almost any NLP task, this is strongly troubling. [1]: "Where did the NLTK pos_tag model come from?" https://github.com/nltk/nltk/issues/1063 https://github.com/nltk/nltk/issues/1063 [2]: The POS tags from NLTK are used for many papers and research - see https://scholar.google.com/scholar?as_ylo=2015&q=nltk&hl=en&as_sdt=0,5 https://scholar.google.com/scholar?as_ylo=2015&q=nltk&hl=en&...

wisty 11y ago
NLTK in research is probably mostly used as glue, its corpus interface, and its standard wrappers to common libraries. Everyone using it for research will do something like "I used data from NLTK, pushed it through my custom parser, and here's how it compares to the wrapped parsers that NLTK also interfaces with". That's why the maintainer said, basically, "nope, we only implement the standard algorithms". Most of the researchers want to get standard data, and compare their new algorithm to the standard algorithms that every other researcher uses.
- syllogism 11y ago
  My blog post does explain the standard algorithms! Just, the ones that are standard now. That reply actually made no sense. I guess the maintainer thought that my post described novel research. It didn't. There's now a ticket to implement the dynamic oracle, as I recommended: https://github.com/nltk/nltk/issues/905 https://github.com/nltk/nltk/issues/905

syllogism 11y ago

Surely there's a place for technical criticism? If you think I've over-stepped that and made comments which reflect personally on the NLTK maintainers I'll apologise and revise. That wasn't my intention. For a long time we've been in a situation where everyone experienced in NLP knows, but nobody says, that you should not use NLTK. That's not a healthy situation. Having a book as baggage explains but does not excuse how out-of-date and low-quality NLTK's software is. The bottom-line is that in 2015 you can't go to NLTK and: a) Learn how modern NLP is done; b) Access a convenient toolkit of reliable, basic NLP components. That's the mission statement, right? Well I think they don't achieve that, and that they do a disservice by pretending they do.

danso 11y ago
I have to disagree with the GP and agree with you...NLTK is a deservedly popular library and an attack on it is going to seem like an attack on its many hard-working volunteers...but good on you for putting the time into building an open source project that follows your vision of an alternative...that's far from being just a griper, and it's often the way that software in a field improves. I'm definitely going to try out spacy.

svckr 11y ago

OP did not "go to <established project> and attemt to educate them". He wrote spaCy after looking at NLTK and noticed a difference in philosophy that could not be overcome by talking it over. The NLTK project has grown a community that values choice and, well, history I guess? The author knows that and does not want to be part of it and that is totally fine. He seems to understand the project goals, he just doesn't agree with them, so started from scratch and wrote a post about it, presenting a different way to do things. All of this is to say, I agree with some points you made but IMO this post is not part of the genre you mentioned.