4 ms·
I disagree strongly. This is the same difference as Linux vs Minix. Minix didn't want anything added as it was said to be for educational purposes even though t
by Smerity 11y ago
I disagree strongly. This is the same difference as Linux vs Minix. Minix didn't want anything added as it was said to be for educational purposes even though that didn't reflect the use cases.
The author points out that whilst the stated aim of NLTK is for education, it's used for far more than that in industry and academia. You'll see it used in papers, you'll see it as the basis of real world projects, etc. This presents a problem if the aims of the project are different to the how the project is used.
The biggest flag for me is, as pointed out in the blog post, when the project doesn't even know how the part of speech (POS) model was trained[1]. That means a lack of reproducibility[2]. Given POS tags are the first level of almost any NLP task, this is strongly troubling.
[1]: "Where did the NLTK pos_tag model come from?" https://github.com/nltk/nltk/issues/1063 https://github.com/nltk/nltk/issues/1063
[2]: The POS tags from NLTK are used for many papers and research - see https://scholar.google.com/scholar?as_ylo=2015&q=nltk&hl=en&as_sdt=0,5 https://scholar.google.com/scholar?as_ylo=2015&q=nltk&hl=en&...
- wisty 11y agoNLTK in research is probably mostly used as glue, its corpus interface, and its standard wrappers to common libraries. Everyone using it for research will do something like "I used data from NLTK, pushed it through my custom parser, and here's how it compares to the wrapped parsers that NLTK also interfaces with". That's why the maintainer said, basically, "nope, we only implement the standard algorithms". Most of the researchers want to get standard data, and compare their new algorithm to the standard algorithms that every other researcher uses.
- syllogism 11y agoMy blog post does explain the standard algorithms! Just, the ones that are standard now. That reply actually made no sense. I guess the maintainer thought that my post described novel research. It didn't. There's now a ticket to implement the dynamic oracle, as I recommended: https://github.com/nltk/nltk/issues/905 https://github.com/nltk/nltk/issues/905