4 ms·
How does it actually compare to fasttext [1] in performance. Building an interface to that in GO shouldn’t be too complicated. The claim that all language ident
by doubtfuluser 5y ago
How does it actually compare to fasttext [1] in performance.
Building an interface to that in GO shouldn’t be too complicated.
The claim that all language identification (lid) relies on ngrams is bold and there has been a switch to pure neural network based approaches.
[1] https://fasttext.cc/docs/en/language-identification.html https://fasttext.cc/docs/en/language-identification.html
- alexott 5y agoYou can try with dataset that I’ve used to evaluate fasttext against different language detectors - it’s linked to this blog post: https://alexott.blogspot.com/2017/10/evaluating-fasttexts-models-for.html https://alexott.blogspot.com/2017/10/evaluating-fasttexts-mo... I’ll try to find time to do it myself, but most probably only tomorrow
- pemistahl 5y agoI've compared the Python implementation of Lingua with fasttext. Lingua performs clearly better. Look here: https://github.com/pemistahl/lingua-py#4-how-good-is-it https://github.com/pemistahl/lingua-py#4-how-good-is-it