Yes, you can get 97-98%, but only when evaluating on data from the same corpus as you trained on. If you evaluate on data from a different corpus, you immediate

Yes, you can get 97-98%, but only when evaluating on data from the same corpus as you trained on. If you evaluate on data from a different corpus, you immediately get a pretty big drop in performance. Thus one person in the field I've talked to even went so far as to say that competing in this part of the field (state-of-the-art performance, basically) is fundamentally a question of "who is the best at overfitting". There's basically no part of NLP that's a solved problem. Even something as superficially simple as segmenting running text into sentences and tokens is decidedly non-trivial.