2 ms·

I've also got remarkably good results pushing regular expressions to the max to perform "NER", while other more fancy (ML) solutions failed. So don't rule them

• chaxor 3y ago
  Perhaps... but If that's true, you're likely not doing what is typically referred to as NER in computational linguistics. Either that or yours going with a type of 'dictionary based' regex system, which gets typically high recall, low precision. Several years ago, when manual FE was a thing, this would be fed to a CRF model or some other Markov like model, perhaps even an LSTM prior to W2V, and be used as a feature - along with other things like 'ends in ly' etc. There's always tradeoffs. Regex is fast, linear CRFs are quite fast too. Simple LSTMs are fast-ish, as well as BERT-like systems, can offer a decent tradeoff in speed/performance. LLMs are much slower, and need some type of distill step by step to get anything very useful out (for task specific model). Ultimately, you're right in that regexes along with other older techniques should be understood and weighed for what is the optimal solution for the task.