3 ms·
Isn't this a problem deep learning could solve without huge amounts of difficulty in implementation? Or am I just imagining getting a list of misspellings of wo
by andy_ppp 6y ago
Isn't this a problem deep learning could solve without huge amounts of difficulty in implementation? Or am I just imagining getting a list of misspellings of words and phonetics for them is not intractable?
- zverok 6y agoThat's a huge topic, which I am planning to cover towards the end of the article series <s>please like and subscribe</s>, but in short: yes, my opinion is that spellchecking is actually a "machine learning problem in disguise", and most of existing dictionaries are more a roundabout way of storing something-not-unlike-models than analytical data. But ML approach will raise a question of data availability. What good your "deep learning OSS spellchecker" will do if there aren't good (and open) models for it which cover as much languages as existing Hunspell dictionaries do? And what if adding a bunch of new words requires laborous model retraining? It is not unsolvable, but non-trivial. I believe all the giants have something like this inside (I don't think spelling correction in Google search bar is handled with Hunspell, right?), but it is much harder to do as an open tool, ready to embedding into other software. There are a notable attempts, though: JamSpell for one (https://github.com/bakwc/JamSpell https://github.com/bakwc/JamSpell), which has an open "free" models, and more precise commercial ones; source code is open (maybe also only for using "simplistic" models, haven't dug deeper).