3 ms·
I've done this before. Sadly - the problem is way harder. First - you get much better results by using language models, n-grams etc. to predict the likely hoo
by jomamaxx 10y ago
I've done this before.
Sadly - the problem is way harder.
First - you get much better results by using language models, n-grams etc. to predict the likely hood of words given previous words. That can be hard.
The really hard part comes down to language that people use.
Colloquialisms, proper names, and mixed-languages ... make this stuff really, really hard.
Getting it 'mostly right' is not hard. Getting it really good is very difficult and depends a lot on context.
'Srinivas' will not be in most people's dictionaries, but it's a common name in India. 'Le' and 'la' are words in french and some similar in Spanish - and a lot of writers jam these in all over the place. Over-correction and beta-errors become a huge problem.
It's a really interesting premise and difficult for Engineers because it's purely probabilistic there is no way to build a perfect spellchecker unless you can agree 100% on what 'language' is, precisely ... and trust me there is no agreement on that. Not even close.
- mrweasel 10y agoWriting spell checkers is also hard in language that e.g. doesn't have spaces between composite nouns. In essence it makes it possible to make up a perfectly valid new word. We expect the spell checker to be able to recognise that word, because it knows the two words that the composite is made of. Finnish is probably even worse because of it's conjugation.