3 ms·
Actually I tend to disagree. Yeah, you can do that for a hobby project. I'm working in OCR / ICR here. We're often depending on dictionaries to fix recognition
by darklajid 14y ago
Actually I tend to disagree. Yeah, you can do that for a hobby project.
I'm working in OCR / ICR here. We're often depending on dictionaries to fix recognition errors. So you want to correct names for example, by creating a more or less complete list of all names (we're assuming you know all the names in advance, for example for customers). The dictionary is huge.
"Looking up a word in this hash table dictionary is a trivial expression, one built into the language. And that's it."
Well, no. A spellchecker that just answers "I don't know that word" would be crap. You need to find similar words. Which isn't part of any standard library I am familiar with.
In addition it's still a challenge to find all candidates that have a levenshtein distance (or LD or .. whatever metric you like) of N from the input, for N > 2 or 3.
Think address. You have a list of all streets, which tend to be rather long (Germany: "Strasse" = street, the pattern "<name> Strasse" is common, let's assume you expect 12-14+ chars). The longer the expected input, the higher I'd set the tolerance for errors. Finding all matches with 3+ substitutions/deletions/additions on a large dataset, in a small amount of time is still interesting and a challenge.
In fact, we partner with one company that provides nothing but exactly this, as their single (well-known, respected) product.
- josephlord 14y agoYou should probably also go Bayesian on word frequencies for the suggestion ordering. Maybe also on error types (missing letter, repeated letter, nearby (on standard keyboard) letter.
- spc476 14y agoI used soundex and metaphone to correct spelling mistakes for a project. My assumption: that most spelling mistakes are people trying to sound out the spelling and not simple typos. For the limited case I was solving (mostly Biblical names) it worked quite well.
- darklajid 14y agoYour comment and the reply to it are good if we're talking spell checker software for stuff people type. What I have to deal with as well are recognition errors. From trivial replacements (I/l/1,C/G,i/j,P/R etc.) to 'engine said that there's some character here according to the character segmentation results, but it has no damn clue what character it could be' that's a different look at the same problem. Also, for typed content I see quite a lot of shortcuts. SwiftKey for Android, nice as it is, seems to be too braindead to correct words that _start_ with the wrong character (if I type vharacter, with just one single slip at the start, it fails miserably and looks baaad imho). Assumptions like 'just check adjacent characters on the keyboard (special points for assuming a keyboard layout)' or 'ah, the user is certainly starting the jnput correctly and just might misspress a letter in the middle of a word' are nice models to make the developer's life easier, but they don't work for a good number of cases as well..