2 ms·
Here's something you may not know, the *-insane dictionaries, which are giant, are functions of OCR output and are known to contain lots of errors. I found a f
by kristopolous 3y ago
Here's something you may not know, the *-insane dictionaries, which are giant, are functions of OCR output and are known to contain lots of errors.
I found a few earlier this year and I was going to file a bug so I did some research to find out this is a known and expected behavior.
If the computer say reads stubborn as stubbum, the smaller dictionaries are the ones that have cross checked and filtered those out. The insane ones do not. It's a good name. "Lack of sanity checks"
Here's an example word I found, "suabilities". You'll find it only on wordlist sites that used this wordlist and I guess, now here.
- schoen 3y agoI've used the insane dictionaries a number of times for puzzle stuff and I never knew that they were derived from OCR output. Thanks for mentioning that!
- colinchartier 3y agoReminds me of the ghost Unicode character saga: https://www.dampfkraft.com/ghost-characters.html https://www.dampfkraft.com/ghost-characters.html
- kristopolous 3y agojust saw this. I've got no idea how kanji ocr works but I do know enough japanese to know what most of those characters are attempting to refer to, my penmanship has certainly been that bad. I still don't understand how it would make its way into the standard unless that part wasn't written by someone who is competent in japanese. I wonder how often that happens - surely there's tons of people dealing with japanese text who can't read it and just use diligence to make sure the "letters are the same"