5 ms·
As heavyset_go mentioned casefolding is your friend here, but you'll also likely want to do some unidecoding too depending on your use case. With that lens, goi
by OisinMoran 5y ago
As heavyset_go mentioned casefolding is your friend here, but you'll also likely want to do some unidecoding too depending on your use case. With that lens, going from one to two characters is pretty tame as there are numerous examples of single characters that unidecode to eight! And also ones that go to zero.
We've done some pretty fun work at Inscribe ensuring things like this work across multiple languages, in particular for matching names. So we can now verify that a document does indeed belong to "Vladimir" even if the name on the document is "Владимир". The trickier part is not simply matching those, but returning the original raw result, as for that you need to keep track of how your normalization has changed the length of things.
If you're interested in this kind of thing, shoot me a mail at oisin@inscribe.ai. We're growing rapidly and hiring to match.