3 ms·
I wrote such a library for Python here: https://github.com/wanderingstan/Confusables https://github.com/wanderingstan/Confusables My use case was to thwart spa
by wanderingstan 3y ago
I wrote such a library for Python here: https://github.com/wanderingstan/Confusables https://github.com/wanderingstan/Confusables
My use case was to thwart spammers in our company’s channels, but I suppose it could be used to also normalize accent encoding issues.
Basically converts a phrase into a regular expression matching confusables.
E.g. "ℍ℮1೦" would match "Hello"
- _nalply 3y agoInteresting. What would you think about this approach: reduce each character to a standard form which is the same for all characters in the same confusable group? Then match all search input to this standard form. This means "ℍ℮1l೦" is converted to "Hello" before searching, for example.
- wanderingstan 3y agoIt’s been a long time since I wrote this, but I think the issue with that approach is the possibility of one character being confusable with more than one letter. I.e. there may not be a single correct form to reduce to.