3 ms·
To be clear, you weren't bitten by Unicode but bitten by bad Unicode usages. Which are prevalent enough that any text processing pipeline has to be fuzzy enough
by lifthrasiir 2y ago
To be clear, you weren't bitten by Unicode but bitten by bad Unicode usages. Which are prevalent enough that any text processing pipeline has to be fuzzy enough to recognize them. I have seen, for example, many uses of archaic Hangul jamo ㆍ (U+318D) in place of middle dots (U+00B7) or bullets (U+2022), while middle dots and bullets themselves are often confused to each other.
- riffraff 2y agoWhy bad? This is the intended use for this character
- lifthrasiir 2y agoHyphen is a distinct (but of course very commonly confused) character from minus, which Unicode separately encodes U+2212. Though it is also possible that an OCR somehow produced a hyphen out of nowhere.
- keybored 2y agoHyphens in front of numbers is not the intended use of hyphen. The PDFs have mangled the symbols.