4 ms·
Yup. I have to manually detect and correct for all the possible ligatures in all possible unicode in my text to speech pre-processor scripts. I hate them.
by superkuh 8y ago
Yup. I have to manually detect and correct for all the possible ligatures in all possible unicode in my text to speech pre-processor scripts. I hate them.
- ahazred8ta 8y ago"this gives us efficient space-time trade-offs" :-(
- dotancohen 8y agoThose are HTML entities. Most modern programming languages come with tools to decode this, e.g. in python: text = urllib.parse.unquote(text)
- jwilk 8y agourllib.parse.unquote() is unrelated to HTML. It undoes URL-encoding: https://docs.python.org/3/library/urllib.parse.html#urllib.parse.unquote https://docs.python.org/3/library/urllib.parse.html#urllib.p... In Python ≥ 3.4, you can use html.unescape() to decode HTML entities: https://docs.python.org/3/library/html.html#html.unescape https://docs.python.org/3/library/html.html#html.unescape
- dotancohen 8y agoYou are 100% correct. I mixed the two encodings up. Thanks.
- dunham 8y agoIf you have a Unicode library available, you might try asking it to convert the text to NFKD or NFKC normalization form. This will take apart ligatures (the former will also take apart accented characters).