2 ms·
Characters in Unicode can have more than one script property, so the question "is this text entirely Bengali/Devanagari" can be answered even though they share
by samatman 3y ago
Characters in Unicode can have more than one script property, so the question "is this text entirely Bengali/Devanagari" can be answered even though they share characters. But Unicode encodes scripts, not languages, and not shapes.
Many things we might want to do with strings require a locale property, which Unicode tried allowing as an inline representation, this was later deprecated. I'm not convinced that was the correct decision, but it is what it is. If you want to properly handle Turkish casing or Swedish collation, you have to know that the text you're working with is Turkish or Swedish, no way around it.