5 ms·
It’s only complex if your language does not support Unicode properly. In something like rust or python you would just ask for the first character rather than th
by Gigachad 5y ago
It’s only complex if your language does not support Unicode properly. In something like rust or python you would just ask for the first character rather than the first byte. And the language will calculate what that is for you.
- chmod600 5y agoI'm not an expert, but your statement seems confusing and possibly misses the point of the parent.
- deleted 5y ago[deleted]
- torstenvl 5y agoA 'character' meaning what? The first code point? The first non-combining code point? The first non-combining code point along with all associated combining code points? The first non-combining code point along with all associated combining code points modified to look like it would look in conjunction with surrounding non-combining code points along with their associated combining code points? The first displayable component of a code point? What are the following? the second character of sœur (o or the ligature?) the second character of حبيبي (the canonical form ب or the contextual form ﺒ ?) the third character of есть (Cyrillic t with or without soft sign, which is always a separate code point and always displayed to the right but changes the sound?) the first character of 실례합니다 (Korean phoneme or syllabic grapheme?) the first character of ﷺ or ﷽ ? The main issue isn't programming language support, it's ambiguity in the concept of "character" and conventions about how languages treat them. Imagine how the letter i would be treated if Unicode were invented by a Turk. The fundamental issue here is that human communication is deeply nuanced in a way that does not lend itself well to systematic encoding or fast/naive algorithms on that encoding. Even in the plain ASCII range it's impossible to know how to render a word like "ANSCHLUSS" in lower case (or how many 'characters' such a word would have) without knowledge of the language, country of origin, and time period in which the word was written.
- Gigachad 5y agoUltimately I think it doesn’t really matter. Not all languages and character sets have orderings. You just order the Latin or closely related characters and numbers and you are ok for almost all situations. If you are a company or product which caters to more languages, you can work out what you do for those. But there is no need to work out how to order every single codepoint in Unicode because it isn’t very useful.
- tialaramex 5y agoYou've gone from arguing that it isn't complex if people have the right language, to redefining the problem as whatever it was easy for you to do in your preferred language. Just say "I don't have the faintest idea how to do this properly". It would be much less of a problem if people would admit they don't know how and need somebody else's help than it is when they insist it "doesn't really matter" as you have and produce garbage that doesn't work but insist that's just because everybody is holding it wrong.
- bmn__ 5y ago> A 'character' meaning what? […] it's ambiguity in the concept of "character" All of this already standardised in detail in Unicode. There is no ambiguity. Read it some time to mend your ignorance, it answers all your questions with justification, I am willing to spend time on the answers only. > the second character of sœur (o or the ligature?) the letter œ, both character (C) and grapheme cluster (GC) the letter o does not exist in this text (again, according to the standard) > the second character of حبيبي (the canonical form ب or the contextual form ﺒ ?) the letter ب, both C and GC the letter ﺒ is a presentation form and does not exist in this text > the third character of есть (Cyrillic t with or without soft sign the letter т, both C and GC the letter ь, both C and GC, is the 4th in the text > the first character of 실례합니다 (Korean phoneme or syllabic grapheme?) the letter 실, both C and GC the text does not contain the jamo ᄉ Unicode is not concerned with phonemes, only with writing > the first character of ﷺ is entirely itself, the letter ﷺ, both C and GC > or ﷽ ? is entirely itself, the symbol ﷽, both C and GC
- 5y ago
- GolDDranks 5y agoThat's not true for Rust; Rust generally supports iterating by bytes and by codepoints, but the grandparent is talking about grapheme clusters. You need an external libary for that. The generally endorsed solution is to use the unicode-segmentation crate: https://crates.io/crates/unicode-segmentation https://crates.io/crates/unicode-segmentation
- guenthert 5y agoIt's about time that someone fixes Turkish.