6 ms·
I don't think a lot of people realize that Unicode itself, in any representation, is complex, and there is no one abstraction that will make it always easy in a
by asveikau 8y ago
I don't think a lot of people realize that Unicode itself, in any representation, is complex, and there is no one abstraction that will make it always easy in all cases. So I think your goal is unobtainable.
Some of that complexity is because world languages themselves are not simple. (eg. If you were to ditch Unicode and start from scratch, it would remain hard to do bi-di text, to pick a random example.)
- sonnyblarney 8y agoTotally see what you are saying (the underlying issue is language itself), but it's pragmatically obtainable.
- nicky0 8y agoThe think is, strings that make sense has been a major goal of Swift, that's why they have been iterating on it so heavily. FWIW I think Swift's is probably one of the cleanest String abstractions out there with respect to unicode correctness balanced against efficiency and usability.
- asveikau 8y agoTo be clear I applaud successful attempts to make it better, what I doubt is a possibility to make the problem completely go away without users of the api ever having to think.
- masklinn 8y agoAh, so your solution is the same old canard: just ignore the complexity and call it pragmatic.
- sonnyblarney 8y agoNo. How about a string that returns a Unicode character length? That's not 'ignoring complexity' and it's pragmatic. That would help in English as it's really hard today just to deal emojis.
- asveikau 8y agoThe point I tried to make: I suspect if you dug really deep into what is a character or glyph you might find "character length" is an imprecise and messy notion, with differing performance costs associated with differing definitions which may or may not apply to you. So even if you pick one definition and decide to live with it always... You'll still have to know what tradeoffs you (or an API designer or implementer) made.
- sonnyblarney 8y agoFirst - a combined unicode length is definitely universally useful. Second - a grapheme length is universally useful. Third - I didn't say 'pick one definition' - you did. A simple function which took as a parameter some kind of linguistic construct would be immensely useful in cases of choice/ambiguity. 'It's complicated' is the utterly the wrong answer as to why we 'can't have better strings'. Strings in almost every programming language were designed before Emjojis and true internationalization. If we designed them today, they would have support for dealing with most of the problems we face. And FYI - there are not infinite corner cases. They are limited, and we can develop constructs for dealing with all of them.
- asveikau 8y agoThere are myriad problems today that 99% of developers couldn't be bothered to think about, but can be a source of bugs. eg. I'm going to guess every time you call tolower() have you haven't thought "I wonder what this does to Turkish I", but it's still there. An API won't fix us from having to think, and having ambiguities occasionally become bugs. (The fact that you've acknowledged the need for multiple string length functions -- I agree with it, but it also means you'll sometimes use the wrong one.) As I said in another comment, I still applaud efforts to improve the status quo.
- masklinn 8y ago> No. How about a string that returns a Unicode character length? That doesn't mean anything. Do you mean a code unit, a codepoint, a grapheme cluster, if the latter is it legacy, extended or tailored, how do you handle the locale dependency, … All of them can be useful in some context, all of them are length. And if anything, the truly useful one is the first one (because it tells you how much room you need to store something). > That's not 'ignoring complexity' and it's pragmatic. It's true that it's not ignoring complexity, because you're ignorant of it to start with. Which isn't really different. > That would help in English as it's really hard today just to deal emojis. It… really isn't.