3 ms·

No. How about a string that returns a Unicode character length? That's not 'ignoring complexity' and it's pragmatic. That would help in English as it's really

No. How about a string that returns a Unicode character length? That's not 'ignoring complexity' and it's pragmatic. That would help in English as it's really hard today just to deal emojis.

asveikau 8y ago
The point I tried to make: I suspect if you dug really deep into what is a character or glyph you might find "character length" is an imprecise and messy notion, with differing performance costs associated with differing definitions which may or may not apply to you. So even if you pick one definition and decide to live with it always... You'll still have to know what tradeoffs you (or an API designer or implementer) made.
- sonnyblarney 8y ago
  First - a combined unicode length is definitely universally useful. Second - a grapheme length is universally useful. Third - I didn't say 'pick one definition' - you did. A simple function which took as a parameter some kind of linguistic construct would be immensely useful in cases of choice/ambiguity. 'It's complicated' is the utterly the wrong answer as to why we 'can't have better strings'. Strings in almost every programming language were designed before Emjojis and true internationalization. If we designed them today, they would have support for dealing with most of the problems we face. And FYI - there are not infinite corner cases. They are limited, and we can develop constructs for dealing with all of them.
  - asveikau 8y ago
    There are myriad problems today that 99% of developers couldn't be bothered to think about, but can be a source of bugs. eg. I'm going to guess every time you call tolower() have you haven't thought "I wonder what this does to Turkish I", but it's still there. An API won't fix us from having to think, and having ambiguities occasionally become bugs. (The fact that you've acknowledged the need for multiple string length functions -- I agree with it, but it also means you'll sometimes use the wrong one.) As I said in another comment, I still applaud efforts to improve the status quo.
masklinn 8y ago
> No. How about a string that returns a Unicode character length? That doesn't mean anything. Do you mean a code unit, a codepoint, a grapheme cluster, if the latter is it legacy, extended or tailored, how do you handle the locale dependency, … All of them can be useful in some context, all of them are length. And if anything, the truly useful one is the first one (because it tells you how much room you need to store something). > That's not 'ignoring complexity' and it's pragmatic. It's true that it's not ignoring complexity, because you're ignorant of it to start with. Which isn't really different. > That would help in English as it's really hard today just to deal emojis. It… really isn't.
- sonnyblarney 8y ago
  Totally wrong. Your implied solution is to just have every single developer, everywhere, to be a linguistics expert and to handle the myriad of international cases for basically everything. This is totally the wrong approach. Language is shifty, but there are definitely a series of tools that should be integrated into every 'String' that can facilitate most problems. "That doesn't mean anything" This is false. It absolutely means something, you can't brush it aside by inventing a separate series of specific measurements. Should we have those other measurements - yes, in many cases, we probably should have them as well -> that's the whole point.
- sonnyblarney 8y ago
  Totally wrong. Your implied solution is to just have every single developer, everywhere, to be a linguistics expert and to handle the myriad of international cases for basically everything. This is totally the wrong approach. Language is complex and sometimes ambiguous, but there are definitely a series of tools that should be integrated into every 'String' that can facilitate most problems. Not every company is trying to satisfy Hindi speakers. "That doesn't mean anything" This is false. It absolutely means something, you can't brush it aside by inventing a separate series of related measurements. Should we have those other measurements? - yes, in many cases, we probably should have them as well - which the whole point. We desperately need a series of common tools built into String, some of them which provide more nuance. "It's true that it's ignoring complexity, because you're ignorant to start with" Well, since I have a background in NLP, I've developed writing recognition and word prediction algorithms, as well as language models for dozens of languages for products used by millions of people every day ... maybe I'm not as ignorant as your snide remark suggests? "> That would help in English as it's really hard today just to deal emojis. It… really isn't." Uh, yes, it absolutely would help, and it's painfully obvious. In most languages, which cover the exceeding majority of markets for most apps made, simply knowing the character length (i.e. 'visible character' as to avoid confusion over messy nomenclature here) is very useful. In Latin languages, after Unicode normalization and other kinds of cleaning, Emojis present a very common case for confusion. There are very small set of tools (or rather, a slightly better, more standardized String approach) which would solve most of the problems for most companies developing for most markets. There are even a few extra tools which would solve, pragmatically all of them.

No. How about a string that returns a Unicode character length? That's not 'ignoring complexity' and it's pragmatic. That would help in English as it's really hard today just to deal emojis.

asveikau 8y ago

The point I tried to make: I suspect if you dug really deep into what is a character or glyph you might find "character length" is an imprecise and messy notion, with differing performance costs associated with differing definitions which may or may not apply to you. So even if you pick one definition and decide to live with it always... You'll still have to know what tradeoffs you (or an API designer or implementer) made.

sonnyblarney 8y ago
First - a combined unicode length is definitely universally useful. Second - a grapheme length is universally useful. Third - I didn't say 'pick one definition' - you did. A simple function which took as a parameter some kind of linguistic construct would be immensely useful in cases of choice/ambiguity. 'It's complicated' is the utterly the wrong answer as to why we 'can't have better strings'. Strings in almost every programming language were designed before Emjojis and true internationalization. If we designed them today, they would have support for dealing with most of the problems we face. And FYI - there are not infinite corner cases. They are limited, and we can develop constructs for dealing with all of them.
- asveikau 8y ago
  There are myriad problems today that 99% of developers couldn't be bothered to think about, but can be a source of bugs. eg. I'm going to guess every time you call tolower() have you haven't thought "I wonder what this does to Turkish I", but it's still there. An API won't fix us from having to think, and having ambiguities occasionally become bugs. (The fact that you've acknowledged the need for multiple string length functions -- I agree with it, but it also means you'll sometimes use the wrong one.) As I said in another comment, I still applaud efforts to improve the status quo.

masklinn 8y ago

> No. How about a string that returns a Unicode character length? That doesn't mean anything. Do you mean a code unit, a codepoint, a grapheme cluster, if the latter is it legacy, extended or tailored, how do you handle the locale dependency, … All of them can be useful in some context, all of them are length. And if anything, the truly useful one is the first one (because it tells you how much room you need to store something). > That's not 'ignoring complexity' and it's pragmatic. It's true that it's not ignoring complexity, because you're ignorant of it to start with. Which isn't really different. > That would help in English as it's really hard today just to deal emojis. It… really isn't.

sonnyblarney 8y ago
Totally wrong. Your implied solution is to just have every single developer, everywhere, to be a linguistics expert and to handle the myriad of international cases for basically everything. This is totally the wrong approach. Language is shifty, but there are definitely a series of tools that should be integrated into every 'String' that can facilitate most problems. "That doesn't mean anything" This is false. It absolutely means something, you can't brush it aside by inventing a separate series of specific measurements. Should we have those other measurements - yes, in many cases, we probably should have them as well -> that's the whole point.
sonnyblarney 8y ago
Totally wrong. Your implied solution is to just have every single developer, everywhere, to be a linguistics expert and to handle the myriad of international cases for basically everything. This is totally the wrong approach. Language is complex and sometimes ambiguous, but there are definitely a series of tools that should be integrated into every 'String' that can facilitate most problems. Not every company is trying to satisfy Hindi speakers. "That doesn't mean anything" This is false. It absolutely means something, you can't brush it aside by inventing a separate series of related measurements. Should we have those other measurements? - yes, in many cases, we probably should have them as well - which the whole point. We desperately need a series of common tools built into String, some of them which provide more nuance. "It's true that it's ignoring complexity, because you're ignorant to start with" Well, since I have a background in NLP, I've developed writing recognition and word prediction algorithms, as well as language models for dozens of languages for products used by millions of people every day ... maybe I'm not as ignorant as your snide remark suggests? "> That would help in English as it's really hard today just to deal emojis. It… really isn't." Uh, yes, it absolutely would help, and it's painfully obvious. In most languages, which cover the exceeding majority of markets for most apps made, simply knowing the character length (i.e. 'visible character' as to avoid confusion over messy nomenclature here) is very useful. In Latin languages, after Unicode normalization and other kinds of cleaning, Emojis present a very common case for confusion. There are very small set of tools (or rather, a slightly better, more standardized String approach) which would solve most of the problems for most companies developing for most markets. There are even a few extra tools which would solve, pragmatically all of them.