4 ms·

The correct answer is that strings shouldn't have length methods. There is no such thing as the 'length' of a string because as you pointed out there is no clea

The correct answer is that strings shouldn't have length methods. There is no such thing as the 'length' of a string because as you pointed out there is no clear definition of what a 'character' is. https://www.mikeash.com/pyblog/friday-qa-2015-11-06-why-is-swifts-string-api-so-hard.html https://www.mikeash.com/pyblog/friday-qa-2015-11-06-why-is-s...

the_mitsuhiko 11y ago
To add to that: O(1) indexing into strings outside of the ASCII range is completely pointless and makes absolutely no sense. A language should disallow that and not encourage it.
- Veedrac 11y ago
  I disagree with it being pointless. Being able to get indices into a string for fast slicing is useful. Consider Regex where references get stored to slices in the array. What you can drop is the idea of the index being an integer with any particular semantic meaning. Rust uses byte indexes with an assertion that the index lies on a codepoint boundary, for instance.
  - the_mitsuhiko 11y ago
    > Being able to get indices into a string for fast slicing is useful. Outside of the ASCII range? I doubt it. That most likely means you are doing something fundamentally wrong.
    - heinrich5991 11y ago
      I think something like a lexer doesn't need to know about graphemes, it just needs to know which codepoints are XID_start and XID_continue for identifiers and hardcoded codepoints for everything else. EDIT: Or a JSON parser, etc.
      - the_mitsuhiko 11y ago
        A lexer scans LTR. It never jumps codepoints ahead. Indexing in a lexer never makes sense.
        tomp 11y ago
        Obviously, it does; instead of allocating a new string for every token, you can just save the start & end index of a token, and actually allocate a new string only in the case where the parser actually requests/uses that token in the AST.
        nostrademons 11y ago
        But the type signature of that token should make it clear those are byte offsets and not character offsets. You can use them for byte-wise operations like memcpy, or to explicitly decode them into a string using the original encoding of the string. You cannot use them for string-wise operations like substring without explicitly casting or converting them to a string. It seems like this whole mess comes from conflating bytes and characters. They are not the same, any more than integers and booleans are the same just because you can jnz in most assembly languages.
        Veedrac 11y ago
        > You cannot use them for string-wise operations like substring without explicitly casting or converting them to a string. Why on earth not? Any slice of a string (with a non-pathological internal encoding) with ends on codepoint boundaries is itself a valid string. Losing that information sounds like a whole lot of pointless hassle and potential for harm. It also forces the internal string encoding to be part of the interface, since taking a slice of a string and getting a byte array requires knowing how the string is encoded to use it, but getting another string means that such details are private. It's true that they're byte offsets, but that doesn't matter. The parser doesn't care whether it's on the 10543th grapheme or the 10597th. It just wants to know where its stuff is.
    - rbanffy 11y ago
      Outside of the anglocentric world we also think in terms of characters and, yet, we consider ASCII as a rather primitive technology.
  - TheCoelacanth 11y ago
    That isn't an index into a string. It is an opaque indicator of position in a string. That opaque indicator of position happens to be implemented as an index into the underlying array, but it is not itself an index.
    - Veedrac 11y ago
      That sounds like an index to me, and the operation surely indexing. I mean, if one has a hashmap hash = {"foo": 1, "bar": 2} we say hash["foo"] is an indexing operation, and "foo" is an index. Seems the same to me.
ubernostrum 11y ago
The correct answer is that strings shouldn't have length methods There are plenty of uses for getting the length of a string where you will not be running into combining characters, clusters or any other things which would require special handling. (remember that on the internet, all the data you receive is initially in string form, and very often we want to receive data representing fixed-length values such as phone numbers or credit card numbers, so "you're wrong for wanting to know the length" is both a non-starter and, well, just plain wrong)
- anon1385 11y ago
  >There are plenty of uses for getting the length of a string where you will not be running into combining characters, clusters or any other things which would require special handling. How do you know that? Any user supplied text may contain those things. >remember that on the internet, all the data you receive is initially in string form It's not. It's bytes. >very often we want to receive data representing fixed-length values such as phone numbers or credit card numbers This is a good example of why strings shouldn't have a length property - people don't understand what length actually does in most languages. If you think that a len() function can verify that the contents of a text field is in the format of a CC number then you don't understand the len function. It can't do what you are asking. See the examples given in the GP (len("\u0301") for example). The fact that the people who advocate for length functions want to use them for things that they are broken for is exactly why length functions shouldn't exist.
  - ubernostrum 11y ago
    How do you know that? Any user supplied text may contain those things. Which is why, in addition to things like length checks, we use other checks. But length is a quick and easy check for many types of values, and should be supported. I've been down in the depths of Unicode behavior a time or two. I know what lurks down there, and I know from experience that your "strings should never allow these operations" stance is just as dumb as the more-common "I don't know what Unicode really is" stance. Can length checks and slicing and other operations run into trouble in some corners of Unicode? Sure. But if "this might cause trouble" were grounds for forbidding programmers ever to do something, we wouldn't have computers, any useful software at all, the internet or most of the other things we like and take for granted despite the fact that we occasionally have to fix bugs in it. Now, get down off your snobby horse and join the rest of the real world, where we know that something might not be guaranteed reliable in all logically-possible cases, but find ways to work with it anyway because in 99.99% (or more) of the cases our code deals with it's good enough.

the_mitsuhiko 11y ago

To add to that: O(1) indexing into strings outside of the ASCII range is completely pointless and makes absolutely no sense. A language should disallow that and not encourage it.

Veedrac 11y ago
I disagree with it being pointless. Being able to get indices into a string for fast slicing is useful. Consider Regex where references get stored to slices in the array. What you can drop is the idea of the index being an integer with any particular semantic meaning. Rust uses byte indexes with an assertion that the index lies on a codepoint boundary, for instance.
- the_mitsuhiko 11y ago
  > Being able to get indices into a string for fast slicing is useful. Outside of the ASCII range? I doubt it. That most likely means you are doing something fundamentally wrong.
  - heinrich5991 11y ago
    I think something like a lexer doesn't need to know about graphemes, it just needs to know which codepoints are XID_start and XID_continue for identifiers and hardcoded codepoints for everything else. EDIT: Or a JSON parser, etc.
    - the_mitsuhiko 11y ago
      A lexer scans LTR. It never jumps codepoints ahead. Indexing in a lexer never makes sense.
      - tomp 11y ago
        Obviously, it does; instead of allocating a new string for every token, you can just save the start & end index of a token, and actually allocate a new string only in the case where the parser actually requests/uses that token in the AST.
        nostrademons 11y ago
        But the type signature of that token should make it clear those are byte offsets and not character offsets. You can use them for byte-wise operations like memcpy, or to explicitly decode them into a string using the original encoding of the string. You cannot use them for string-wise operations like substring without explicitly casting or converting them to a string. It seems like this whole mess comes from conflating bytes and characters. They are not the same, any more than integers and booleans are the same just because you can jnz in most assembly languages.
        Veedrac 11y ago
        > You cannot use them for string-wise operations like substring without explicitly casting or converting them to a string. Why on earth not? Any slice of a string (with a non-pathological internal encoding) with ends on codepoint boundaries is itself a valid string. Losing that information sounds like a whole lot of pointless hassle and potential for harm. It also forces the internal string encoding to be part of the interface, since taking a slice of a string and getting a byte array requires knowing how the string is encoded to use it, but getting another string means that such details are private. It's true that they're byte offsets, but that doesn't matter. The parser doesn't care whether it's on the 10543th grapheme or the 10597th. It just wants to know where its stuff is.
  - rbanffy 11y ago
    Outside of the anglocentric world we also think in terms of characters and, yet, we consider ASCII as a rather primitive technology.
- TheCoelacanth 11y ago
  That isn't an index into a string. It is an opaque indicator of position in a string. That opaque indicator of position happens to be implemented as an index into the underlying array, but it is not itself an index.
  - Veedrac 11y ago
    That sounds like an index to me, and the operation surely indexing. I mean, if one has a hashmap hash = {"foo": 1, "bar": 2} we say hash["foo"] is an indexing operation, and "foo" is an index. Seems the same to me.

ubernostrum 11y ago

The correct answer is that strings shouldn't have length methods There are plenty of uses for getting the length of a string where you will not be running into combining characters, clusters or any other things which would require special handling. (remember that on the internet, all the data you receive is initially in string form, and very often we want to receive data representing fixed-length values such as phone numbers or credit card numbers, so "you're wrong for wanting to know the length" is both a non-starter and, well, just plain wrong)

anon1385 11y ago
>There are plenty of uses for getting the length of a string where you will not be running into combining characters, clusters or any other things which would require special handling. How do you know that? Any user supplied text may contain those things. >remember that on the internet, all the data you receive is initially in string form It's not. It's bytes. >very often we want to receive data representing fixed-length values such as phone numbers or credit card numbers This is a good example of why strings shouldn't have a length property - people don't understand what length actually does in most languages. If you think that a len() function can verify that the contents of a text field is in the format of a CC number then you don't understand the len function. It can't do what you are asking. See the examples given in the GP (len("\u0301") for example). The fact that the people who advocate for length functions want to use them for things that they are broken for is exactly why length functions shouldn't exist.
- ubernostrum 11y ago
  How do you know that? Any user supplied text may contain those things. Which is why, in addition to things like length checks, we use other checks. But length is a quick and easy check for many types of values, and should be supported. I've been down in the depths of Unicode behavior a time or two. I know what lurks down there, and I know from experience that your "strings should never allow these operations" stance is just as dumb as the more-common "I don't know what Unicode really is" stance. Can length checks and slicing and other operations run into trouble in some corners of Unicode? Sure. But if "this might cause trouble" were grounds for forbidding programmers ever to do something, we wouldn't have computers, any useful software at all, the internet or most of the other things we like and take for granted despite the fact that we occasionally have to fix bugs in it. Now, get down off your snobby horse and join the rest of the real world, where we know that something might not be guaranteed reliable in all logically-possible cases, but find ways to work with it anyway because in 99.99% (or more) of the cases our code deals with it's good enough.