5 ms·

What I want from strings in a programming language: 1) Memory-safe (can't touch memory that you don't own) 2) Opaque (standard library can change the implemen

What I want from strings in a programming language: 1) Memory-safe (can't touch memory that you don't own) 2) Opaque (standard library can change the implementation later) 3) Immutable (can be passed around like values) 4) Unicode (no fixed width chars, no default conversion to bytes) I don't know any good reason to deviate from these rules. Even systems languages would do well to follow them, IMO.

std_throwaway 11y ago
Zero terminated strings make a lot of sense when programming 8 bit microcontrollers: 1) No protection exists there. 2) Abstraction costs too much. 3) Store it in Flash. 4) Be happy your characters may even use the 8th bit.
- pjmlp 11y ago
  Except even there, there were computers using safer system programming languages in the 60's where a modern 8 bit microcontroller would seem like a workstation in those days.
josteink 11y ago
> 3) Immutable (can be passed around like values) > I don't know any good reason to deviate from these rules. Even systems languages would do well to follow them, IMO. Systems-languages are used for things like OS kernels and god knows what where other equally important rules apply: 1. being able to manipulate things directly, in place, without having to allocate additional memory (i.e. creating a new copy) 2. efficiency may be more crucial than type-system guaranteed safety. Not saying I disagree with your general position, I'm just open to the fact that there's always valid exceptions to any rules.
- pjmlp 11y ago
  And that is why sane system programming languages like Burroughs Algol, Mesa, Modula-2, Modula-3, Ada, .... provide escape hatches for the few uses where breaking the rules is really required. But, importantly, they don't do it by default as C does.
- vardump 11y ago
  Hmm... I can't remember any scenario I needed to manipulate (mutate) strings in kernel in place. For guaranteed small, say below 64 bytes, strings stack is usually fine. Not saying it doesn't happen, only that it is rarely needed. Usually when you can't allocate memory (like IRQ handler), you don't do string processing either.
deleted 11y ago
[deleted]
geocar 11y ago
I don't think any of these are valuable, let alone essential. 1) Memory-safe costs a lot and gives me little. If I know I'm not going to touch memory incorrectly I should be able to take advantage of that. 2) Knowing the cost of operations is essential to do any kind of performant programming in a reasonable amount of time; e.g. whether `length(x)` is O(N) or O(1). The operations specified is hugely important. 3) When I've got a 300GB log to update, an in-place algorithm will be better for everybody. Microcontrollers don't want to waste space with extra copies either, and a better algorithm wins. 4) I'm not sure having an opinion about Unicode in the language is ideal: You can either store an array of bytes, or you can store an array of code points, or you could store a list of UTF8 "characters", and I see value in all of these interpretations (comparison/copying, rendering, transcoding). I'd prefer flexibility, and the language not switching them around on me behind my back (see #2)
- cousin_it 11y ago
  1) Hopefully memory-safety without GC will be mainstream soon. Rust is showing the way. 2) I agree that time and space complexity should be part of the contract. 3) I agree that escape hatches are needed, but they shouldn't be the default. Most code isn't performance sensitive, but all code is bug sensitive. 4) The best internal representation is probably UTF-8, but I don't feel strongly about that. What matters is that the available string operations are specified in terms of Unicode, not bytes. I grew up in Russia and I remember very well the "fun" of programming with Cyrillic strings in a mixture of Win-1251, KOI8-R, and Unicode. Working with strings as untagged bytes is a disaster everywhere except English speaking countries.
  - geocar 11y ago
    I once observed a lot of bugs showing up in a Python project because of transparent conversion between byte-strings and unicode-strings. Programmers would test with byte-strings, everything is fine, customer is in east Europe, and you know what happens next. I think I decided that UTF-8 was actually worse than UTF-8-xor-69 so that the innermost code should be able to tell whether or note this was a bytestring or not. Explicit tagging found loads of bugs that day. An array of bytes is a storage medium. If you don't need the number of utf-8 characters (and given things like combining characters and invisible spaces, it's rare that's what I actually need) and just need to print and compare, byte-strings are efficient and acceptable. The text remains utf-8 encoded, but `length` is `O(1)` or a simple scan, and `substr` (or similar) are `O(1)`+the cost of allocating. However if I need to recode things, an array of 32-bit (or 64-bit hah) ints is sometimes better. If the language has a real iterator, UTF-8 might still be okay, but then you are still using a bytestring anyway. Lastly, if the "string" actually knows its character set, then using things besides UTF-8 might be good because it saves a lot of space. Maybe. I'm not really convinced because the tag for the character set (as you rightly think necessary) takes up space too. Dealing with these kinds of "strings" is really complicated, so I'd agree that anyone who would jump into complexity without some real justification is, as you say, a bozo.

std_throwaway 11y ago

Zero terminated strings make a lot of sense when programming 8 bit microcontrollers: 1) No protection exists there. 2) Abstraction costs too much. 3) Store it in Flash. 4) Be happy your characters may even use the 8th bit.

pjmlp 11y ago
Except even there, there were computers using safer system programming languages in the 60's where a modern 8 bit microcontroller would seem like a workstation in those days.

josteink 11y ago

> 3) Immutable (can be passed around like values) > I don't know any good reason to deviate from these rules. Even systems languages would do well to follow them, IMO. Systems-languages are used for things like OS kernels and god knows what where other equally important rules apply: 1. being able to manipulate things directly, in place, without having to allocate additional memory (i.e. creating a new copy) 2. efficiency may be more crucial than type-system guaranteed safety. Not saying I disagree with your general position, I'm just open to the fact that there's always valid exceptions to any rules.

pjmlp 11y ago
And that is why sane system programming languages like Burroughs Algol, Mesa, Modula-2, Modula-3, Ada, .... provide escape hatches for the few uses where breaking the rules is really required. But, importantly, they don't do it by default as C does.
vardump 11y ago
Hmm... I can't remember any scenario I needed to manipulate (mutate) strings in kernel in place. For guaranteed small, say below 64 bytes, strings stack is usually fine. Not saying it doesn't happen, only that it is rarely needed. Usually when you can't allocate memory (like IRQ handler), you don't do string processing either.

deleted 11y ago

[deleted]

geocar 11y ago

I don't think any of these are valuable, let alone essential. 1) Memory-safe costs a lot and gives me little. If I know I'm not going to touch memory incorrectly I should be able to take advantage of that. 2) Knowing the cost of operations is essential to do any kind of performant programming in a reasonable amount of time; e.g. whether `length(x)` is O(N) or O(1). The operations specified is hugely important. 3) When I've got a 300GB log to update, an in-place algorithm will be better for everybody. Microcontrollers don't want to waste space with extra copies either, and a better algorithm wins. 4) I'm not sure having an opinion about Unicode in the language is ideal: You can either store an array of bytes, or you can store an array of code points, or you could store a list of UTF8 "characters", and I see value in all of these interpretations (comparison/copying, rendering, transcoding). I'd prefer flexibility, and the language not switching them around on me behind my back (see #2)

cousin_it 11y ago
1) Hopefully memory-safety without GC will be mainstream soon. Rust is showing the way. 2) I agree that time and space complexity should be part of the contract. 3) I agree that escape hatches are needed, but they shouldn't be the default. Most code isn't performance sensitive, but all code is bug sensitive. 4) The best internal representation is probably UTF-8, but I don't feel strongly about that. What matters is that the available string operations are specified in terms of Unicode, not bytes. I grew up in Russia and I remember very well the "fun" of programming with Cyrillic strings in a mixture of Win-1251, KOI8-R, and Unicode. Working with strings as untagged bytes is a disaster everywhere except English speaking countries.
- geocar 11y ago
  I once observed a lot of bugs showing up in a Python project because of transparent conversion between byte-strings and unicode-strings. Programmers would test with byte-strings, everything is fine, customer is in east Europe, and you know what happens next. I think I decided that UTF-8 was actually worse than UTF-8-xor-69 so that the innermost code should be able to tell whether or note this was a bytestring or not. Explicit tagging found loads of bugs that day. An array of bytes is a storage medium. If you don't need the number of utf-8 characters (and given things like combining characters and invisible spaces, it's rare that's what I actually need) and just need to print and compare, byte-strings are efficient and acceptable. The text remains utf-8 encoded, but `length` is `O(1)` or a simple scan, and `substr` (or similar) are `O(1)`+the cost of allocating. However if I need to recode things, an array of 32-bit (or 64-bit hah) ints is sometimes better. If the language has a real iterator, UTF-8 might still be okay, but then you are still using a bytestring anyway. Lastly, if the "string" actually knows its character set, then using things besides UTF-8 might be good because it saves a lot of space. Maybe. I'm not really convinced because the tag for the character set (as you rightly think necessary) takes up space too. Dealing with these kinds of "strings" is really complicated, so I'd agree that anyone who would jump into complexity without some real justification is, as you say, a bozo.