11 ms·
IMHO Zig's builtin string handling (or rather, lack of) is exactly right for a systems programming language. Zig avoids the biggest problem of C strings and tre
by flohofwoe 5y ago
IMHO Zig's builtin string handling (or rather, lack of) is exactly right for a systems programming language. Zig avoids the biggest problem of C strings and treats strings as ptr/length slices, not as zero-terminated. UTF-8 for string literals is also fine.
That's all that's needed (and should be implemented) on the language level, everything else should go into the standard library, and additional specialized string processing libraries (because string handling can never be a "one-size fits all" solution, it's too complex for that).
- turminal 5y agoThe issue with treating strings as slices/arrays is that it exposes (writably!) the underlying representation, which is almost never what you want and leads to subtle bugs. The downside is of course the fact that you have to account for encodings in your language now, but picking the one and only sensible encoding really shouldn't be a problem in 2021.
- von_lohengramm 5y agoIt's not that simple. Obviously, you'd choose UTF-8, but that exposes another issue: a codepoint is not a character. There's no simple & elegant solution to this at the language level.
- turminal 5y ago> a codepoint is not a character Precisely. That's why I think representing strings as character slices and letting third party libraries handle them is not a good solution. Deciding what features to put in the language vs. the stdlib vs. third party libraries is one of the hardest parts of language design. Personally I believe strings are important and frequent enough they deserve special treatment at the language level. Edit (late addition): I think not treating strings specially is mostly fine in C, but Zig seems to aim at being a little less lowlevel.
- von_lohengramm 5y ago> representing strings as character slices Zig does not do this. It represents strings as byte slices. A UTF-8 character could be multiple codepoints with each codepoint being multiple bytes. A big thing about Zig is not hiding complexity. If UTF-8 were implemented at a language level (whatever that means), then "language level" string operations would be non-linear, which would be very non-Ziggy. I could see value in a standard library UTF-8 implementation, but a LOT of forethought would need to be put into it. I think keeping UTF-8 string manipulation at the third-party library level is a good choice for now. Maybe once the language is finalized, the ecosystem is more developed, and lessons have been learned from the third-party libraries, then the standard library can implement this.
- turminal 5y ago> character slices Sorry, byte slices is what I had in mind. I'm not talking about language level string operations as in concatenation with + or something like that at all, because that certainly wouldn't make sense in language like Zig. I'm not advocating for string functionality in the language, I'm advocating for a way to not allow byte slice functionality on a thing that is clearly not a byte slice.
- von_lohengramm 5y ago> a way to not allow byte slice functionality on a thing that is clearly not a byte slice This already exists in the form of structs or opaque types. Both of these approaches would end up being implemented in "userspace" anyways, whether that's standard library or third-party. However, (UTF-8) strings are byte slices. You can do simple manipulation with them as byte slices safely and validly. Split on spaces? Sure. Tokenize? Sure. Find substring? Sure. You can't do things that depend on say UTF-8 graphemes, but you can safely do most things that depend on bytes. For most purposes, treating strings as byte slices is the safest and correct approach.
- lmm 5y agoDoing find substring by find byte subsequence won't behave correctly in many cases, where semantically equivalent strings have multiple different bytesequence representation. Treating strings as byte slices exposes a lot of footguns; it shouldn't be easy just as e.g. treating floating-point numbers as byte sequences shouldn't be easy.
- coldtea 5y agoSo let's have everybody write their own incompatible libs and partial solutions... It worked for C /s
- diegocg 5y agoThey can put it on the standard library or the core language or wherever they want, but they absolutely need to provide good string handling. And regardless of where they put the code, this is something that needs to be done by the core team. Otherwise you will end with too many string libraries, all of them trying to solve a particular problem and doing bad at everything else, with bad documentation and different API styles and making difficult to mix code that uses two different libraries.
- AndyKelley 5y agoA lot of people reach for string handling when the actual correct thing to do is intentionally avoid string handling, and only handle strings as opaque encoded UTF-8 bytes, that cannot be reasoned about in terms of human language. I would even argue that having string handling in a standard library (or language) has the potential to cause a net increase in bugs, because of people thinking they are handling strings when actually they are just screwing around with codepoints. Go's string handling is completely broken, for example. As a result of strings in the language, Go programs tend to be more broken than C programs in terms of string handling.
- gautamcgoel 5y agoCan you elaborate on why you think Go's string handling is broken?
- tdeck 5y ago> the actual correct thing to do is intentionally avoid string handling, That sounds nice and all, but wr have 50+ years of protocols and formats and APIs built up around strings. Unless you're just writing code to run on a small microcontroller, you need to be able to parse and generate strings. So its going to be pretty frustrating not to have good support for them, or to have every codebase use its own libraries and idioms for working with strings.
- AndyKelley 5y agoProtocols and formats absolutely should not require decoding strings. I think you are mistaken. Can you name any well-established protocol or format that does not treat strings as opaque encoded bytes? Edit: so far these examples have been given: * HTTP: wrong. the spec does not tell you to decode any strings * CSV: wrong. the spec does not tell you to decode any strings, nor is it necessary to have any unicode awareness in order to properly read and parse the data or deal with the delimiters. Additional request: if you attempt to provide a counter-example, please also point to the place in the spec where it tells you to decode a string.
- winter_squirrel 5y ago
- formerly_proven 5y agou8 slices strike me as exactly the correct thing on the language level as well. In particular, it allows very straightforward zero-copy parsing/tokenization of strings. Most parsing of strings doesn't really need to know about UTF-8 and works correctly in the presence of non-ASCII codepoints. String literally are not really UTF-8. Rather, zig source files are UTF-8 (by definition), and string literals are u8 literals; putting some UTF-8 between quotes just puts the literal UTF-8 encoded text into the literal because those are the bytes that are in the source file. In particular, string literals can contain arbitrary binary data and null bytes by way of \x00 escapes. For Unicode handling there's some basic stuff in std.unicode (conversion between different UTF encodings, checking validity, decoding to codepoints etc.). This is used e.g. on Windows for checking filesystem paths. I don't general libraries of encodings is really that important today, iso-8859 and shift JIS might be useful sometimes, but everything else probably doesn't need to bloat up a standard library (iirc Python's codecs package, which contains dozens upon dozens of encodings, is like a third of the standard library by size).
- anonymoushn 5y agoZig already ships a bunch of libcs so I don't think there's much concern about bloating the standard library.