3 ms·

I use strtok_r from time to time, it does the job if you have a mutable input. Of course having to write zeroes is a bit cumbersome but it's one of the drawback

by simias 8y ago

I use strtok_r from time to time, it does the job if you have a mutable input. Of course having to write zeroes is a bit cumbersome but it's one of the drawbacks of C-style strings. The plain truth is that string handling in C is a huge pain in the ass no matter how you look at it. Splitting, concatenating, regex-ing... All of that is a huge pain in C. If you need to write a high-performance parser then it might be worth it but if you're just parsing a fancy command line format and performances don't matter it's just incredibly frustrating and error prone. Rust fares better here because its str type is not NUL-terminated but actually keeps track of the length separately which makes it significantly more flexible and versatile. Of course you could do that in C but you'll be incompatible with any code dealing with native C-strings. And of course you make one mistake and you have a buffer overflow vulnerability... So yeah, if you program in C please use strtok_r if applicable, otherwise considering offloading the parsing to an other part of your application written in a language better suited for that and hand over binary data to the C library. If everything else fails then consider handwriting your parser and may god be with you. Oh and if your grammar is complex enough to warrant it, there's always lex/yacc.

paavoova 8y ago
> but you'll be incompatible with any code dealing with native C-strings Not entirely, see https://github.com/antirez/sds https://github.com/antirez/sds Basically, you have a header storing length, etc, but still null terminate, so library functions like strlen are none the wiser.
- the_duke 8y ago
  Nim (the programming language) implements strings this way, AFAIK, with the same reason: to be easily compatible with the C ecosystem.
- saagarjha 8y ago
  You lose the ability to pass create zero-cost slices, though. C++11 actually implements strings like this, but iterators allow you to pass in parts of the string as necessary.
Matthias247 8y ago
> The plain truth is that string handling in C is a huge pain in the ass no matter how you look at it. It is. And it’s not even only Cs fault. 80% of it is bad API Design. Strings could be accepted as a struct consisting of a pointer and a length, aka string_view. And there could be some manipulation functions around it. That would make those APIs a lot more flexible (one no longer needs to care whether things are null terminated and there would be less pointless copies). For these reasons my estimate in the meantime is that the average C program which uses stdlib functions is less efficient than an implementation in another language, even though the authors would claim otherwise (its C, it must be fast).
- paavoova 8y ago
  What happens if you truncate your string? You lose information about buffer size. So now you need to store two sizes for such strings to be useful, string size and buffer size, which is 16 bytes for size_t on 64-bit systems. On top of that, strings are no longer arrays. So either the language would have to incorporate first class support for these strings, or you'll have people extracting the string pointer from the struct to perform indexing operations on themselves.
  - Matthias247 8y ago
    The approach obviously only works for read access and not for mutation. However those are the most often required operations. For owned strings and mutations different APIs are required. C++ string_view vs string, and Rusts str slices vs owned Strings work like this.
WalterBright 8y ago
One of the big performance wins of the D programming language over C is that arrays are length terminated instead of 0 terminated, so you can "slice" strings to get substrings, rather than allocate/copy/zero (and then get the free in the right place!).

by simias 8y ago

paavoova 8y ago

> but you'll be incompatible with any code dealing with native C-strings Not entirely, see https://github.com/antirez/sds https://github.com/antirez/sds Basically, you have a header storing length, etc, but still null terminate, so library functions like strlen are none the wiser.

the_duke 8y ago
Nim (the programming language) implements strings this way, AFAIK, with the same reason: to be easily compatible with the C ecosystem.
saagarjha 8y ago
You lose the ability to pass create zero-cost slices, though. C++11 actually implements strings like this, but iterators allow you to pass in parts of the string as necessary.

Matthias247 8y ago

> The plain truth is that string handling in C is a huge pain in the ass no matter how you look at it. It is. And it’s not even only Cs fault. 80% of it is bad API Design. Strings could be accepted as a struct consisting of a pointer and a length, aka string_view. And there could be some manipulation functions around it. That would make those APIs a lot more flexible (one no longer needs to care whether things are null terminated and there would be less pointless copies). For these reasons my estimate in the meantime is that the average C program which uses stdlib functions is less efficient than an implementation in another language, even though the authors would claim otherwise (its C, it must be fast).

paavoova 8y ago
What happens if you truncate your string? You lose information about buffer size. So now you need to store two sizes for such strings to be useful, string size and buffer size, which is 16 bytes for size_t on 64-bit systems. On top of that, strings are no longer arrays. So either the language would have to incorporate first class support for these strings, or you'll have people extracting the string pointer from the struct to perform indexing operations on themselves.
- Matthias247 8y ago
  The approach obviously only works for read access and not for mutation. However those are the most often required operations. For owned strings and mutations different APIs are required. C++ string_view vs string, and Rusts str slices vs owned Strings work like this.

WalterBright 8y ago

One of the big performance wins of the D programming language over C is that arrays are length terminated instead of 0 terminated, so you can "slice" strings to get substrings, rather than allocate/copy/zero (and then get the free in the right place!).