5 ms·

> If it were obvious it would have been done already. Witness the many variants that try to make it better but don't. Every other language with mutable strings

> If it were obvious it would have been done already. Witness the many variants that try to make it better but don't. Every other language with mutable strings, including C++, does it like that. It is obvious. The reason it is not done in C is not ignorance, it is laziness. > Which you then can't pass to any other library. String management is very easy to solve within the boundaries of your own code. But you'll need to interact with existing code as well. Ignoring the also obvious solution of just keeping a null terminator around (see: C++ std::string), you should only worry about it at the boundary with the other library. Same as converting from utf-8 to utf-16 to talk to the Windows API for example.

jjav 3y ago
> The reason it is not done in C is not ignorance, it is laziness. Of course not. C has been around since the dawn of UNIX and the majority of important libraries at the OS level are written in it. Compatibility with such a vast amount of code is a lot more important than anything else. If it were so easy why do you think nobody has done it? > Ignoring the also obvious solution of just keeping a null terminator around That's not very useful for the general case. If your code relies on the extra metadata (length, size) being correct and you're passing that null-terminated buffer around to libraries outside your code, it won't be correct since nothing else is aware of it.
- sirwhinesalot 3y ago
  > If it were so easy why do you think nobody has done it? People have done it, there are plenty strbuf implementations to go around. Even the kernel has seq_buf. How you handle string manipulation internally in your codebase does not matter for compatibility with existing libraries. > That's not very useful for the general case. If your code relies on the extra metadata (length, size) being correct and you're passing that null-terminated buffer around to libraries outside your code, it won't be correct since nothing else is aware of it. You can safely pass the char* buffer inside a std::string to any C library with no conversion. You're making up issues in your head. Don't excuse incompetence.
  - jjav 3y ago
    > People have done it, there are plenty strbuf implementations to go around. Precisely! Why plenty and why is none of them the standard in C?
    - sirwhinesalot 3y ago
      The TL;DR on that is basically "lazy, security unconscious assholes keep shutting it down". Dennies Ritchie strongly suggested C should add fat pointers all the way back in 1990. Other people have pointed out the issues with zero terminated strings and arrays decaying into pointers (and the ways to deal with them even with backwards compatibility constraints) for years. One of the most prominent was Walter Bright's article on "C's Biggest Mistake" back in 2009 and he was a C/C++ commercial compiler developer. There is no excuse.
      - jjav 3y ago
        It is easy to document mistakes in hindsight, since hindsight is 20/20. It is very easy to write your own one-off secure string handling library. This is a common assignment in intro to C programming classes. So why isn't it standard in C already? You offer a theory that there is a gang of "security unconscious assholes [who] keep shutting it down". This gang is so well organized that they have managed to block an easy improvement for many many decades for unknown reasons. That's a pretty wild theory. Or Occam's razor suggests a different answer: It's actually difficult. No, not the writing code part, that's easy. It's the seamlessly integrating with ~60 years of mission critical codebases part that's hard.
        sirwhinesalot 3y ago
        There's no need to integrate with 60 years of mission critical codebases, you're making up a problem in your head that doesn't exist. Nothing needs to be fixed, all it takes is to stop doing the stupid thing. It does not take a "coordinated gang" to shut down C standard proposals, them getting shut down is the default. You seem to be neither familiar with the nature of the problem or the struggle that is getting anything passed through ISO standardization. I don't mean to belittle you by saying this, I just hope to make you understand that you are assuming things that are simply not based in reality. It doesn't even need to be in the standard btw. Just write your own. It's a few lines of code. As you say, a beginner exercise. Yet there is code written after the year 2000 that still uses the strxcpy family. Long after the issues have been known and what the solution is. "Backwards compatibility" is a total red herring. C++ has the solution right there in its standard library. A backwards compatible string buffer implementation.
lelanthran 3y ago
> you should only worry about it at the boundary with the other library. If this was a mitigation, it would solve all problems with nul-terminated strings i.e. do strict and error-checked conversions to nul-terminated strings at all boundaries to the program, and then nul-terminated strings and len-specified strings are equivalently dangerous (or safe, depending on your perspective). The problem is precisely that unsanitised input makes its way into the application, bypassing any checks.
- sirwhinesalot 3y ago
  It's impossible to avoid "sanitizing" input if you have a conversion step from a library provided char* to a strbuf type. Any use of the strbuf API is guaranteed to be correct. That's very different from needing to be on your toes with every usage of the strxcpy family.
  - lelanthran 3y ago
    > It's impossible to avoid "sanitizing" input if you have a conversion step from a library provided char* to a strbuf type. Any use of the strbuf API is guaranteed to be correct. I agree: having a datatype beats sanitising input (I think there's a popular essay somewhere about parsing input vs sanitising input which makes pretty much the same point as you do), but it's still only partially correct. To get to fully correct you don't need a new string type, you need developers to recognise that the fields "Full Name" and "Email address" and "Phone number", while all being stored as strings, are actually different types and to handle them as such by making those types incompatible so that a `string_copy` function must produce a compilation failure when the destination is "EmailAddressType" and the source is "FullNameType". Developers in C can, right now, do that with only a few minutes of extra typing effort. Adding a "proper" string type is still going to result in someone, somewhere, parsing a uint8_t from a string into a uint64_t, and then (after some computation) reversing that (now overflowing) uint64_t back into a uint8_t. If you're doing the right thing and creating types because "Parse, Don't Validate", a better string type doesn't bring any benefits. If you're doing the wrong thing and validating inputs, then you're going to miss one anyway, no matter the underlying string type.

jjav 3y ago

> The reason it is not done in C is not ignorance, it is laziness. Of course not. C has been around since the dawn of UNIX and the majority of important libraries at the OS level are written in it. Compatibility with such a vast amount of code is a lot more important than anything else. If it were so easy why do you think nobody has done it? > Ignoring the also obvious solution of just keeping a null terminator around That's not very useful for the general case. If your code relies on the extra metadata (length, size) being correct and you're passing that null-terminated buffer around to libraries outside your code, it won't be correct since nothing else is aware of it.

sirwhinesalot 3y ago
> If it were so easy why do you think nobody has done it? People have done it, there are plenty strbuf implementations to go around. Even the kernel has seq_buf. How you handle string manipulation internally in your codebase does not matter for compatibility with existing libraries. > That's not very useful for the general case. If your code relies on the extra metadata (length, size) being correct and you're passing that null-terminated buffer around to libraries outside your code, it won't be correct since nothing else is aware of it. You can safely pass the char* buffer inside a std::string to any C library with no conversion. You're making up issues in your head. Don't excuse incompetence.
- jjav 3y ago
  > People have done it, there are plenty strbuf implementations to go around. Precisely! Why plenty and why is none of them the standard in C?
  - sirwhinesalot 3y ago
    The TL;DR on that is basically "lazy, security unconscious assholes keep shutting it down". Dennies Ritchie strongly suggested C should add fat pointers all the way back in 1990. Other people have pointed out the issues with zero terminated strings and arrays decaying into pointers (and the ways to deal with them even with backwards compatibility constraints) for years. One of the most prominent was Walter Bright's article on "C's Biggest Mistake" back in 2009 and he was a C/C++ commercial compiler developer. There is no excuse.
    - jjav 3y ago
      It is easy to document mistakes in hindsight, since hindsight is 20/20. It is very easy to write your own one-off secure string handling library. This is a common assignment in intro to C programming classes. So why isn't it standard in C already? You offer a theory that there is a gang of "security unconscious assholes [who] keep shutting it down". This gang is so well organized that they have managed to block an easy improvement for many many decades for unknown reasons. That's a pretty wild theory. Or Occam's razor suggests a different answer: It's actually difficult. No, not the writing code part, that's easy. It's the seamlessly integrating with ~60 years of mission critical codebases part that's hard.
      - sirwhinesalot 3y ago
        There's no need to integrate with 60 years of mission critical codebases, you're making up a problem in your head that doesn't exist. Nothing needs to be fixed, all it takes is to stop doing the stupid thing. It does not take a "coordinated gang" to shut down C standard proposals, them getting shut down is the default. You seem to be neither familiar with the nature of the problem or the struggle that is getting anything passed through ISO standardization. I don't mean to belittle you by saying this, I just hope to make you understand that you are assuming things that are simply not based in reality. It doesn't even need to be in the standard btw. Just write your own. It's a few lines of code. As you say, a beginner exercise. Yet there is code written after the year 2000 that still uses the strxcpy family. Long after the issues have been known and what the solution is. "Backwards compatibility" is a total red herring. C++ has the solution right there in its standard library. A backwards compatible string buffer implementation.

lelanthran 3y ago

> you should only worry about it at the boundary with the other library. If this was a mitigation, it would solve all problems with nul-terminated strings i.e. do strict and error-checked conversions to nul-terminated strings at all boundaries to the program, and then nul-terminated strings and len-specified strings are equivalently dangerous (or safe, depending on your perspective). The problem is precisely that unsanitised input makes its way into the application, bypassing any checks.

sirwhinesalot 3y ago
It's impossible to avoid "sanitizing" input if you have a conversion step from a library provided char* to a strbuf type. Any use of the strbuf API is guaranteed to be correct. That's very different from needing to be on your toes with every usage of the strxcpy family.
- lelanthran 3y ago
  > It's impossible to avoid "sanitizing" input if you have a conversion step from a library provided char* to a strbuf type. Any use of the strbuf API is guaranteed to be correct. I agree: having a datatype beats sanitising input (I think there's a popular essay somewhere about parsing input vs sanitising input which makes pretty much the same point as you do), but it's still only partially correct. To get to fully correct you don't need a new string type, you need developers to recognise that the fields "Full Name" and "Email address" and "Phone number", while all being stored as strings, are actually different types and to handle them as such by making those types incompatible so that a `string_copy` function must produce a compilation failure when the destination is "EmailAddressType" and the source is "FullNameType". Developers in C can, right now, do that with only a few minutes of extra typing effort. Adding a "proper" string type is still going to result in someone, somewhere, parsing a uint8_t from a string into a uint64_t, and then (after some computation) reversing that (now overflowing) uint64_t back into a uint8_t. If you're doing the right thing and creating types because "Parse, Don't Validate", a better string type doesn't bring any benefits. If you're doing the wrong thing and validating inputs, then you're going to miss one anyway, no matter the underlying string type.