8 ms·

I've always wondered at the motivatons of the various string routines in C - every one of them seems to have some huge caveat which makes them useless. After y

• formerly_proven 10mo ago
  strncpy is fairly easy, that's a special-purpose function for copying a C string into a fixed-width string, like typically used in old C applications for on-disk formats. E.g. you might have a char username[20] field which can contain up to 20 characters, with unused characters filled with NULs. That's what strncpy is for. The destination argument should always be a fixed-size char array. A couple years ago we got a new manual page courtesy of Alejandro Colomar just about this: https://man.archlinux.org/man/string_copying.7.en https://man.archlinux.org/man/string_copying.7.en
  • ufo 10mo ago
    A big footgun with strncpy is that the output string may not be null terminated.
    • kccqzy 10mo ago
      Yeah but fixed width strings don’t need null termination. You know exactly how long the string is. No need to find that null byte.
      • ninkendo 10mo ago
        Until you pass them as a `char *` by accident and it eventually makes its way to some code that does expect null termination. There’s languages where you can be quite confident your string will never need null termination… but C is not one of them.
        • deleted 10mo ago
          [deleted]
        • kccqzy 10mo ago
          You don’t do that by accident. Fixed-width strings are thoroughly outdated and unusual. Your mental model of them is very different from regular C strings.
          • arka2147483647 10mo ago
            Sadly, all the bug trackers are full of bugs relating to char*. So you very much do those by accident. And in C, fixed width strings are not in any way rare or unusual. Go to any c codebase you will find stuff like: char buf[12]; sprintf(buf, "%s%s", this, that); // or strcat(buf, ...) // or strncpy(buf, ...) // and so on..
            • snickerbockers 10mo ago
              Thats only really a problem if this and that are coming from an external source and have not been truncated. I really don't see this as any more significant of a problem than all the many high level scripting languages where you can potentially inject code into a variable and interpret it. There are certainly ways in which the c library could've been better (eg making strncpy handle the case where the source string is longer than n) but ultimately it will always need to operate under the assumption that the people using it are both competent and acting in good faith.
            • kccqzy 10mo ago
              When you write such code your mental model is C strings, not fixed-width strings, the intended use case for strncpy.
          • ninkendo 10mo ago
            The mental model doesn’t matter, it’s the compiler’s model that is going to bite you. If the compiler doesn’t reject it, it will happen eventually.
      • Sharlin 10mo ago
        Good luck though remembering not to pass one to any function that does expect to find a null terminator.
        • andrepd 10mo ago
          Seriously. We have type systems and compilers that help us to not forget these things. It's not the 70s anymore!
          • deleted 10mo ago
            [deleted]
        • kevin_thibedeau 10mo ago
          Ignore the prefix and always treat strncpy() as a special binary data operation for an era where shaving bytes on storage was important. It's for copying into a struct with array fields or direct to an encoded block of memory. In that context you will never be dependent on the presence of NUL. The only safe usage with strings is to check for NUL on every use or wrap it. At that point you may as well switch to a new function with better semantics.
          • masklinn 10mo ago
            > an era where shaving bytes on storage was important Fixed size strings don’t save bytes on storage tho, when the bank reserves 20 bytes for first name and you’re called Jon that’s 17 bytes doing fuckall. What they do is make the entire record fixed size and give every field a fixed relative position so it’s very easy to access items, move record around, reuse allocations (or use static allocation), … cycles is what they save.
            • josefx 10mo ago
              > Fixed size strings don’t save bytes on storage tho I have seen plenty of fixed strings in the 8 to 20 byte range, not much, but often enough for a passable identifier. The memory management overhead for a simple dynamically allocated string is probably larger than that even on a 32 bit system.
  • Cyph0n 10mo ago
    strncpy doesn’t handle overlapping buffers (undefined behavior). Better to use strncpy_s (if you can) as it is safer overall. See: https://en.cppreference.com/w/c/string/byte/strncpy.html https://en.cppreference.com/w/c/string/byte/strncpy.html. As an aside, this is part of the reason why there are so many C successor languages: you can end up with undefined behavior if you don’t always carefully read the docs.
    • Asooka 10mo ago
      Back when strncpy was written there was no undefined behaviour (as the compiler interprets it today). The result would depend on the implementation and might differ between invocations, but it was never the "this will not happen" footgun of today. The modern interpretation of undefined behaviour in C is a big blemish on the otherwise excellent standards committee, committed (hah) in the name of extremely dubious performance claims. If "undefined" meaning "left to the implementation" was good enough when CPU frequency was measured in MHz and nobody had more than one, surely it is good enough today too. Also I'm not sure what you mean with C successor languages not having undefined behaviour, as both Rust and Zig inherit it wholesale from LLVM. At least last I checked that was the case, correct me if I am wrong. Go, Java and C# all have sane behaviour, but those are much higher level.
      • Cyph0n 10mo ago
        The problem isn't undefined behavior per se; I was using it as an example for strncpy. Rust is a no - in fact, the goal of (safe) Rust is to eliminate undefined behavior. Zig on the other hand I don't know about. In general, I see two issues at play here: 1. C relies heavily on unsized pointers (vs. fat pointers), which is why strncpy_s had to "break" strncpy in order to improve bounds checks. 2. strncpy memory aliasing restrictions are not encoded in the API and can only be conveyed through docs. This is a footgun. For (1), Rust APIs of this type operate on sized slices, or in the case of strings, string slices. Zig defines strings as sized byte slices. For (2), Rust enforces this invariant via the borrow checker by disallowing (at compile-time) a shared slice reference that points to an overlapping mutable slice reference. In other words, an API like this is simply not possible to define in (safe) Rust, which means you (as the user) do not need to pore over the docs for each stdlib function you use looking for memory-related footguns.
  • dundarious 10mo ago
    Yes, these were also common in several wire formats I had to use for market data/entry. You would think char symbol[20] would be inefficient for such performance sensitive software, but for the vast majority of exchanges, their technical competencies were not there to properly replace these readable symbol/IDs with a compact/opaque integer ID like a u32. Several exchanges tried and they had numerous issues with IDs not being "properly" unique across symbol types, or time (restarts intra-day or shortly before the open were a common nightmare), etc. A char symbol[20] and strncpy was a dream by comparison.
  • dingi 10mo ago
    Isn't strlcpy the safer solution these days?
    • jandrese 10mo ago
      I don't think anybody in this thread read the article. Strlcpy tries to improve the situation but still has problems. As the article points out it is almost never desirable to truncate a string passed into strXcpy, yet that is what all of those functions do. Even worse, they attempt to run to the end of the string regardless of the size parameter so they don't even necessarily save you from the unterminated string case. They also do loads of unnecessary work, especially if your source string is very long (like a mmaped text file). Strncpy got this behavior because it was trying to implement the dubious truncation feature and needed to tell the programmer where their data was truncated. Strlcpy adopted the same behavior because it was trying to be a drop in replacement. But it was a dumb idea from the start and it causes a lot of pain unnecessarily. The crazy thing is that strcpy has the best interface, but of course it's only useful in cases where you have externally verified that the copy is safe before you call it, and as the article points out if you know this then you can just use memcpy instead. As you ponder the situation you inevitably come to the conclusion that it would have been better if strings brought along their own length parameter instead of relying on a terminator, but then you realize that in order to support editing of the string as well as passing substrings you'll need to have some struct that has the base pointer, length, and possibly a substring offset and length and you've just re-invented slices. It's also clear why a system like this was not invented for the original C that was developed on PDP machines with just a few hundred KB of RAM. Is it really too late for the C committee to not develop a modern string library that ships with base C26 or C27? I get that they really hate adding features, but C strings have been a problem for over 50 years now, and I'm not advocating for the old strings to be removed or even deprecated at this time. Just that a modern replacement be available and to encourage people to use them for new code.
      • cyberpunk 10mo ago
        Do they really need to at this point? Just include bstrlib and stop thinking about it?
• lesuorac 10mo ago
  It's from a time before computer viruses no? But also all of this book-keeping takes up extra time and space which is a trade-off easily made nowadays.
  • rini17 10mo ago
    Yes, in the old times if you crashed a program or whole computer with invalid input, it was your fault. Viruses did exist, and these were considered users' fault too.
• alexfoo 10mo ago
  Yet software developed in C, with all of the foibles of its string routines, has been sold and running for years with trillions of USD is total sales. A library that records how much memory is allocated to a string along with the pointer isn't a necessity. Most people who write in C professionally are completely used to it although the footgun is (and all of the others are) always there lurking. You'd generally just see code like this:- char hostname[20]; ... strncpy( hostname, input, 20 ); hostname[19]=0; The problem obviously comes if you forget the line to NUL that last byte AND you have a input that is greater than 19 characters long. (It's also very easy to get this wrong, I almost wrote `hostname[20]=0;` first time round.) I remember debugging a problem 20+ years ago on a customer site with some software that used Sybase Open/Server that was crashing on startup. The underlying TDS communications protocol (https://www.freetds.org/tds.html https://www.freetds.org/tds.html) had a fixed 30 byte field for the hostname and the customer had a particularly long FQDN that was being copied in without any checks on its length. An easy fix once identified. Back then though the consequences of a buffer overrun were usually just a mild annoyance like a random crash or something like the Morris worm. Nowadays such a buffer overrun is deadly serious as it can easily lead to data exfiltration, an RCE and/or a complete compromise. Heartbleed and Mongobleed had nothing to do with C string functions. They were both caused by trusting user supplied payload lengths. (C string functions are still a huge source of problems though.)
  • __float 10mo ago
    > Yet software developed in C, with all of the foibles of its string routines, has been sold and running for years with trillions of USD is total sales. This doesn't seem very relevant. The same can be said of countless other bad APIs: see years of bad PHP, tons of memory safety bugs in C, and things that have surely led to significant sums of money lost. > It's also very easy to get this wrong, I almost wrote `hostname[20]=0;` first time round. Why would you do this separately every single time, then? The problem with bad APIs is that even the best programmers will occasionally make a mistake, and you should use interfaces (or...languages!) that prevent it from happening in the first place. The fact we've gotten as far as we have with C does not mean this is a defensible API.
    • 10mo ago
• jerf 10mo ago
  Yes, not having a length along with the string was a mistake. It dates from an era where every byte was precious and the thought of having two bytes instead of one for length was a significant loss. I have long wondered how terrible it would have been to have some sort of "varint" at the beginning instead of a hard-coded number of bytes, but I don't have enough experience with that generation to have a good feel for it.
• Someone 10mo ago
  > I've always wondered at the motivatons of the various string routines in C This idiom: char hostname[20]; ... strncpy( hostname, input, 20 ); hostname[19]=0; exists because strncpy was invented for copying file names that got stored in 14-byte arrays, zero terminated only if space permitted (https://stackoverflow.com/a/1454071 https://stackoverflow.com/a/1454071)
  • masklinn 10mo ago
    Technically strncpy was invented to interact with null-padded fixed-size strings in general. We’ve mostly (though not entirely) moved away from them but fixed-size strings used to be very common. You can see them all over old file formats still.
  • BobbyTables2 10mo ago
    It’s also horrible because each project ends up reinventing their own abstractions or solutions for dealing with common things. Destroys a lot of opportunity for code reuse / integration. Especially within a company. Alternatively their code base remains a steaming pile of crap riddled with vulnerabilities.
    • delaminator 10mo ago
      That's how everything works. You start off with some atomics and build up from there. Things that people like get standardized, And before you know what's going on it's called stdlib. It took a decade between Stroustrup's 1985 book "The C++ Programming Language" and the STL proposed and accepted by the ANSI/ISO committee in 1994.
      • BobbyTables2 10mo ago
        Yeah but those standard libraries are still inadequate 30 years later. Imagine a hypothetical Python scenario where every application and library had their own list implementations with differently named methods and behavior. Yet C is exactly that. Is it not common place to log messages to a file? How many applications and code bases DO NOT use syslog(). And if one wants to use syslog, then assert failures won’t get logged there… Sure, with a custom assertion and logging framework one can get reasonable behavior. But it’s not automatic. Hence, a Tower of Babel …
• CupricTea 10mo ago
  >every one of them seems to have some huge caveat which makes them useless They were added into C before enough of the people designing it knew the consequences they would bring. Another fundamentally broken oversight is array-to-pointer demotion in function signatures instead of having fat pointer types.
• Gibbon1 10mo ago
  I remember some story that early on they thought function calls had little overhead and so used lots of little functions. And then found that actually no they were spending a lot of time doing function calls. But turned out no one cared. To me the string library looks like small snippets of early leet C code hoisted into a library.