9 ms·

The decision to make C strings null terminated with implied length instead of length + blob continues to trip us up, 30+ years later. There's a good reason the

• jrimbault 6y ago
  30+ years -> 50+ years Funny mind thing to forget to increment counters each year.
  • segf4ult 6y ago
    C89 was 32 years ago, so I think saying 30+ years is fair.
    • lamontcg 6y ago
      Some of us learned C off of the original K&R book.
• Blikkentrekker 6y ago
  > But also, "strings" and "time" are actually very complex concepts, and these functions operate on often outdated assumptions about those underlying abstractions. Even in safer languages such as Rust, there are often quæstions as to why certain string operations are either impossible, or need to be quite complicated for a rather simple operation and are then met with responses such as “*Did you know that the length of a string can grow from a capitalization operation depending on locale settings of environment variables? P.s.: In fact, I would argue that strings are not necessarily all that complicated, but simply that many assume that they are simpler than they are, and that code that handles them is thus written on such assumptions that the length of a string remain the same after capitalization, or that the result not be under influence of environment variables.
  • munchbunny 6y ago
    > locale settings of environment variables Also known as "why does my code that parses floats fail in Turkey?" Also also known as the discrepancy between a string's length-as-in-bytes, its length-as-in-code-points, and its length-as-in-how-humans-count-glyphs. Strings are hard. Edit to respond to your addendum: > P.s.: In fact, I would argue that strings are not necessarily all that complicated, but simply that many assume that they are simpler than they are, and that code that handles them is thus written on such assumptions that the length of a string remain the same after capitalization, or that the result not be under influence of environment variables. I don't think I agree with that, though we may just be disagreeing on semantics. I think the big mistake many of us make is confusing two different abstractions for the same one. We've got this high level abstraction for "text" that includes issues like locale and encoding and several other things. And then we've got this low level abstraction for "text" that is just a blob of bytes. And we often mix the abstractions because it often turns out okay anyway. Otherwise we have to confront demons like "a UTF-8 string containing 10 characters can be anywhere between 10 and 40 bytes long".
    • kazinator 6y ago
      > Why does my code that parses floats fail in Turkey Because you, or someone, called fuck_my_program(); which is defined in "idiot.h" as #define fuck_my_program() setlocale(LC_ALL, "") and the project is missing: #define setlocale(x, y) BANNED(setlocale) Hope that helps!
      • account42 6y ago
        I agreee with you but it would be much better if every locale-dependent function had an argument for an explicit locale object.
        • kazinator 6y ago
          The problem is you'd have to pass an annoying extra argument (even if just a NULL) to numerous functions which have no alternative without that argument. Technically it would be better, especially from a multi-threading point of view. The locale stuff was designed in the 1980's, before multi-threading was a mainstream technique. Say you have a multi-threaded global server which has to localize something in the context of a session, to the locale of the user making the request. Still, for thread support, you don't necessarily need a cluttering argument. The locale can be made into a thread-specific variable. In Lisp I would almost certainly prefer for the local to be a dynamic variable. (It would be pretty silly to be passing an argument to influence whether he decimal point is a comma, while the radix of integers is being controlled by *print-base*.) What you want is for the locale stuff to be broken out into a complete separate library: a whole separate set of loc_* functions: loc_strtod, loc_printf, and so on.*
• retrac 6y ago
  For reasons that were never clearly articulated, the prefix approach was considered odd, backwards, and to have numerous downsides, at least where I learned C. In hindsight, I can only cringe at that attitude. Strings as added in later Pascal, about 40 years ago now, were memory safe in a way that C strings still are not.
  • JoeAltmaier 6y ago
    Hey, languages used length,blob even when C was invented. HP Access BASIC used that kind. It was a limitation, because they chose a byte length (to save space). So strings up to 255 characters only. It was decades before folks were comfortable with 32-bit length fields. And that still limited you to 4GB strings. In the bad old days, memory usage was king.
    • selfhoster11 6y ago
      The funny thing is that you can just use the topmost bit of the length to indicate that the string length is >127, and chain as many length bytes as you want before you begin the string proper (to save space). It would be still a better encoding than a null at the end.
      • deleted 6y ago
        [deleted]
      • michaf 6y ago
        This way you would trade in a null-byte-terminated variable length string for essentially a null-bit-terminated variable length number (plus the remaining string). I am not convinced that this actually would be much safer.
        • pertymcpert 6y ago
          This is just the LEB128 format, which is used commonly used and I don't think there's any serious problems with it.
• kazinator 6y ago
  The reason that the safe functions take length parameters is that they produce a new object in uninitialized memory, a pointer to which is specified by the caller. It has nothing to do with null termination. And that uninitialized memory is not self-describing in any way in the C language. Which is that way in machine language also. This is a problem you have to bootstrap yourself somehow if you are to have any higher level language. The machine just gives you a way to carve out blocks of memory that don't know their own type or size. C doesn't improve on that, but it is not the root cause of the situation. Without C, you still have to somehow go from that chaos to order. Copying two null terminated strings into an existing null-terminated string can be perfectly safe without any size parameters. void replace_str(char *dest_str, const char *src_left, const char *src_right); If dest_str is a string of 17 characters, we know we have 18 bytes in which to catenate src_left and src_right. This is not very useful though. Now what might be a bit more useful would be if dest_str had two sizes: the length of string currently stored in it, and the size of the underlying storage. This particular operation would ignore the former, and use the latter. It could replace a string of three characters with a 27 character one.
• coliveira 6y ago
  Null terminated strings are remnants of an era when computers had little memory available. So, at the time it seemed smart to discard the length field and use a single byte-sized terminator (null). If you are writing an operating system for a machine with little memory to spare, this seems like a good decision. Of course things are very different now when memory is not a problem and the goal is safety.
  • a1369209993 6y ago
    > If you are writing an operating system for a machine with little memory to spare, this seems like a good decision. Also registers! Especially in syscall interface, consider eg: int renameat(int olddirfd,char* oldpath,int newdirfd,char* newpath); /* first example I found that had 2 paths */ If you have registers edx,ecx,esi,edi,ebx available, nul-terminated strings make this fit into: edx olddirfd,ecx oldpath, esi newdirfd,edi newpath If you need separate length fields, there simply aren't enough registers: edx olddirfd,ecx oldpath.ptr,esi oldpath.len, edi newdirfd,ebx newpath.ptr,??? newpath.len
  • scoutt 6y ago
    > memory is not a problem and the goal is safety. People keep repeating this. What about embedded systems? For instance, I have to know how an object is structured and how is allocated, exactly and without surprises. The behavior has to be predictable and as simple and fast as possible. You can (likely) achieve that with C.
• kiwidrew 6y ago
  I would argue that C's fundamental mistake (well, more like limitation due to hardware of the time) was allowing arrays to decay to pointers; arrays hold valuable type information (the length!) that is lost once converted to a pointer. C99 came so very very close with VLAs. You can declare a function like: int main(int argc, char *argv[argc]) { ... } But C99 requires the compiler to discard the type annotations and treat the declaration as equivalent to: int main(int argc, char **argv) { ... } Imagine a world where the C string functions were declared as: char *strndup(s, n) const char *s[n]; size_t n; { /* now we can do sizeof(s) and bounds checking! */ } (You'd have to use K&R style declarations to get around the fact that the pointer argument comes before the length argument, alas.) Edit: and then C11 made VLA support optional, since the feature didn't get used much, because the feature was only half-baked to begin with... sigh.
  • moonchild 6y ago
    It's because to get that to actually work you need dependent types. Which—it's not gonna happen.
    • kiwidrew 6y ago
      But the caller to strnlen() has already provided both the (pointer to the) array and the length! Note that C99 does permit declaring a VLA in the body of the function: char *strndup(size_t n; const char *s[n], size_t n) { char buf[n]; /* alloc a temporary VLA */ assert(sizeof(buf) == n); /* yep! */ assert(sizeof(s) == n); /* nope, sizeof(s) == 1 */ } So there's absolutely no reason (other than being in violation of the C99 specification) for the compiler to refuse to let you make the assertion that sizeof(s) == n. And given the prototype for this VLA-enhanced strndup(), a smart C compiler could catch errors like this: char * bugged_func() { char buf[20]; /* do stuff with buf, e.g. snprintf() into it */ return strndup(buf, 30); /* error: 30 > sizeof(buf) */ } Since of course within a function the C type system is already tracking the size of an array -- so no additional type information is required, and certainly not dependent types!
      • 6y ago
• m463 6y ago
  I think 30-40 years ago it was perfectly appropriate to null-terminate strings. Every byte actually counted. I remember thinking about setting the high bit to denote the end of string to save space. Nowadays the binary for "hello world" might be as big as a whole operating system of the past. (though honestly I can't recall the size of the OS on a boot floppy, but the original floppies were 160k)
  • tdeck 6y ago
    Just a surreal reminder that 30 years ago was 1991. By that time it already didn't make sense to null terminate strings (except perhaps on embedded platforms).
    • munchbunny 6y ago
      To be fair, in 1991, 1mb of RAM was a substantial amount for a PC, so a single word per string instead of a byte per string could still add up quickly.