5 ms·

> Some code practices are what we now consider to be terrible, optimizing for the limitations of that time. This. If you went back to 1972 when Dennis Ritchie

> Some code practices are what we now consider to be terrible, optimizing for the limitations of that time. This. If you went back to 1972 when Dennis Ritchie was working on C and said "String literals should have an extra machine word for their length and strings should have yet another machine word for the capacity of their buffer" you'd be considered a moron for wasting so much memory.

mavhc 3y ago
Depends if you managed to explain why
ajross 3y ago
Not really. C was an iconoclast even at the time. Pascal was the en vogue language of the moment, and it used a length-prefixed string format. But sure: it's true that in (a half century of!) hindsight, C strings were probably a mistake. But don't sell null-terminated strings short either. C could play tricks that Pascal couldn't. Iterating over the characters of a string has a natural expression using the same compiler features as arrays. A pointer into the middle of a string was still a valid string. You could lex a string inline during parsing by adding termination to the end markers or whitespace, etc...
- Joker_vD 3y ago
  So you just don't store the array's length near the arrays beginning but instead inside the slice-typed variable which lives somewhere else entirely. Boom, you got the best of the both worlds: trivial slicing and reliable bounds checking.
  - ajross 3y ago
    > instead inside the slice-typed variable which lives somewhere else entirely. OK... where? Now you have a complicated heap-like semantic inside your compiler internals. But not everyone wants their string metadata in the heap. So now you need allocator semantics a-la C++, which ultimately leads to move semantics, etc... Good luck getting that done in 1972. No, DMR was right, Pascal was wrong, and fancy modern string semantics were still two decades in the future. C was the best it could be given the constraints of the era.
    - Joker_vD 3y ago
      > OK... where? In the automatic storage, duh, where everybody else puts them today. > But not everyone wants their string metadata in the heap. Exactly, so you don't put them there, which is what I am proposing: instead of putting the string length before, or the null delimiter after, the string itself, you put it near the pointer that points to the beginning of the (sub)string.
      - ajross 3y ago
        > you put it near the pointer that points to the beginning of the (sub)string. And now you have a "string pointer" which is distinct from a "data pointer". You can't allocate a heap block and "put a string in it" because the special thing at the start needs to go with the pointer and not the data in the block. And your 1970's compiler on your PDP-11 with 48kb of RAM needs to manage that. Good luck to you. Again, we're not answering the question "Can a better string implementation than C be designed?". Clearly the answer is yes. Go pick your favorite, there are literally dozens. We're asking "What else should Richie have done with C that would have been better?". And... it's not what you're suggesting for sure.
        Joker_vD 3y ago
        > And now you have a "string pointer" which is distinct from a "data pointer". You can't allocate a heap block and "put a string in it" because the special thing at the start needs to go with the pointer and not the data in the block. The second sentence doesn't follow from the first one. struct string { char (*ptr)[static len]; size_t len; }; struct string new_copy = { .ptr = malloc(100), .len = 100; }; new_copy = strcpy(new_copy, "some other string"); // string literal is a syntactic sugar for statically allocated 'struct string' with 'ptr' set to an unnamed, statically allocated char[] buffer inside it, and proper 'len' field. struct string strcpy(struct string dest, struct string src) { struct string result = { .ptr = dest.ptr, .len = min(dest.len, src.len) }; memcpy(result.ptr, src.ptr, result.len); return result; } Or you can do struct string *new_copy = malloc(100 + sizeof(struct string)); new_copy->ptr = (char*)(new_copy + 1); new_copy->len = 100; if you really want to. Just don't pass naked "char*" around, that thing has no obvious extent.
- knorker 3y ago
  Pascal strings had the same size overhead as C strings. One using the "byte" for length, the other for size.
  - elteto 3y ago
    But C strings could at least be arbitrarily long, Pascal strings were extremely handicapped. It’s like they understood why length prefix is better, but then picked the worst possible implementation.
    - compiler-guy 3y ago
      Pascal strings were a bad implementation of general strings, but were extraordinarily easy to implement.
    - chinabot 3y ago
      screens were generally 80 characters wide 256 was good enough, if you needed to go bigger null terminated could still be done in pascal, you just had to roll your own. The editors I wrote back in the eighties were just linked lists of pascal strings so a line was limited to 256 characters but these were very fast in VMS and DOS Also we only had a few character sets and you could live your whole life in ASCII
  - compiler-guy 3y ago
    And worse for pascal, a 256 byte limit on the length and api shenanigans so that not every string allocated needed to consume all that memory if the string itself was shorter.
derefr 3y ago
> so much memory Back then "runtime strings" — as in, temporary heap allocations that were allocated to hold onto a bit of text separately from the buffer it originated from — were extremely rare. Tools that manipulated text in Unix C, weren't copying strings out of buffers onto the heap, holding onto them in data structures, and then later passing them piecewise to write(2). Instead, they were read(2)ing fixed-sized chunks of text from STDIN to a data-section-preallocated input buffer; running a resumable stream-processing state-machine (e.g. a lexer) over that input buffer to feed a transformation step; potentially building a result into a (again data-section-preallocated) output buffer; and then emitting either directly from the transformation step, or from the output buffer, using write(2). This was the real "innovation" of Unix: you don't need so much memory, or so much copying, if your tools can work in terms of streams of characters. You just need two static arrays and four pointers per process, plus whatever per-line book-keeping state your state-machine uses. It's like each Unix tool is a virtual implementation of a hardware DSP! (And this is also why so many Unix tools are so weird. tr(1), for example, would never have arisen as the precisely-designed solution to anyone's particular problem; but tr(1) fits in perfectly as an "obvious" primitive in a toolkit of command-line DSPs.) Note that NUL-delimiting "strings" makes perfect sense when most "strings" aren't runtime strings, but rather are "pieces of text living inside a large static char-array buffer they were either directly read(2) into, or strcpy(2)ed into." The NUL isn't supposed to tell you the end of the buffer; the NUL is supposed to tell you where that individual string ends within the buffer — and so where you'll then find either the next string, or garbage. Nobody was calling strlen(2) + malloc(2) + strcpy(2). char-pointers mainly existed to be the C equivalent of Go slices — i.e. to keep track of some text living inside a larger buffer. If you ever malloc(2) or free(2) anything, it's the containing fixed-sized buffer, not the string! I'm not sure if many Unix tool implementations (e.g. GNU coreutils) these days retain this architectural philosophy; but you can still clearly see the remnants of it in the places where Unixisms became ossified parts of the C ABI. For example, while ARGV and ENVP are these days developer-visible as char[][], the C ABI requires these to be backed by a pair of contiguous NUL-delimited and double-NUL-terminated buffers that get fed into the process's address space by exec(2). In original Unix C, it's these raw buffers themselves that were simply passed directly to `main`; and it was the developer's job to parse flags and env-vars out of them, if they wished — not by pre-strcpy(2)ing data out, but rather by simply running a flags-parser directly over the entire buffer, reacting to each flag as it was observed within the buffer. (And in fact, on many Unixes, getenv(2) is still implemented as a streaming search through the underlying char-array buffer, because it's simply more CPU-efficient and cache-coherent than repeatedly indirecting through the developer-visible array-of-pointers.)

mavhc 3y ago

Depends if you managed to explain why

ajross 3y ago

Not really. C was an iconoclast even at the time. Pascal was the en vogue language of the moment, and it used a length-prefixed string format. But sure: it's true that in (a half century of!) hindsight, C strings were probably a mistake. But don't sell null-terminated strings short either. C could play tricks that Pascal couldn't. Iterating over the characters of a string has a natural expression using the same compiler features as arrays. A pointer into the middle of a string was still a valid string. You could lex a string inline during parsing by adding termination to the end markers or whitespace, etc...

Joker_vD 3y ago
So you just don't store the array's length near the arrays beginning but instead inside the slice-typed variable which lives somewhere else entirely. Boom, you got the best of the both worlds: trivial slicing and reliable bounds checking.
- ajross 3y ago
  > instead inside the slice-typed variable which lives somewhere else entirely. OK... where? Now you have a complicated heap-like semantic inside your compiler internals. But not everyone wants their string metadata in the heap. So now you need allocator semantics a-la C++, which ultimately leads to move semantics, etc... Good luck getting that done in 1972. No, DMR was right, Pascal was wrong, and fancy modern string semantics were still two decades in the future. C was the best it could be given the constraints of the era.
  - Joker_vD 3y ago
    > OK... where? In the automatic storage, duh, where everybody else puts them today. > But not everyone wants their string metadata in the heap. Exactly, so you don't put them there, which is what I am proposing: instead of putting the string length before, or the null delimiter after, the string itself, you put it near the pointer that points to the beginning of the (sub)string.
    - ajross 3y ago
      > you put it near the pointer that points to the beginning of the (sub)string. And now you have a "string pointer" which is distinct from a "data pointer". You can't allocate a heap block and "put a string in it" because the special thing at the start needs to go with the pointer and not the data in the block. And your 1970's compiler on your PDP-11 with 48kb of RAM needs to manage that. Good luck to you. Again, we're not answering the question "Can a better string implementation than C be designed?". Clearly the answer is yes. Go pick your favorite, there are literally dozens. We're asking "What else should Richie have done with C that would have been better?". And... it's not what you're suggesting for sure.
      - Joker_vD 3y ago
        > And now you have a "string pointer" which is distinct from a "data pointer". You can't allocate a heap block and "put a string in it" because the special thing at the start needs to go with the pointer and not the data in the block. The second sentence doesn't follow from the first one. struct string { char (*ptr)[static len]; size_t len; }; struct string new_copy = { .ptr = malloc(100), .len = 100; }; new_copy = strcpy(new_copy, "some other string"); // string literal is a syntactic sugar for statically allocated 'struct string' with 'ptr' set to an unnamed, statically allocated char[] buffer inside it, and proper 'len' field. struct string strcpy(struct string dest, struct string src) { struct string result = { .ptr = dest.ptr, .len = min(dest.len, src.len) }; memcpy(result.ptr, src.ptr, result.len); return result; } Or you can do struct string *new_copy = malloc(100 + sizeof(struct string)); new_copy->ptr = (char*)(new_copy + 1); new_copy->len = 100; if you really want to. Just don't pass naked "char*" around, that thing has no obvious extent.
knorker 3y ago
Pascal strings had the same size overhead as C strings. One using the "byte" for length, the other for size.
- elteto 3y ago
  But C strings could at least be arbitrarily long, Pascal strings were extremely handicapped. It’s like they understood why length prefix is better, but then picked the worst possible implementation.
  - compiler-guy 3y ago
    Pascal strings were a bad implementation of general strings, but were extraordinarily easy to implement.
  - chinabot 3y ago
    screens were generally 80 characters wide 256 was good enough, if you needed to go bigger null terminated could still be done in pascal, you just had to roll your own. The editors I wrote back in the eighties were just linked lists of pascal strings so a line was limited to 256 characters but these were very fast in VMS and DOS Also we only had a few character sets and you could live your whole life in ASCII
- compiler-guy 3y ago
  And worse for pascal, a 256 byte limit on the length and api shenanigans so that not every string allocated needed to consume all that memory if the string itself was shorter.

derefr 3y ago

> so much memory Back then "runtime strings" — as in, temporary heap allocations that were allocated to hold onto a bit of text separately from the buffer it originated from — were extremely rare. Tools that manipulated text in Unix C, weren't copying strings out of buffers onto the heap, holding onto them in data structures, and then later passing them piecewise to write(2). Instead, they were read(2)ing fixed-sized chunks of text from STDIN to a data-section-preallocated input buffer; running a resumable stream-processing state-machine (e.g. a lexer) over that input buffer to feed a transformation step; potentially building a result into a (again data-section-preallocated) output buffer; and then emitting either directly from the transformation step, or from the output buffer, using write(2). This was the real "innovation" of Unix: you don't need so much memory, or so much copying, if your tools can work in terms of streams of characters. You just need two static arrays and four pointers per process, plus whatever per-line book-keeping state your state-machine uses. It's like each Unix tool is a virtual implementation of a hardware DSP! (And this is also why so many Unix tools are so weird. tr(1), for example, would never have arisen as the precisely-designed solution to anyone's particular problem; but tr(1) fits in perfectly as an "obvious" primitive in a toolkit of command-line DSPs.) Note that NUL-delimiting "strings" makes perfect sense when most "strings" aren't runtime strings, but rather are "pieces of text living inside a large static char-array buffer they were either directly read(2) into, or strcpy(2)ed into." The NUL isn't supposed to tell you the end of the buffer; the NUL is supposed to tell you where that individual string ends within the buffer — and so where you'll then find either the next string, or garbage. Nobody was calling strlen(2) + malloc(2) + strcpy(2). char-pointers mainly existed to be the C equivalent of Go slices — i.e. to keep track of some text living inside a larger buffer. If you ever malloc(2) or free(2) anything, it's the containing fixed-sized buffer, not the string! I'm not sure if many Unix tool implementations (e.g. GNU coreutils) these days retain this architectural philosophy; but you can still clearly see the remnants of it in the places where Unixisms became ossified parts of the C ABI. For example, while ARGV and ENVP are these days developer-visible as char[][], the C ABI requires these to be backed by a pair of contiguous NUL-delimited and double-NUL-terminated buffers that get fed into the process's address space by exec(2). In original Unix C, it's these raw buffers themselves that were simply passed directly to `main`; and it was the developer's job to parse flags and env-vars out of them, if they wished — not by pre-strcpy(2)ing data out, but rather by simply running a flags-parser directly over the entire buffer, reacting to each flag as it was observed within the buffer. (And in fact, on many Unixes, getenv(2) is still implemented as a streaming search through the underlying char-array buffer, because it's simply more CPU-efficient and cache-coherent than repeatedly indirecting through the developer-visible array-of-pointers.)