5 ms·

char foo[9]; strcpy(foo,"abc"); strcat(foo,"1234"); And yes, that's specifically written to be awful and unsafe, but there are circumstances where you need to

char foo[9]; strcpy(foo,"abc"); strcat(foo,"1234"); And yes, that's specifically written to be awful and unsafe, but there are circumstances where you need to be close to the metal and carefully resort to more complicated variations of such things. That's what C is fairly uniquely appropriate for.

WalterBright 8y ago
Your example uses three strlen's. It also is at high risk for buffer overflows. Whenever I review code like this, sure as shootin', there's an error in it in the lengths somewhere. Here's the same code using dynamic arrays: char foo[9]; foo[] = "abc"; foo[3..3+5] = "1234"; No unchecked buffer overflows, and no calls to strlen. The +5 puts the terminating 0 on.
- drfuchs 8y ago
  You seem to be confused about the implementation of C's standard string library. There are certainly not 3 strlen calls underneath a strcpy() plus a strcat() call. But what's the mention of "terminating 0"? The article says that terminating zeros should not be needed under its proposal; and that's what I was saying didn't make sense. [Added:] So, if you didn't just happen to know that the global string variable foo contains a string that was 3 characters long, how would you concatenate "1234" to it? I don't see any way without either double-fat pointers, or terminating NUL.
  - vardump 8y ago
    > There are certainly not 3 strlen calls underneath a strcpy() plus a strcat() call. No, but strcpy does need to check each char for NUL byte as it copies the string. And strcat will need to redo same check on the original strcpy'ied string plus the new one. So "abc" length is effectively checked twice and "1234" once. So there's some truth to the matter, even though they're not "true" strlen calls.
  - WalterBright 8y ago
    strcpy(s1,s2) does a strlen on s2. strcat(s1,s2) does a strlen on s1 and s2. Now, two of the strlen's can be replaced with byte-by-byte copies checking for 0 for each, but that tends to lose the efficiency that a memcpy would bring, so you're pretty much suffering from it anyway. BTW, here's the strcat I wrote eons ago: https://github.com/DigitalMars/dmc/blob/master/src/CORE32/STRCAT.ASM https://github.com/DigitalMars/dmc/blob/master/src/CORE32/ST... It does do two strlen's (the repne scasb instructions). With the improvements in CPUs since there are probably better ways to write it, but that was pretty good for its day. Here's strcpy: https://github.com/DigitalMars/dmc/blob/master/src/CORE32/STRFUNC.C#L194 https://github.com/DigitalMars/dmc/blob/master/src/CORE32/ST... which does the test-every-byte method. I think Steve Russell wrote it, but I'm not sure. If there's anything efficiently implemented in a C compiler, it's memcpy. Being able to implement string processing in terms of memcpy leverages that very nicely. strcat() and strcpy() don't leverage it. Which do you think is faster (s2 is 1024 bytes long)? strcpy(s1, s2); memcpy(s1, s2, 1024); I've dramatically speeded up a lot of my code and other peoples' by replacing the strxxx functions with memcpy. It's low hanging fruit and one of the first things I look for.
    - shakna 8y ago
      I'm usually one to lean on your knowledge, but looking at glibc's strcpy [0], I don't see a strlen. I do see two macros, CHECK_BOUNDS_LOW and CHECK_BOUNDS_HIGH, which get defined about here [1]. Am I missing something? [0] https://github.com/lattera/glibc/blob/master/string/strcpy.c https://github.com/lattera/glibc/blob/master/string/strcpy.c [1] https://github.com/lattera/glibc/blob/master/sysdeps/generic/bp-checks.h#L28 https://github.com/lattera/glibc/blob/master/sysdeps/generic...
      - WalterBright 8y ago
        Good question. I preface this by the fact that I am not any expert on gcc compiler internals, I am assuming how I'd do it. 1. This implementation tests every byte, as discussed in other posts here. That makes it slow. 2. This implementation is likely not used - the gcc compiler probably has an internal code sequence it emits for a strcpy.
      - 8y ago
  - WalterBright 8y ago
    The "terminating 0" is when you need it, for example when interfacing with the C standard library. D, for example, adds a terminating 0 to string literals so that they'll work with C code. For code that uses slices, the 0 is not necessary.
    - drfuchs 8y ago
      So, if you don't just happen to know that the global variable "char foo[9]" currently contains a string that is 3 characters long, how would you concatenate "1234" to it in place? I don't see any way without either double-fat pointers, or terminating NUL. Non-dynamic arrays of char is just supposed to be a simplistic representation of a sort of thing that one has occasion to want to do in C that doesn't seem to fit into the proposed model without going into "old C NUL-termination" mode, or a "keep track of the string's length yourself" scheme, either of which would seem to ruin the whole thing. Thus my claim that this single feature would be hard to graft onto C in a useful, upward-compatible way. It's fine to have a language where all strings are dynamically allocated on the heap, or have an immutable known length, but that's a non-starter in the existing C universe. The point isn't that doing things D's way isn't great; the point is that there's no reasonable way to put this feature into C. Every reasonable approach to string (and pointer) safety ends up being a new language: C#, Swift, Java, etc.
      - WalterBright 8y ago
        You're trying to require this to manage memory allocation. It does not address memory allocation any more than C pointers do. Memory allocation remains up to the programmer with this proposal.
      - jibal 8y ago
        " I don't see any way without either double-fat pointers, or terminating NUL." You don't seem to have followed the discussion or the paper. The array length is known; a slice is a "fat pointer", but not "double-fat". "The point isn't that doing things D's way isn't great; the point is that there's no reasonable way to put this feature into C." You're plainly wrong; the proposal in the article does exactly that.
  - deleted 8y ago
    [deleted]
WalterBright 8y ago
Your example does save on code size (but not performance, due to the 3 strlen()'s). But where it won't save on code size is when, say, one wants to retrieve just the filename from filename.ext. One has to strlen it, allocate memory, copy the memory, then remember to free it exactly once. With dynamic arrays, just return a slice.
- kyberias 8y ago
  You say strlen() but you probably mean "code has to iterate all the string elements". This doesn't happen 3 times, though: > char foo[9]; strcpy(foo,"abc"); strcat(foo,"1234"); Strcpy has to iterate through all the elements because it copies them. This would happen regardless. It doesn't do strlen. Strcat has to find the end of the destination string, so it has to iterate (or call strlen). Then it's just strcpy again. Instead of 3 strlens, there is 1. Do you not understand how C string/arrays work, or why do you insist on 3 strlens?
  - WalterBright 8y ago
    See my other reply to this above.
  - tjoff 8y ago
    > Strcpy has to iterate through all the elements because it copies them. This would happen regardless. It doesn't do strlen. Iterate all the elements and copy a fixed size is two very different things. strcpy has to read, byte for byte, and check it for null (which is exactly what strlen does). A "real" copy would just blindly copy a chunk of memory with no other processing on it. The speed difference is huge.
    - WalterBright 8y ago
      You're right. I apologize for using strlen metaphorically rather than literally.
  - jibal 8y ago
    > You say strlen() but you probably mean "code has to iterate all the string elements". Yes, but that's much more expensive than a memcpy (of the two source strings) or just knowing the length (of the string in the target buffer). > Strcpy has to iterate through all the elements because it copies them. This would happen regardless. No, it wouldn't; memcpy is generally a lot faster than strcpy. > Do you not understand how C string/arrays work Do you not understand that he's written a few C compilers, and designed and implemented a language that is known for its runtime compatibility with C? > or why do you insist on 3 strlens? Um, you already noted that he really means "code has to iterate all the string elements". What he should have said is that you have to find 3 NULs. That's an expensive operation even when you're copying the string while finding it.
Too 8y ago
Ok, so pre-allocating then. That's sort of equivalent to new StringBuffer(9), except java uses the heap. Safe types doesn't stop you from doing this. You do need another length-field though, one for the allocated size and one for the used size. In c++, std::string already has this feature with the reserve/capacity functions, STL is also heap based but it is possible with some effort to pass it a stack allocator. Now c++ isn't exactly the best reference when it comes to these things either but just saying conceptually fat pointers doesn't stop you from doing these things, see WalterBrights reply for a better example.
bumholio 8y ago
> there are circumstances where you need to be close to the metal and carefully resort to more complicated variations of such things. That's what C is fairly uniquely appropriate for. You can still do that in a C dialect with fat pointers. You can have strings with 5 byte chars and use 0x1337 as string terminator if that's what your metal needs. The point is, you don't have to, the compiler provides you with a sane array implementation that is adequate for 99% higher level algorithmic tasks.

WalterBright 8y ago

Your example uses three strlen's. It also is at high risk for buffer overflows. Whenever I review code like this, sure as shootin', there's an error in it in the lengths somewhere. Here's the same code using dynamic arrays: char foo[9]; foo[] = "abc"; foo[3..3+5] = "1234"; No unchecked buffer overflows, and no calls to strlen. The +5 puts the terminating 0 on.

drfuchs 8y ago
You seem to be confused about the implementation of C's standard string library. There are certainly not 3 strlen calls underneath a strcpy() plus a strcat() call. But what's the mention of "terminating 0"? The article says that terminating zeros should not be needed under its proposal; and that's what I was saying didn't make sense. [Added:] So, if you didn't just happen to know that the global string variable foo contains a string that was 3 characters long, how would you concatenate "1234" to it? I don't see any way without either double-fat pointers, or terminating NUL.
- vardump 8y ago
  > There are certainly not 3 strlen calls underneath a strcpy() plus a strcat() call. No, but strcpy does need to check each char for NUL byte as it copies the string. And strcat will need to redo same check on the original strcpy'ied string plus the new one. So "abc" length is effectively checked twice and "1234" once. So there's some truth to the matter, even though they're not "true" strlen calls.
- WalterBright 8y ago
  strcpy(s1,s2) does a strlen on s2. strcat(s1,s2) does a strlen on s1 and s2. Now, two of the strlen's can be replaced with byte-by-byte copies checking for 0 for each, but that tends to lose the efficiency that a memcpy would bring, so you're pretty much suffering from it anyway. BTW, here's the strcat I wrote eons ago: https://github.com/DigitalMars/dmc/blob/master/src/CORE32/STRCAT.ASM https://github.com/DigitalMars/dmc/blob/master/src/CORE32/ST... It does do two strlen's (the repne scasb instructions). With the improvements in CPUs since there are probably better ways to write it, but that was pretty good for its day. Here's strcpy: https://github.com/DigitalMars/dmc/blob/master/src/CORE32/STRFUNC.C#L194 https://github.com/DigitalMars/dmc/blob/master/src/CORE32/ST... which does the test-every-byte method. I think Steve Russell wrote it, but I'm not sure. If there's anything efficiently implemented in a C compiler, it's memcpy. Being able to implement string processing in terms of memcpy leverages that very nicely. strcat() and strcpy() don't leverage it. Which do you think is faster (s2 is 1024 bytes long)? strcpy(s1, s2); memcpy(s1, s2, 1024); I've dramatically speeded up a lot of my code and other peoples' by replacing the strxxx functions with memcpy. It's low hanging fruit and one of the first things I look for.
  - shakna 8y ago
    I'm usually one to lean on your knowledge, but looking at glibc's strcpy [0], I don't see a strlen. I do see two macros, CHECK_BOUNDS_LOW and CHECK_BOUNDS_HIGH, which get defined about here [1]. Am I missing something? [0] https://github.com/lattera/glibc/blob/master/string/strcpy.c https://github.com/lattera/glibc/blob/master/string/strcpy.c [1] https://github.com/lattera/glibc/blob/master/sysdeps/generic/bp-checks.h#L28 https://github.com/lattera/glibc/blob/master/sysdeps/generic...
    - WalterBright 8y ago
      Good question. I preface this by the fact that I am not any expert on gcc compiler internals, I am assuming how I'd do it. 1. This implementation tests every byte, as discussed in other posts here. That makes it slow. 2. This implementation is likely not used - the gcc compiler probably has an internal code sequence it emits for a strcpy.
    - 8y ago
- WalterBright 8y ago
  The "terminating 0" is when you need it, for example when interfacing with the C standard library. D, for example, adds a terminating 0 to string literals so that they'll work with C code. For code that uses slices, the 0 is not necessary.
  - drfuchs 8y ago
    So, if you don't just happen to know that the global variable "char foo[9]" currently contains a string that is 3 characters long, how would you concatenate "1234" to it in place? I don't see any way without either double-fat pointers, or terminating NUL. Non-dynamic arrays of char is just supposed to be a simplistic representation of a sort of thing that one has occasion to want to do in C that doesn't seem to fit into the proposed model without going into "old C NUL-termination" mode, or a "keep track of the string's length yourself" scheme, either of which would seem to ruin the whole thing. Thus my claim that this single feature would be hard to graft onto C in a useful, upward-compatible way. It's fine to have a language where all strings are dynamically allocated on the heap, or have an immutable known length, but that's a non-starter in the existing C universe. The point isn't that doing things D's way isn't great; the point is that there's no reasonable way to put this feature into C. Every reasonable approach to string (and pointer) safety ends up being a new language: C#, Swift, Java, etc.
    - WalterBright 8y ago
      You're trying to require this to manage memory allocation. It does not address memory allocation any more than C pointers do. Memory allocation remains up to the programmer with this proposal.
    - jibal 8y ago
      " I don't see any way without either double-fat pointers, or terminating NUL." You don't seem to have followed the discussion or the paper. The array length is known; a slice is a "fat pointer", but not "double-fat". "The point isn't that doing things D's way isn't great; the point is that there's no reasonable way to put this feature into C." You're plainly wrong; the proposal in the article does exactly that.
- deleted 8y ago
  [deleted]

WalterBright 8y ago

Your example does save on code size (but not performance, due to the 3 strlen()'s). But where it won't save on code size is when, say, one wants to retrieve just the filename from filename.ext. One has to strlen it, allocate memory, copy the memory, then remember to free it exactly once. With dynamic arrays, just return a slice.

kyberias 8y ago
You say strlen() but you probably mean "code has to iterate all the string elements". This doesn't happen 3 times, though: > char foo[9]; strcpy(foo,"abc"); strcat(foo,"1234"); Strcpy has to iterate through all the elements because it copies them. This would happen regardless. It doesn't do strlen. Strcat has to find the end of the destination string, so it has to iterate (or call strlen). Then it's just strcpy again. Instead of 3 strlens, there is 1. Do you not understand how C string/arrays work, or why do you insist on 3 strlens?
- WalterBright 8y ago
  See my other reply to this above.
- tjoff 8y ago
  > Strcpy has to iterate through all the elements because it copies them. This would happen regardless. It doesn't do strlen. Iterate all the elements and copy a fixed size is two very different things. strcpy has to read, byte for byte, and check it for null (which is exactly what strlen does). A "real" copy would just blindly copy a chunk of memory with no other processing on it. The speed difference is huge.
  - WalterBright 8y ago
    You're right. I apologize for using strlen metaphorically rather than literally.
- jibal 8y ago
  > You say strlen() but you probably mean "code has to iterate all the string elements". Yes, but that's much more expensive than a memcpy (of the two source strings) or just knowing the length (of the string in the target buffer). > Strcpy has to iterate through all the elements because it copies them. This would happen regardless. No, it wouldn't; memcpy is generally a lot faster than strcpy. > Do you not understand how C string/arrays work Do you not understand that he's written a few C compilers, and designed and implemented a language that is known for its runtime compatibility with C? > or why do you insist on 3 strlens? Um, you already noted that he really means "code has to iterate all the string elements". What he should have said is that you have to find 3 NULs. That's an expensive operation even when you're copying the string while finding it.

Too 8y ago

Ok, so pre-allocating then. That's sort of equivalent to new StringBuffer(9), except java uses the heap. Safe types doesn't stop you from doing this. You do need another length-field though, one for the allocated size and one for the used size. In c++, std::string already has this feature with the reserve/capacity functions, STL is also heap based but it is possible with some effort to pass it a stack allocator. Now c++ isn't exactly the best reference when it comes to these things either but just saying conceptually fat pointers doesn't stop you from doing these things, see WalterBrights reply for a better example.

bumholio 8y ago

> there are circumstances where you need to be close to the metal and carefully resort to more complicated variations of such things. That's what C is fairly uniquely appropriate for. You can still do that in a C dialect with fat pointers. You can have strings with 5 byte chars and use 0x1337 as string terminator if that's what your metal needs. The point is, you don't have to, the compiler provides you with a sane array implementation that is adequate for 99% higher level algorithmic tasks.