4 ms·

shrug It doesn't fall over. I've done it, the openBSD team has done it. DJB has done it. Maybe something is wrong with your implementation that I can help you w

shrug It doesn't fall over. I've done it, the openBSD team has done it. DJB has done it. Maybe something is wrong with your implementation that I can help you with?

WalterBright 9y ago
I'm curious. Got links?
- ktRolster 9y ago
  OpenBSD takes a fairly minimalist approach, which is vaguely described here: http://www.freebsdforums.org/forums/showthread.php?threadid=9019 http://www.freebsdforums.org/forums/showthread.php?threadid=... They basically replace the unsafe functions with things that are easier to use. Their idea is that it isn't the format of the C-string that causes security issues (null-terminated string), it's the poorly defined functions (with weird corner cases that are hard to get right). It's worked well for their use cases. DJB did something similar in qmail, I don't recall the details but you can look at the source code as easily as I can, and it eliminated security problems. When I'm working in Java, I find that most of my string parsing uses the split() function. This is a pain in C, because even if you had a split() function you'd need to deal with memory allocations. Most of these are solved with a memory pool. In my own library, I also added runtime, grammar-based parsing functionality. So to parse a CSV line you might do something like this: char *g = " S -> WORD | WORD , S;" "WORD -> [^,]"; results = parsegram(g, inputString); Grammar parsing + memory pools makes string parsing in C easier than in Java. The biggest difficulty with this kind of library is to do it right, you need to be something of a unicode expert, and that's tough.
  - WalterBright 9y ago
    I used snprintf(), too, but it is only a minor improvement. Problematic in C is something as simple as concatenating strings: Mystring s,t; t = "hello"; t = cat(s,s); t = cat(s,s,s); t = cat("hello",s); t = cat(s,"world"); t = cat("hello","world"); Even such a simple use case is fraught with major problems: 1. who allocates needed memory? 2. who free's it? 3. can the compiler constant fold cat("hello","world") ? Does the result wind up allocating memory anyway? 4. what about the lack of function overloading to handle the permutations?
    - ktRolster 9y ago
      I assume you're referring to OpenBSD here, they didn't use snprintf(). They used asnprintf(), which solves the problem of who should allocate (but not who should free).
      - WalterBright 9y ago
        From the link: "That means that we have been going through the tree cleaning out all calls to sprintf(), strcpy(), and strcat(). Instead, these things are being rewritten to use asprintf(), snprintf(), strlcpy(), and strlcat()." Maybe the author made a typo.
        ktRolster 9y ago
        Oh yeah, you're right. Another thing I've done that will work if you have a lot of strcat(), is make a string struct: ktString { int len; int memlen; char *str; } It keeps track of the string's actual length, and the size of the underlying buffer. Then you can 'override' the various string functions: bool ktStrcat(ktString s1, ktString s2); bool ktSprintf(ktString s1, ...); These functions will take care of buffer-size checking, and reallocation if necessary. For cases where you need to interface with pre-existing libraries, you can return a cstring(). Make it a function/macro to enable you to change the struct definition in the future: #define ktCstr(x) (x)->str then you can pass it into write() or whatever you need: write(sock, ktCstr(s), s->len);
        lomnakkus 9y ago
        ... and end up with silent truncation unless you happen to always remember to use only C library functions with explicit length arguments (and which do not assume NUL-terminated strings). Look, I get that there is a place for C, but string manipulation is absurdly bad and error-prone.
        ktRolster 9y ago
        Hi! I can't imagine how you understood what I wrote. I specifically said to not use those C library string functions. I fully admitted that string manipulation is absurdly bad and error-prone, then built on that by showing a way to make it better. Use ktStrcat() instead of strCat(), then you don't have to worry about truncation. Use ktSprintf() instead of snprintf(), then you don't have to worry about truncation. I wish you had understood.
deleted 9y ago
[deleted]

shrug It doesn't fall over. I've done it, the openBSD team has done it. DJB has done it. Maybe something is wrong with your implementation that I can help you with?

WalterBright 9y ago

I'm curious. Got links?

ktRolster 9y ago
OpenBSD takes a fairly minimalist approach, which is vaguely described here: http://www.freebsdforums.org/forums/showthread.php?threadid=9019 http://www.freebsdforums.org/forums/showthread.php?threadid=... They basically replace the unsafe functions with things that are easier to use. Their idea is that it isn't the format of the C-string that causes security issues (null-terminated string), it's the poorly defined functions (with weird corner cases that are hard to get right). It's worked well for their use cases. DJB did something similar in qmail, I don't recall the details but you can look at the source code as easily as I can, and it eliminated security problems. When I'm working in Java, I find that most of my string parsing uses the split() function. This is a pain in C, because even if you had a split() function you'd need to deal with memory allocations. Most of these are solved with a memory pool. In my own library, I also added runtime, grammar-based parsing functionality. So to parse a CSV line you might do something like this: char *g = " S -> WORD | WORD , S;" "WORD -> [^,]"; results = parsegram(g, inputString); Grammar parsing + memory pools makes string parsing in C easier than in Java. The biggest difficulty with this kind of library is to do it right, you need to be something of a unicode expert, and that's tough.
- WalterBright 9y ago
  I used snprintf(), too, but it is only a minor improvement. Problematic in C is something as simple as concatenating strings: Mystring s,t; t = "hello"; t = cat(s,s); t = cat(s,s,s); t = cat("hello",s); t = cat(s,"world"); t = cat("hello","world"); Even such a simple use case is fraught with major problems: 1. who allocates needed memory? 2. who free's it? 3. can the compiler constant fold cat("hello","world") ? Does the result wind up allocating memory anyway? 4. what about the lack of function overloading to handle the permutations?
  - ktRolster 9y ago
    I assume you're referring to OpenBSD here, they didn't use snprintf(). They used asnprintf(), which solves the problem of who should allocate (but not who should free).
    - WalterBright 9y ago
      From the link: "That means that we have been going through the tree cleaning out all calls to sprintf(), strcpy(), and strcat(). Instead, these things are being rewritten to use asprintf(), snprintf(), strlcpy(), and strlcat()." Maybe the author made a typo.
      - ktRolster 9y ago
        Oh yeah, you're right. Another thing I've done that will work if you have a lot of strcat(), is make a string struct: ktString { int len; int memlen; char *str; } It keeps track of the string's actual length, and the size of the underlying buffer. Then you can 'override' the various string functions: bool ktStrcat(ktString s1, ktString s2); bool ktSprintf(ktString s1, ...); These functions will take care of buffer-size checking, and reallocation if necessary. For cases where you need to interface with pre-existing libraries, you can return a cstring(). Make it a function/macro to enable you to change the struct definition in the future: #define ktCstr(x) (x)->str then you can pass it into write() or whatever you need: write(sock, ktCstr(s), s->len);
        lomnakkus 9y ago
        ... and end up with silent truncation unless you happen to always remember to use only C library functions with explicit length arguments (and which do not assume NUL-terminated strings). Look, I get that there is a place for C, but string manipulation is absurdly bad and error-prone.
        ktRolster 9y ago
        Hi! I can't imagine how you understood what I wrote. I specifically said to not use those C library string functions. I fully admitted that string manipulation is absurdly bad and error-prone, then built on that by showing a way to make it better. Use ktStrcat() instead of strCat(), then you don't have to worry about truncation. Use ktSprintf() instead of snprintf(), then you don't have to worry about truncation. I wish you had understood.

deleted 9y ago

[deleted]