3 ms·

It's important to note that standards other than C can place constraints on C implementations and make things that are "undefined" according to ANSI C perfectly

It's important to note that standards other than C can place constraints on C implementations and make things that are "undefined" according to ANSI C perfectly valid in specific environments. For example, POSIX now mandates that the NULL pointer has all zero bits: http://austingroupbugs.net/view.php?id=940 http://austingroupbugs.net/view.php?id=940

wahern 10y ago
The representation of types is either unspecified or implementation-defined in C, depending on the context. See C11 6.2.6; also see Annex J. Relying on particular representations does not make your program per se invalid C. Reliance on undefined behavior does make it invalid. Representations are readily observable because the standard permits all objects to be cast to char type. Therefore the standard basically says that you can read and even modify objects through a char array as long as you obey house rules. So POSIX isn't changing what undefined means by mandating that CHAR_BIT is 8, or that null pointers can be represented as all-zero bits. By contrast, I think the definition of dlsym() does depend on undefined behavior because C does not permit casting function pointers to void pointers or vice-versa. For various reasons that's a significant deficiency. Keeping that behavior undefined is intended to, among other things, preserve the ability to implement tagged pointers in a standards compliant manner, or to even create implementations for Harvard architecture machines. Also, I think the distinction between undefined behavior and unspecified or implementation-defined behavior preserves the ability to implement practical, quality static analyzers that can analyze all C code, whether or not dependent on certain implementation-specific behaviors. Some amount of license is a practical necessity; too much makes the standard useless. C permits assignment of pointers to functions of incompatible types as long as the functions aren't invoked. But mixing pointers to functions and pointers to objects is a definite no-no. So POSIX will probably add a sister to dlsym that returns a generic function pointer type like (void (*)(void)).
- quotemstr 10y ago
  > So POSIX will probably add a sister to dlsym that returns a generic function pointer type like (void (*)(void)). I think that change would be a mistake: it makes no practical difference on the machines for which POSIX is intended, so this sister function would bloat the API only to satisfy pedantic concerns that don't really matter. Besides, POSIX systems have to keep the traditional dlsym around for compatibility, which further erodes the advantage of your proposed function. Right now, POSIX mandates by implication that interconversation of function and data pointers works, and I think that's a perfectly reasonable state of affairs. There is nothing wrong with POSIX constraining the C standard.
  - wahern 10y ago
    A year ago I'd probably agree with you. But since then I've researched various avenues of software hardening and I think the future of C programming will begin to increasingly leverage the restriction. For example, C++ class vtables are a prime target for heap overflow attacks because they store function pointers on the heap. And with the anti-ROP measures coming around the corner it's a target that will be increasingly used to create and execute a ROP chain. So, for example, I can easily imagine new compiler modes and coding styles that outright ban casting function pointers to object pointers, and ban casting pointers to function pointers to other object pointers. PaX and W^X don't help you here as the issue isn't writing over code, but writing over pointers to code. Banning those conversions is neither sufficient nor necessary, but nonetheless a very plausible and practical counter-measure. In that context dlsym sticks out like a sore thumb. One of the easiest solutions is to just standardize a routine for returning a function pointer. That's arguably better than forcing compilers to understand the semantics of dlsym. Note that implementations can still support mixing pointer types, and POSIX could even still require it so existing software can continue using dlsym the same way. But it would simplify things for the tooling and runtimes that enforce the distinction. Given that it makes POSIX more consistent with the C standard, it's a win-win.
    - quotemstr 10y ago
      > C++ class vtables are a prime target for heap overflow attacks because they store function pointers on the heap Is that so? The actual pointers should be in .data. The pointer to the vtable is on the heap (or on the stack, or wherever your object lives). In any case, you should be able to use something like [1] to place constraints on indirect calls, right? There are lots of approaches to double checking function calls that don't involve increasing function pointer size. Besides, there are lots of bits of code that depend on function- and data-pointer interconvertibility --- people can stuff function pointers in raw void* cookies to callbacks. Rather than make the effort to make existing code work in this (in practice, new) subset of customary (as opposed to standard) C, wouldn't it be better to just write components in a language that's memory-safe to start with? [1] https://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=get&target=cmtice.pdf https://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=g...
jmgao 10y ago
Strictly speaking, POSIX mandates that a pointer with all zero bits is a null pointer, not necessarily the other way around.

wahern 10y ago

The representation of types is either unspecified or implementation-defined in C, depending on the context. See C11 6.2.6; also see Annex J. Relying on particular representations does not make your program per se invalid C. Reliance on undefined behavior does make it invalid. Representations are readily observable because the standard permits all objects to be cast to char type. Therefore the standard basically says that you can read and even modify objects through a char array as long as you obey house rules. So POSIX isn't changing what undefined means by mandating that CHAR_BIT is 8, or that null pointers can be represented as all-zero bits. By contrast, I think the definition of dlsym() does depend on undefined behavior because C does not permit casting function pointers to void pointers or vice-versa. For various reasons that's a significant deficiency. Keeping that behavior undefined is intended to, among other things, preserve the ability to implement tagged pointers in a standards compliant manner, or to even create implementations for Harvard architecture machines. Also, I think the distinction between undefined behavior and unspecified or implementation-defined behavior preserves the ability to implement practical, quality static analyzers that can analyze all C code, whether or not dependent on certain implementation-specific behaviors. Some amount of license is a practical necessity; too much makes the standard useless. C permits assignment of pointers to functions of incompatible types as long as the functions aren't invoked. But mixing pointers to functions and pointers to objects is a definite no-no. So POSIX will probably add a sister to dlsym that returns a generic function pointer type like (void (*)(void)).

quotemstr 10y ago
> So POSIX will probably add a sister to dlsym that returns a generic function pointer type like (void (*)(void)). I think that change would be a mistake: it makes no practical difference on the machines for which POSIX is intended, so this sister function would bloat the API only to satisfy pedantic concerns that don't really matter. Besides, POSIX systems have to keep the traditional dlsym around for compatibility, which further erodes the advantage of your proposed function. Right now, POSIX mandates by implication that interconversation of function and data pointers works, and I think that's a perfectly reasonable state of affairs. There is nothing wrong with POSIX constraining the C standard.
- wahern 10y ago
  A year ago I'd probably agree with you. But since then I've researched various avenues of software hardening and I think the future of C programming will begin to increasingly leverage the restriction. For example, C++ class vtables are a prime target for heap overflow attacks because they store function pointers on the heap. And with the anti-ROP measures coming around the corner it's a target that will be increasingly used to create and execute a ROP chain. So, for example, I can easily imagine new compiler modes and coding styles that outright ban casting function pointers to object pointers, and ban casting pointers to function pointers to other object pointers. PaX and W^X don't help you here as the issue isn't writing over code, but writing over pointers to code. Banning those conversions is neither sufficient nor necessary, but nonetheless a very plausible and practical counter-measure. In that context dlsym sticks out like a sore thumb. One of the easiest solutions is to just standardize a routine for returning a function pointer. That's arguably better than forcing compilers to understand the semantics of dlsym. Note that implementations can still support mixing pointer types, and POSIX could even still require it so existing software can continue using dlsym the same way. But it would simplify things for the tooling and runtimes that enforce the distinction. Given that it makes POSIX more consistent with the C standard, it's a win-win.
  - quotemstr 10y ago
    > C++ class vtables are a prime target for heap overflow attacks because they store function pointers on the heap Is that so? The actual pointers should be in .data. The pointer to the vtable is on the heap (or on the stack, or wherever your object lives). In any case, you should be able to use something like [1] to place constraints on indirect calls, right? There are lots of approaches to double checking function calls that don't involve increasing function pointer size. Besides, there are lots of bits of code that depend on function- and data-pointer interconvertibility --- people can stuff function pointers in raw void* cookies to callbacks. Rather than make the effort to make existing code work in this (in practice, new) subset of customary (as opposed to standard) C, wouldn't it be better to just write components in a language that's memory-safe to start with? [1] https://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=get&target=cmtice.pdf https://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=g...

jmgao 10y ago

Strictly speaking, POSIX mandates that a pointer with all zero bits is a null pointer, not necessarily the other way around.