6 ms·

I am sorry I do not have an answer to your question. It's a very valid one and I would be interested in any pointer to an answer. What I can say while we are o

I am sorry I do not have an answer to your question. It's a very valid one and I would be interested in any pointer to an answer. What I can say while we are on the subject, is that I have seen C code (most often C code that started its life in the 1990s, to be fair) that instead of showing an abstract struct in the public interface, showed a different struct definition. Please don't do this. Yes, when compiling nowadays, eventually every compilation unit ends up as object files passed to a linker that doesn't know about types, but this is undefined behavior. It makes it difficult to find undefined behavior in the rest of the code because there is a big undefined behavior right in the middle of it.

beefhash 6y ago
Wait, doesn't this mean that the BSD sockets API is inherently dependent on UB, casing different socket types to each other and sometimes only using the first few members, or am I misunderstanding you?
- pascal_cuoq 6y ago
  Yes and no. The thing I am describing is when you link a compilation unit using: struct internal_state { int dummy; } state; with another compilation unit that defined the same state differently: struct internal_state { int actual_meaningful_member_1; unsigned long actual_meaningful_member_2; } state; As far as I know, BSD socked do not do this. Zlib was doing this (https://github.com/pascal-cuoq/zlib-fork/blob/a52f0241f72433b69fd558100a32d927d9571e20/zlib.h#L1740 https://github.com/pascal-cuoq/zlib-fork/blob/a52f0241f72433... ), but I have had the privilege of discussing this with Mark Adler, and I think the no-longer-necessary hack was removed from Zlib. BSD sockets probably have a different kind of UB, related to so-call “strict aliasing” rules, unless they have been carefully audited and revised since the carefree times in which they were written. I am going to have to let you read this article for details (example st1, page 5): https://trust-in-soft.com/wp-content/uploads/2017/01/vmcai.pdf https://trust-in-soft.com/wp-content/uploads/2017/01/vmcai.p...
  - loeg 6y ago
    BSD sockets are weird in that the first struct's (sockaddr) size wasn't big enough, so APIs all take a nominal pointer to sockaddr but may require larger storage (sockaddr_storage) depending on the actual address. /* * Structure used by kernel to store most * addresses. */ struct sockaddr { unsigned char sa_len; /* total length */ sa_family_t sa_family; /* address family */ char sa_data[14]; /* actually longer; address value */ }; /* * RFC 2553: protocol-independent placeholder for socket addresses */ #define _SS_MAXSIZE 128U #define _SS_ALIGNSIZE (sizeof(__int64_t)) #define _SS_PAD1SIZE (_SS_ALIGNSIZE - sizeof(unsigned char) - \ sizeof(sa_family_t)) #define _SS_PAD2SIZE (_SS_MAXSIZE - sizeof(unsigned char) - \ sizeof(sa_family_t) - _SS_PAD1SIZE - _SS_ALIGNSIZE) struct sockaddr_storage { unsigned char ss_len; /* address length */ sa_family_t ss_family; /* address family */ char __ss_pad1[_SS_PAD1SIZE]; __int64_t __ss_align; /* force desired struct alignment */ char __ss_pad2[_SS_PAD2SIZE]; };
    - wahern 6y ago
      struct sockaddr_storage is insufficient as well. A Unix domain socket path can be longer than `sizeof ((struct sockaddr_un){ 0}).sun_path`. That's a major reason why all the socket APIs take a separate socklen_t argument. Most people just assume that a domain socket path is limited to a relatively short string, but it's not (except possibly Minix, IIRC).
      - asveikau 6y ago
        > A Unix domain socket path can be longer than `sizeof ((struct sockaddr_un){ 0}).sun_path` Hm, I didn't realize this, or if I knew this I had forgotten. It makes sense because sun_path is usually pretty small, I believe 108 chars is the most common choice, and typically file paths are allowed to be much longer. Do you have a citation for this behavior? I can't seem to find it, though I'm not looking very hard. I guess you are right that any syscall taking a struct sockaddr * also has a length passed to it... Some systems have sa_len inside struct sockaddr to indicate length, but IIRC linux does not. I've often thought that length parameter was sort of redundant, because (1) some platforms have sa_len, and (2) even without that, you should be able to derive length from family. But your Unix domain socket example breaks (2). Without being able to do that, I start to imagine that the kernel would need to probe for NUL chars terminating the C string anytime it inspects a struct sockaddr_un, rather than block-copying the expected size of the structure -- that would be needlessly complicated.
- emilfihlman 6y ago
  Yeah, it depends on well agreed convention but which is ub according to the standard.
- loeg 6y ago
  Yeah, the BSD socket API is kind of terrible like that. You could consider it an unspecified union type, or use memcpy() exclusively to access it safely.
rmind 6y ago
I assume you mean something like that: struct obj_impl { // real members ... }; In public API header: struct obj { unsigned char _private[N]; // -- where N is the size of obj_impl }; I have seen such code too. It is also potentially error-prone. Certainly not advocating for it.

beefhash 6y ago

Wait, doesn't this mean that the BSD sockets API is inherently dependent on UB, casing different socket types to each other and sometimes only using the first few members, or am I misunderstanding you?

pascal_cuoq 6y ago
Yes and no. The thing I am describing is when you link a compilation unit using: struct internal_state { int dummy; } state; with another compilation unit that defined the same state differently: struct internal_state { int actual_meaningful_member_1; unsigned long actual_meaningful_member_2; } state; As far as I know, BSD socked do not do this. Zlib was doing this (https://github.com/pascal-cuoq/zlib-fork/blob/a52f0241f72433b69fd558100a32d927d9571e20/zlib.h#L1740 https://github.com/pascal-cuoq/zlib-fork/blob/a52f0241f72433... ), but I have had the privilege of discussing this with Mark Adler, and I think the no-longer-necessary hack was removed from Zlib. BSD sockets probably have a different kind of UB, related to so-call “strict aliasing” rules, unless they have been carefully audited and revised since the carefree times in which they were written. I am going to have to let you read this article for details (example st1, page 5): https://trust-in-soft.com/wp-content/uploads/2017/01/vmcai.pdf https://trust-in-soft.com/wp-content/uploads/2017/01/vmcai.p...
- loeg 6y ago
  BSD sockets are weird in that the first struct's (sockaddr) size wasn't big enough, so APIs all take a nominal pointer to sockaddr but may require larger storage (sockaddr_storage) depending on the actual address. /* * Structure used by kernel to store most * addresses. */ struct sockaddr { unsigned char sa_len; /* total length */ sa_family_t sa_family; /* address family */ char sa_data[14]; /* actually longer; address value */ }; /* * RFC 2553: protocol-independent placeholder for socket addresses */ #define _SS_MAXSIZE 128U #define _SS_ALIGNSIZE (sizeof(__int64_t)) #define _SS_PAD1SIZE (_SS_ALIGNSIZE - sizeof(unsigned char) - \ sizeof(sa_family_t)) #define _SS_PAD2SIZE (_SS_MAXSIZE - sizeof(unsigned char) - \ sizeof(sa_family_t) - _SS_PAD1SIZE - _SS_ALIGNSIZE) struct sockaddr_storage { unsigned char ss_len; /* address length */ sa_family_t ss_family; /* address family */ char __ss_pad1[_SS_PAD1SIZE]; __int64_t __ss_align; /* force desired struct alignment */ char __ss_pad2[_SS_PAD2SIZE]; };
  - wahern 6y ago
    struct sockaddr_storage is insufficient as well. A Unix domain socket path can be longer than `sizeof ((struct sockaddr_un){ 0}).sun_path`. That's a major reason why all the socket APIs take a separate socklen_t argument. Most people just assume that a domain socket path is limited to a relatively short string, but it's not (except possibly Minix, IIRC).
    - asveikau 6y ago
      > A Unix domain socket path can be longer than `sizeof ((struct sockaddr_un){ 0}).sun_path` Hm, I didn't realize this, or if I knew this I had forgotten. It makes sense because sun_path is usually pretty small, I believe 108 chars is the most common choice, and typically file paths are allowed to be much longer. Do you have a citation for this behavior? I can't seem to find it, though I'm not looking very hard. I guess you are right that any syscall taking a struct sockaddr * also has a length passed to it... Some systems have sa_len inside struct sockaddr to indicate length, but IIRC linux does not. I've often thought that length parameter was sort of redundant, because (1) some platforms have sa_len, and (2) even without that, you should be able to derive length from family. But your Unix domain socket example breaks (2). Without being able to do that, I start to imagine that the kernel would need to probe for NUL chars terminating the C string anytime it inspects a struct sockaddr_un, rather than block-copying the expected size of the structure -- that would be needlessly complicated.
emilfihlman 6y ago
Yeah, it depends on well agreed convention but which is ub according to the standard.
loeg 6y ago
Yeah, the BSD socket API is kind of terrible like that. You could consider it an unspecified union type, or use memcpy() exclusively to access it safely.

rmind 6y ago

I assume you mean something like that: struct obj_impl { // real members ... }; In public API header: struct obj { unsigned char _private[N]; // -- where N is the size of obj_impl }; I have seen such code too. It is also potentially error-prone. Certainly not advocating for it.