5 ms·

Easiest way to send data over the wire. struct mytype s; s.field1 = something; s.field2 = something else; send(socket, &s, sizeof(s), 0) or, using

Easiest way to send data over the wire. struct mytype s; s.field1 = something; s.field2 = something else; send(socket, &s, sizeof(s), 0) or, using a language with a good type system like Haskell data MyData = MyData Int Int deriving Generic instance Storable MyData alloca $ \buf -> do let d = MyData field1 field2 poke buf d send socket buf (sizeof d)

ericpauley 8y ago
C doesn't make struct memory layout guarantees, endian guarantees, or size guarantees in some cases, so unless you're running the exact same binary this is a very poor serialization technique.
- tathougies 8y ago
  C does not, but few compilers implement standard C. Most C compilers do have documentation on how structs are laid out and many options to change the behavior. Most C compilers document this.
amluto 8y ago
This may be easy, but it’s wrong. Endianness issues are just the start. Information leaks due to padding are a big deal. And it straight up doesn’t work if nontrivial data structures are involved.
- tathougies 8y ago
  Endianness issues are something every programmer should be aware of when sending data over the wire. I'm sorry I didn't insert the htons, htonls in my C code. In the Haskell code, endianness is handled by your Storable implementation (you define it after all, and can customize it however you want). I agree my example is somewhat tongue in cheek. The point though is that most languages have standard libraries for dealing with binary data that are almost universally more straightforward than proto bufs.
  - amluto 8y ago
    htons, etc are very 1980s, and I’ll make a fairly strong claim: they should never be used in new code, with a single exception. The reason is that an int with network endianness simply should not exist. In other words, when someone sends you a four byte big-endian integer, they sent four bytes, not an int. You can turn it into an int by shifting each byte by the relevant amount and oring them together. And a modern compiler will generate good code. The sole exception is legacy APIs like inet_aton() that actually require these nonsensical conversions.
    - tathougies 8y ago
      You could also use the functions with explicit bit widths, like bswap64 and bswap32.
      - amluto 8y ago
        No, you should not. By the time you have a wrong-endian uint64_t or whatever, you’ve already done it wrong.
        tathougies 8y ago
        After re-reading your comment above, I'm actually confused. You think you should never store a big-endian int? That is ridiculous. Some architectures are big-endian. You should not be using custom bitswapping as part of application code, because you cannot know the endianness of your architecture. The ntoh* functions are the right approach, and your claim is not only strong, it's wrong. The ntoh* functions exist to transform network byte-order to host byte-order. Depending on your architecture endianness, their functionality will change.
        amluto 8y ago
        Let me try saying it differently. The following code is poorly written: char *buf = ...; uint32_t word = *(int32_t *)buf; uint32_t host_word = ntohl(word); Because you just type-punned the read from buf. (In fact, this code is UB.) You could write it a little better like: char *buf = ...; uint32_t word; memcpy(&word, buf, 4); uint32_t host_word = ntohl(word); Although IIRC there is or at least was still some disagreement as to whether this might be UB. You could use a union to make it definitely not UB. But none of these variants are sensible, and, in fact, they don't even translate to most safer languages than C. The correct way to write this code is: char *buf = ...; uint32_t host_word = ((uint32_t)buf[0] << 24) | ((uint32_t)buf[1] << 16) | ((uint32_t)buf[2] << 8) | (uint32_t)buf[3]; On any recent compiler, this will generate as good or better code, and it doesn't make pointless assumptions about the representation of uint32_t on the platform you're using. So I stand by my claim: well-written modern C code should not contain any "network-order" values. They should contain bytes, vectors of bytes, and numbers.

ericpauley 8y ago

C doesn't make struct memory layout guarantees, endian guarantees, or size guarantees in some cases, so unless you're running the exact same binary this is a very poor serialization technique.

tathougies 8y ago
C does not, but few compilers implement standard C. Most C compilers do have documentation on how structs are laid out and many options to change the behavior. Most C compilers document this.

amluto 8y ago

This may be easy, but it’s wrong. Endianness issues are just the start. Information leaks due to padding are a big deal. And it straight up doesn’t work if nontrivial data structures are involved.

tathougies 8y ago
Endianness issues are something every programmer should be aware of when sending data over the wire. I'm sorry I didn't insert the htons, htonls in my C code. In the Haskell code, endianness is handled by your Storable implementation (you define it after all, and can customize it however you want). I agree my example is somewhat tongue in cheek. The point though is that most languages have standard libraries for dealing with binary data that are almost universally more straightforward than proto bufs.
- amluto 8y ago
  htons, etc are very 1980s, and I’ll make a fairly strong claim: they should never be used in new code, with a single exception. The reason is that an int with network endianness simply should not exist. In other words, when someone sends you a four byte big-endian integer, they sent four bytes, not an int. You can turn it into an int by shifting each byte by the relevant amount and oring them together. And a modern compiler will generate good code. The sole exception is legacy APIs like inet_aton() that actually require these nonsensical conversions.
  - tathougies 8y ago
    You could also use the functions with explicit bit widths, like bswap64 and bswap32.
    - amluto 8y ago
      No, you should not. By the time you have a wrong-endian uint64_t or whatever, you’ve already done it wrong.
      - tathougies 8y ago
        After re-reading your comment above, I'm actually confused. You think you should never store a big-endian int? That is ridiculous. Some architectures are big-endian. You should not be using custom bitswapping as part of application code, because you cannot know the endianness of your architecture. The ntoh* functions are the right approach, and your claim is not only strong, it's wrong. The ntoh* functions exist to transform network byte-order to host byte-order. Depending on your architecture endianness, their functionality will change.
        amluto 8y ago
        Let me try saying it differently. The following code is poorly written: char *buf = ...; uint32_t word = *(int32_t *)buf; uint32_t host_word = ntohl(word); Because you just type-punned the read from buf. (In fact, this code is UB.) You could write it a little better like: char *buf = ...; uint32_t word; memcpy(&word, buf, 4); uint32_t host_word = ntohl(word); Although IIRC there is or at least was still some disagreement as to whether this might be UB. You could use a union to make it definitely not UB. But none of these variants are sensible, and, in fact, they don't even translate to most safer languages than C. The correct way to write this code is: char *buf = ...; uint32_t host_word = ((uint32_t)buf[0] << 24) | ((uint32_t)buf[1] << 16) | ((uint32_t)buf[2] << 8) | (uint32_t)buf[3]; On any recent compiler, this will generate as good or better code, and it doesn't make pointless assumptions about the representation of uint32_t on the platform you're using. So I stand by my claim: well-written modern C code should not contain any "network-order" values. They should contain bytes, vectors of bytes, and numbers.