7 ms·
Rust gets this right. These primitives are available for all the numeric types. u32::from_le_byte(bytes) // u32 from 4 bytes, little endian u32::from_b
by Animats 6y ago
Rust gets this right. These primitives are available for all the numeric types.
u32::from_le_byte(bytes) // u32 from 4 bytes, little endian
u32::from_be_byte(bytes) // u32 from 4 bytes, big endian
u32::to_le_bytes(num) // u32 to 4 bytes, little endian
u32::to_be_bytes(num) // u32 to 4 bytes, big endian
This was very useful to me recently as I had to write the marshaling and un-marshaling for a game networking format with hundreds of messages. With primitives like this, you can see what's going on.
- Negitivefrags 6y agoUnless you are planning on running your game on a mainframe, just don’t bother with endianness for the networking. Big endian is dead for game developers. Copy entire arrays of structs onto the wire without fear! (Just #pragma pack them first)
- musicale 6y ago> game on a mainframe Maybe your program isn't a game. Maybe you have to deal a server that uses Power, or an embedded system that uses PowerPC (or ARM or MIPS in big-endian mode). Maybe you're running on an older architecture (SPARC, PowerPC, 68K.) Maybe you have to deal with a pre-defined data format (e.g. TCP/IP packet headers) that uses big-endian byte ordering for some of its components.
- Aeolun 6y agoThat’s theoretically possible. But I’d be very interested in why. Especially if you are doing anything involving networking.
- Animats 6y agoBecause it makes it clear what's going on. Most of those functions just generate a move, but it's the correct move. I had to read through excessively-clever C++ code that did the same thing to figure out what conversions were happening, then re-express it in Rust. I'm re-implementing a legacy mess that people are afraid to work on. As it happens, in this message system, some items, mainly packet sequence numbers, are big-endian, because they were following what IP and UDP do, and everything else is little endian. I know how to do this with shifts and masks, and I've done things like that when programming in assembly. That was a long time ago. There's been progress in how to write programs.
- Negitivefrags 6y agoObviously if you are interacting with an old protocol that uses network byte order for some things, you will need to use these functions. But what's the argument for using them in new game code? The code will not be running anywhere that has big-endianness. No current platform a game could run on uses it, and I can't imagine a scenario where a new platform would come into existance and use it either. If you insist on using network-byte order anyway, then you have to do an extra bswap op for each bit of data you send. Sure, the cost of that is super minor and probably not worth worrying about. But the bigger cost is that you can't just send whole structures at a time. You have to individually serialise each thing. Now you have to have a whole serialisation concept. You have to have some way of enumerating all the fields. You have to walk all the structures. What a pain. If you want to send a thing over the network, just send it.
- silon42 6y agoI disagree... but what I would do is use little endian, so the legacy machines need to byte swap.
- infradig 6y agoThere are equivalent functions in C too. The point of the article is about not using them. So how would you implement the above functions in Rust would be more pertinent.
- froh 6y agoisnt the point to be careful when implementing them? so the compiler detects the intention to byteswap? when we ported little endian x86 Linux to the big endian mainframe we sprinkled hton/ntoh all over the place, happily so. they are the way to go and they should be implemented properly, not be replaced by a homegrown version. all that said, I'm surprised 64bit htonll and ntohll are not standard yet. anybody knows why?
- thechao 6y agoBlech. I learned to program (around ‘99) by implementing the crusty old FCS1.0 format, which allows for aggressively weird wire formats. Our machine was a PDP-11/72 with its head sawzalled off and custom wire wrap boards dropped in. The “native” format (coming from analog) was 2143 order as a 36b packet. The bits were [8,0:7] (using verilog notation). However, sprinkled randomly in the binary header were chunks of 7- and 8- bit ANSI (packed) and some mutant knockoff 6-bit EBCDIC. The original listing was written by “Jennifer — please call me if you have troubles”, an undergraduate from MIT. It was hand-assembled machine code, in a neat hand in a big blue binder. That code ran non-stop except for a few hurricanes from 1988 until 2008; bug-free as far as I could tell. Jennifer last-name-unknown, you were my idol & my demon! I swore off programming for nearly a year after that.
- jart 6y agoFunctions like ntohl and htonl are the biggest blemish in the design of the Berkeley Sockets API because it's defined to read memory off the wire without deserializing it. Those functions shouldn't have been invented for the reasons described in the linked blog posts. The C standard isn't going to evolve to include functions that only exist to accommodate code that misunderstands the standard.
- lilyball 6y ago
- pajko 6y agoIt's about accessing memory plus an extra conversion step vs. accessing memory in the right order in one step. As an extra, platform-dependent implementations of the accessors could be done, like using the LWL+LWR instruction pair on MIPS. For reference, check how Linux does it. https://elixir.bootlin.com/linux/latest/source/include/linux/unaligned/le_byteshift.h https://elixir.bootlin.com/linux/latest/source/include/linux...
- matheusmoreira 6y agoDoesn't this run into the same issue described by the author? > Because unsigned char in C expressions gets type promoted to the signed type int. > So if we say 0x80<<24 it overwrites the sign bit, which is an undefined behavior Does the u8 type protect against this?