3 ms·
In addition to all the things mentioned above, there's also another kind of byte: structure padding. Nobody really knows how to deal with it, and it has extreme
by saagarjha 6y ago
In addition to all the things mentioned above, there's also another kind of byte: structure padding. Nobody really knows how to deal with it, and it has extremely strange rules–it's kind of uninitialized in some cases, but you can give it a value sometimes…and sometimes you can't, or not in any way that sticks. It's truly strange stuff, and its exact nature is still being worked out.
- nwmcsween 6y agoPack your structs to not have padding? The probable reason it sometimes works is purely how the compiler decides to do the copy. Anything that relies on implicit padding is horribly broken software.
- saagarjha 6y agoI think you might be misunderstanding my comment, I’m not complaining about struct padding causing issues in my code, I’m saying the standard is wishy-washy about it. This can be problematic when you are doing things like memcpying a structure, or trying to zero it out before sending it somewhere else. (And, FWIW, there is no standard way to disable struct padding, and doing so will likely generate slower code anyways.)
- nwmcsween 6y agoHow is the standard not explicit? copying with padding doesn't need to copy the padding at all or can if it's faster, memcpy is a intrinsic the compiler can copy whatever it deems wrt padding.
- saagarjha 6y agoBecause there are certain cases in C11 where padding can be legally observed because it is defined to be set to zero–brace initialization, for example. It is unclear what happens in this case when you do anything with the memory other than read it (for example, storing to a structure member, based on the current reading of the standard, may make padding indeterminate again–plus, is "memcpy" a "read"?). See here for some of the questions raised: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1793.pdf http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1793.pdf
- asiachick 6y agoIs padding specified by the spec? Because if it's not then padding won't help you either since you don't know that your padding aligns with the compiler's. Rather AFAIK you need to write some compile time checks that what you think the compiler is going to do is what it does and fail compilation if doesn't and then add whatever compiler/platform/option specific code to get your stuff to match along with more tests.
- nwmcsween 6y agoYou would match by adding explicit padding bytes which would be archish specific (32 bit vs 64bit). A much better alternative to making life hard though is to just manually pack the struct
- nextaccountic 6y agoHere's the author of this article (Ralf Jung) writing about padding bytes in the context of Rust (but I suppose it mostly applies to C and C++, since Rust inherited their memory model) https://stackoverflow.com/questions/61114026/does-stdptrwrite-transfer-the-uninitialized-ness-of-the-bytes-it-writes https://stackoverflow.com/questions/61114026/does-stdptrwrit... Also, he wrote a follow up article to deal just with uninitialized memory https://www.ralfj.de/blog/2019/07/14/uninit.html https://www.ralfj.de/blog/2019/07/14/uninit.html edit: the "other kind of byte" that is the padding byte is just uninitialized memory - so perhaps not another kind?
- saagarjha 6y agoI believe I disagree with that model (well, the one in issue linked from the Stack Overflow post), where memcpy needs to copy all the bytes it is told to–I think it is only required to copy bytes that are observable. I agree with the blog post for the most part but it fails to bring up the example of whether you can "freeze" something that is is uninitialized by reading its value and using it in an expression that can only have one value, such as int x; // uninitialized int y = x; // initialized? "frozen"? int z = y ^ y; // zero? I do not think this has been worked out yet–although I never get invited to these discussions, so perhaps I missed recent developments in this area. The other missing part is trap representations–I think Rust only has unspecified, but not indeterminate values. And finally, padding is different from uninitialized memory because it is quite difficult to actually initialize it–the standard lists some ways to zero it out (brace initializer lists) but it seems like pretty much any operation will move it back to being uninitialized. So that's a different than normal uninitialized memory for like static variables, since those have values that you can actually give them and they stick around rather than staying in this weird indeterminate state.
- nextaccountic 6y agoI don't know abou the memcpy part from Stack Overflow, but about the second part: It's UB to read from uninitialized data, and the thing about UB is that the compiler doesn't need to be specially consistent about it. It doesn't need to select the same y for y ^ y, it might transform this into 0 ^ 0 or 10 ^ 5 or select any other arbitrary value for each instance of y in this expression. So the value of z can't be relied upon. even int z = 0*y; can't be relied to produce z equal to 0! The compiler might even delete the whole code after int y = x; because after this line the code is UB. That's why the compiler might remove null-checks (and all code associated with it) done after you dereference a pointer, because since it's UB to follow a null pointer (the only way the program could be possibly correct is if the pointer were not null!), all further checks become dead code. Also, the UB story in Rust is still being developed, but regarding memory representation it's expected to be the same as C. That's because LLVM is quite happy in doing such atrocious but spec-compliant optimizations. Also, now that LLVM supports freeze as an operation (see links below), Rust might surface freeze to the programmer as a primitive low level operation, in order to control the propagation of uninitialized-ness and avoid doing UB when reading variables. It would be something like let y = mem::freeze(x); https://internals.rust-lang.org/t/taming-undefined-behavior-in-llvm/6183 https://internals.rust-lang.org/t/taming-undefined-behavior-... https://www.reddit.com/r/rust/comments/gnx7y0/update_to_llvm_10_got_merged/ https://www.reddit.com/r/rust/comments/gnx7y0/update_to_llvm...