9 ms·

It is a ridiculous feature of modern C that you have to write the super verbose "mask and shift" code, which then gets compiled to a simple `mov` and maybe a `b

• nly 5y ago
  > I think there should really be a dialect of C(++) where the machine model is exactly the physical machine. Sounds great, until you have to rewrite all your software to go from x86-64 to ARM
  • pjmlp 5y ago
    Quite common when coding games back in the 8 and 16 bit days. :) However for the case in hand, it would suffice to just write the key routines in Assembly, not everything.
• simias 5y ago
  > Wheras, the direct equivalent in C, an assignment with a (type changing) cast, is illegal. I don't understand what you mean by that. The direct equivalent of what? Endianess is not part of the type system in C so I'm not sure I follow. > I think there should really be a dialect of C(++) where the machine model is exactly the physical machine. Linus agrees with you here, and I disagree with both of you. Some UBs could certainly be relaxed, but as a rule I want my code to be portable and for the compiler to have enough leeway to correctly optimize my code for different targets without having to tweak my code. I want strict aliasing and I want the compiler to delete extraneous NULL pointer checks. Strict overflow I'm willing to concede, at the very least the standard should mandate wrap-on-overflow ever for signed integers IMO.
  • lanstin 5y ago
    I am sympathetic, but portability was more important in the past and gets less important each year. I used to write code strictly keeping the difference between numeric types and sequences of bytes in mind, hoping to one day run on an Alpha or a Tandem or something, but it has been a long time since I have written code that runs on non-(Intel AMD or le ARM)
    • alexvoda 5y ago
      x86_32, x86_64, arm, arm64, POWER , RISC-V and several others are alive and kicking. China is making their own ISA. And there is still plenty of space and time for new ISAs to be created. Portability is still plenty relevant.
• ajross 5y ago
  > There is a huge mismatch between the assumptions of the C spec and actual machine code. Right, which is why the kind of UB pedantry in the linked article is hurting and not helping. Cranky old man perspective here: Folks: the fact that compilers will routinely exploit edge cases in undefined behavior in the language specification to miscompile obvious idiomatic code is a terrible bug in the compilers. Period. And we should address that by fixing the compilers, potentially by amending the spec if feasible. But instead the community wants to all look smart by showing how much they understand about "UB" with blog posts and (worse) drive-by submissions to open source projects (with passive agressive sneers about code quality), so nothing gets better. Seriously: don't tell people to shift and mask. Don't pontificate over compiler flags. Stop the masturbatory use of ubsan (though the tool itself is great). And start submitting bugs against the toolchain to get this fixed.
  • wnoise 5y ago
    I read this, and go "yes, yes, yes", and then "NO!". Shifts and ors really is the sanest and simplest way to express "assembling an integer from bytes". Masking is _a_ way to deal with the current C spec which has silly promotion rules. Unsigned everything is more fundamental than signed.
  • jart 5y ago
    I agree but language of the standard very unambiguously lets them do it. Quoth X3.159-1988 * Undefined behavior --- behavior, upon use of a nonportable or erroneous program construct, of erroneous data, or of indeterminately-valued objects, for which the Standard imposes no requirements. Permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message). In the past compilers "behaved during translation or program execution in a documented manner characteristic of the environment" and now they've decided to "ignore the situation completely with unpredictable results". So yes what gcc and clang are doing is hostile and dangerous, but it's legal. https://justine.lol/undefined.png https://justine.lol/undefined.png So let's fix our code. The blog post is intended to help people do that.
• cbmuser 5y ago
  > One of the few reasons I ever even reached to C is the ability to slurp in data and reinterpret it as a struct, or the ability to reason in which registers things will show up and mix in some `asm` with my C. Which results in undefined behavior according to the C ISO standard. Quote: “2 All declarations that refer to the same object or function shall have compatible type; otherwise, the behavior is undefined.” From: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf 6.2.7
  • jstanley 5y ago
    Exactly.
  • innocenat 5y ago
    How? I mean, doesn't GP mean this? struct whatever p; fread(p, sizeof(p), 1, fp);
  • deleted 5y ago
    [deleted]
  • tsimionescu 5y ago
    It should be perfectly fine to do this: union reinterpret { char raw[100]; struct myStruct interpreted; } example; read(fd, &example.raw) struct myStruct dest = interpreted; This is standard-compliant C code, and it is a common way of reading IP addresses from packets, for example.
    • saagarjha 5y ago
      (It should be noted that this is not valid C++ code.)
      • jart 5y ago
        Sensitive emotional subjects shouldn't be noted. Reminding C developers of the void* incompatibility is a good way to get them to feel triggered because it makes the language unpleasant.
• MrBuddyCasino 5y ago
  > There is a huge mismatch between the assumptions of the C spec and actual machine code. People like to say „C is close to the metal“. Really not true at all anymore.
  • goldenkey 5y ago
    Actually, it is true - which is why endian is a problem in the first place. ASM code is different when written for little endian vs big endian. Access patterns are positively offset instead of negatively. A language that does the same things regardless of endianness would not have pointer arithmetic. That is not ASM and not C.
• mhh__ 5y ago
  D's machine model does actually assume the hardware, and using the compile time metaprogramming you can pretty much do whatever you want when it comes to bit twiddling - whether that means assembly, flags etc.
• pjmlp 5y ago
  It does, macro assemblers, specially those with PC and Amiga roots. Which given its heritage, that is what PDP-11 C used to be, after all BCPL origin was as minimal language required to bootstrap CPL, nothing else. Actually, I think TI has a macro Assembler with a C like syntax, just cannot recall the name any longer.
• klodolph 5y ago
  You don’t have to mask and shift. You can memcpy and then byte swap in a function. It will get inlined as mov/bswap. Practically speaking, common compilers have intrinsics for bswap. The memcpy function can be thought of as an intrinsic for unaligned load/store.
  • user-the-name 5y ago
    When do you byte swap?
    • klodolph 5y ago
      24 hours a day, man. I'm always byte swapping. (I'm not sure how to answer the question... what do you mean, "when?")
      • user-the-name 5y ago
        The entire problem of using byte swaps is that you need to use them when your native platform's byte order is different from that of the data you are reading. You know the byte order of the data. But the tricky part is, what is the byte order of the platform?
        • klodolph 5y ago
          Whether it is tricky depends on what platforms you care about.
          • user-the-name 5y ago
            Or, you can just follow the advice of the article, and not need to worry about it because the compiler takes care of it for you.
            • account42 5y ago
              > because the compiler takes care of it for you. It will always be correct, but you can't just assume that the compiler will optimize the shifts into a byteswap instructions. If you look at the article you will see that it tires to no-true-scotsman that concern away by talking about a "good modern compiler".
• nine_k 5y ago
  I suspect you might like C--. https://en.m.wikipedia.org/wiki/C-- https://en.m.wikipedia.org/wiki/C--
• pm215 5y ago
  So in your 'machine model is the physical machine' flavour, should "I cast an unaligned pointer to a byte array to int32_t and deref" on SPARC (a) do a bunch of byte-load-and-shift-and-OR or (b) emit a simple word load which segfaults? If the former, it's not what the physical machine does, and if the latter, then you still need to write the code as "some portable other thing". Which is to say that the spec's UB here is in service of "allow the compiler to just emit a word load when you write *(int32_t)p". What I think the language is missing is a way to clearly write "this might be unaligned and/or wrong endianness, handle that". (Sometimes compilers provide intrinsics for this sort of gap, as they do with popcount and count-leading-zeroes; sometimes they recognize common open-coded idioms. But proper standardised support would be nicer.)
  • jart 5y ago
    Endianness doesn't matter though, for the reasons Rob Pike explained. For example, the bits inside each byte have an endianness probably inside the CPU but they're not addressable so no one thinks about that. The brilliance of Rob Pike's recommendation is that it allows our code to be byte order agnostic for the same reasons our code is already bit order agnostic. I agree about bsf/bsr/popcnt. I wish ASCII had more punctuation marks because those operations are as fundamental as xor/and/or/shl/shr/sar.
• froh 5y ago
  you could instead simply use hton/ntoh and trust the library properly does The Right Thing tm
• pornel 5y ago
  Of course nobody wants C to backstab them with UB, but at the same time programmers want compilers to generate optimal code. That's the market pressure that forces optimizers to be so aggressive. If you can accept less optimized code, why aren't you using tcc? The idea of C that "just" does a straightforward machine translation breaks down almost immediately. For example, you'd want `int` to just overflow instead of being UB. But then it turns out indexing `arr[i]` can't use 64-bit memory addressing modes, because they don't overflow like a 32-bit int does. With UB it doesn't matter, but a "straightforward C" would emit unnecessary separate 32-bit mul/shift instructions. https://gist.github.com/rygorous/e0f055bfb74e3d5f0af20690759de5a7 https://gist.github.com/rygorous/e0f055bfb74e3d5f0af20690759...
  • MaxBarraclough 5y ago
    > nobody wants C to backstab them with UB, but at the same time programmers want compilers to generate optimal code The value of compiler optimization isn't the same thing as the value of having extensive undefined behaviour in a programming language. Rust and Ada perform about the same as C, but lack C's many footguns. > indexing `arr[i]` can't use 64-bit memory addressing modes What do you mean here?
    • remexre 5y ago
      Typically, the assembly instruction that would do the read in arr[i] can do something like: x = *(y + z); where y and z are both 64-bit integers. If I had int arr[1000]; initialize(&arr); int i = read_int(); int x = arr[i]; print(x); then to get x I'd need to do something like, tmp = i * 4; tmp1 = (uint64_t)tmp; x = *(arr + tmp1); Which, since i is signed, can't just be a cheap shift, and then needs to be upcasted to a uint64_t (which is cheap, at least).
• alexvoda 5y ago
  > That doesn't mean the compiler can't do optimizations, but it shouldn't do things like prove code as UB and fold everything to a no-op. UB doesn't just mean the compiler can treat it as a no-op. It means the compiler can do whatever it likes and still be compliant with the spec. From the POV of someone consulting the spec, if something results in UB, what it means is: "Don't look here for documentation, look in the documentation of your compiler!". Many compilers prefer to do a no-op because it is the cheapest thing to do.
  • erik_seaberg 5y ago
    C is carful to distinguish “unspecified behavior” (every compiler must document a consistent choice) and “undefined behavior” which doesn’t necessarily have any safe uses.
  • jart 5y ago
    My read of the standard is the worst the compiler can do, is to do nothing. For example, the blog post links a tweet where clang doing nothing meant generating an empty function so that calling it execution fell through to a different function the author had written which formats the hard drive. However it wouldn't be kosher for the compiler to generate the asm that formats your hard drive itself as an intended punishment for UB since the standard recommends, if you're not going to do nothing, then you can have the compiler act in a way that's characteristic of the environment, or you can crash with an error.