8 ms·

> A similar thing is that I often come across people who are well-informed but still surprised that compilers will combine two observable operations into one W

> A similar thing is that I often come across people who are well-informed but still surprised that compilers will combine two observable operations into one Well, think of it this way, if you're not into compilers wouldn't you expect your code to run as written? Especially if you put a breakpoint in there and you can see each step happening sequently. I don't really think people that don't know these things should be viewed as ignorant because the compiler tends to be doing some very unintuitive things. IMO everyone would be better off if somehow the tooling or the compiler could report some of these things in an easy to visualize not-super-verbose way.

chrisseaton 5y ago
> Well, think of it this way, if you're not into compilers wouldn't you expect your code to run as written? But the possible and actual interleavings aren't written in the program - so how can I guess what the user needs from what's written?
- Retric 5y ago
  Consider the reverse. What if the interleaving was undesirable, would you like some way to be informed that it’s impossible so you don’t have to reason about it? When trying to reason about code, finding somethings that looks like a bug but isn’t can result in a lot of wasted effort.
  - chrisseaton 5y ago
    > What if the interleaving was undesirable, would you like some way to be informed that it’s impossible That's like synchronised in Java or any other kind of atomic block or serialising block - that's already a thing.
    - Retric 5y ago
      That information is moving in the wrong direction. I don’t want the programmer to be telling a compiler what to do, I want the compiler to inform the programmer what it did.
      - chrisseaton 5y ago
        > That information is moving in the wrong direction. You set the direction not me! You asked me 'would you like some way to be informed that [the interleaving is] impossible'. Yes... and the user gives me that information through a synchronised block. If they don't tell me they want an interleaving to be impossible then why should I not make it possible? There are countless other interleavings I make impossible or possible when I do any trivial compiler task such as scheduling. Should I consult the user on all of them? I don't think you'd like all the questions you'd get!
        Retric 5y ago
        Informing the user isn’t about what you did, it’s about the user having some way of knowing what you did. At least compilers give you assembly code to look at, CPU’s are little better than black magic from a programmers perspective.
        chrisseaton 5y ago
        The user can dump the compiler output and intermediate structures if they want. Or they could program against the spec and not worry.
      - kaba0 5y ago
        Check the generated machine/byte code?
- overgard 5y ago
  The answer is: don't guess.
  - chrisseaton 5y ago
    We don't - and in this example we'd combine atomics rather than guess that the user wanted the thread to yield between them, since they never told us that.
    - overgard 5y ago
      That's not true, you are making a guess: that the user doesn't care about the intermediate stage. Not guessing would mean generating the code as specified. It doesn't matter if you think its a bad idea, the whole ethos of C is trust the programmer.
      - chrisseaton 5y ago
        > you are making a guess: that the user doesn't care about the intermediate stage That's not a guess - the user didn't put anything in between them so we know they haven't required them to be separate. Keeping them separate would be making the guess - a guess that they need them to be separate. Why should we invent that constraint when the user never wrote it? > Not guessing would mean generating the code as specified. We are generating it as specified - nothing in the specification says atomic actions have to be run independently. Why should I invent extra constraints that don't exist in the specification?
        overgard 5y ago
        Oh come on, that language lawyering is sophistry. If I write 4 instructions that are "atomic" I expect 4 instructions. If I end up with one, they werent really "atoms" were they? I do not expect what you just said. I may think, gee, perhaps I don't need 4 instructions, but what you are doing is undeniably changing the meaning of the program, even if you think it's close enough. The fact that there's an article about this tends to agree with me that most people think: wtf happened?
sharikone 5y ago
Not to mention that a big part of the confusion is not even the compilation - it's not beyond possibilities to inspect the machine code - but the processor itself still does a lot of unintuitive stuff, even more so with concurrency. I wish the tooling you mention existed, but also some microcode simulation/implementation to look beyond that.
- kaba0 5y ago
  I’m not sure what are you thinking of, but CPUs are guaranteed to run as if the given instructions were executed one after another, even though in practice it is free to reorder them slightly.
  - astrange 5y ago
    Memory operations aren't required to behave like that, although on x86 they mostly do. ARM and Alpha certainly don't.
  - chrisseaton 5y ago
    > CPUs are guaranteed to run as if the given instructions were executed one after another That’s just not true. Intel models will reorder reads and other processors will even reorder writes - and I’m talking about observably.
  - bregma 5y ago
    I've dealt with many CPUs designed to run out of sequence and to guarantee that. I've dealt with multiprocessor computers where order is not guaranteed at all outside the use of hardware interlinks. I've dealt with embarrassingly parallel processors where instruction order is orthogonal. Perhaps you're thinking of a simple abstract CPUs used in "introduction to computers" courses instead of the real-life silicon used in most applications?
ip26 5y ago
This is more of a multi threading & ordering topic than a compiler topic. There were never any guarantees about another thread discretely observing those two observable operations in the first place, due to the memory model.
Someone 5y ago
You use a compiler to abstract away the CPU. Because of that, there is no “run as written”. Simple example: if you declare a structure that is 14-bytes in size, declare variables foo and bar of that type, and then write foo = bar; C must behave as if 14 bytes are copied from the memory where bar is stored to the memory where foo is stored, wit two observable states: foo either has its old contents or the new ones. Chances are very high that your CPU cannot do that. The compiler will instead do something different. It might call memcpy, it might copy 8 bytes first, then 4 more, then the last 2, it might copy the first 8, then the last 8 (copying the two center bytes twice), etc. Similarly, if you declare an int named i and do i += 1; There’s no guarantee that will translate to a single instruction (what if your CPU has 8-bit registers, but your compiler chose to have 16-bit ints?). Even if it does, does “as written” mean an increase instruction or an addition? Your CPU might have neither, but only subtraction and negation. Also, a literal interpretation of “as written” requires spilling the result of every statement back to memory; a strict literal interpretation even forbids using registers (if you don’t write “copy i into a register, add 1, write the result back to i”, the compiler can’t do that if it has to run your code “as written”) But yes, it would be awesome if somehow the tooling or the compiler could report some of these things in an easy to visualize not-super-verbose way. Unfortunately, that isn’t possible. Modern languages and their compilers are just too far abstracted away from modern hardware.
- overgard 5y ago
  > You use a compiler to abstract away the CPU. Because of that, there is no “run as written”. Um, no. I use a compiler to generate code that would otherwise be very tedious to write. I do not expect it in any way to abstract away the CPU. I do not want it to abstract away the CPU. If I did I would write Java or python. I'm using C++. I am very aware of the CPU. What I'm not aware of is whatever silly transformations the optimizer thinks it can do to my carefully crafted program because I didn't read page 783 of the spec.
  - gizmo686 5y ago
    You want a macro assembler. C is not a macro assembler. C was never supposed to be a macro assembler. It was designed as a cross platform language, which fundamentally means you are programming to an abstract machine, which necessarily means that you cannot have an macro assembler [0]. I can, however, see an argument that people want a C like language which is a macro assembler. I am lost again when people start asking for a C++ like macro assembler. The features that C++ brings over C are not concepts that exist on any mainstream CPU. [0] Unless you decide to build a concrete machine (either in bare metal, or as a VM).
pornel 5y ago
"run as written" implies targeting an abstract machine that performs operations the way programmer imagines it does. This machine isn't specified anywhere. The hard part is that programmers don't imagine the code to be executed in a naive unoptimized way, but still expect it to be reasonably well optimized (converted to "obviously" equivalent assembly), except where optimizations combine into a result they haven't thought of.

chrisseaton 5y ago

> Well, think of it this way, if you're not into compilers wouldn't you expect your code to run as written? But the possible and actual interleavings aren't written in the program - so how can I guess what the user needs from what's written?

Retric 5y ago
Consider the reverse. What if the interleaving was undesirable, would you like some way to be informed that it’s impossible so you don’t have to reason about it? When trying to reason about code, finding somethings that looks like a bug but isn’t can result in a lot of wasted effort.
- chrisseaton 5y ago
  > What if the interleaving was undesirable, would you like some way to be informed that it’s impossible That's like synchronised in Java or any other kind of atomic block or serialising block - that's already a thing.
  - Retric 5y ago
    That information is moving in the wrong direction. I don’t want the programmer to be telling a compiler what to do, I want the compiler to inform the programmer what it did.
    - chrisseaton 5y ago
      > That information is moving in the wrong direction. You set the direction not me! You asked me 'would you like some way to be informed that [the interleaving is] impossible'. Yes... and the user gives me that information through a synchronised block. If they don't tell me they want an interleaving to be impossible then why should I not make it possible? There are countless other interleavings I make impossible or possible when I do any trivial compiler task such as scheduling. Should I consult the user on all of them? I don't think you'd like all the questions you'd get!
      - Retric 5y ago
        Informing the user isn’t about what you did, it’s about the user having some way of knowing what you did. At least compilers give you assembly code to look at, CPU’s are little better than black magic from a programmers perspective.
        chrisseaton 5y ago
        The user can dump the compiler output and intermediate structures if they want. Or they could program against the spec and not worry.
    - kaba0 5y ago
      Check the generated machine/byte code?
overgard 5y ago
The answer is: don't guess.
- chrisseaton 5y ago
  We don't - and in this example we'd combine atomics rather than guess that the user wanted the thread to yield between them, since they never told us that.
  - overgard 5y ago
    That's not true, you are making a guess: that the user doesn't care about the intermediate stage. Not guessing would mean generating the code as specified. It doesn't matter if you think its a bad idea, the whole ethos of C is trust the programmer.
    - chrisseaton 5y ago
      > you are making a guess: that the user doesn't care about the intermediate stage That's not a guess - the user didn't put anything in between them so we know they haven't required them to be separate. Keeping them separate would be making the guess - a guess that they need them to be separate. Why should we invent that constraint when the user never wrote it? > Not guessing would mean generating the code as specified. We are generating it as specified - nothing in the specification says atomic actions have to be run independently. Why should I invent extra constraints that don't exist in the specification?
      - overgard 5y ago
        Oh come on, that language lawyering is sophistry. If I write 4 instructions that are "atomic" I expect 4 instructions. If I end up with one, they werent really "atoms" were they? I do not expect what you just said. I may think, gee, perhaps I don't need 4 instructions, but what you are doing is undeniably changing the meaning of the program, even if you think it's close enough. The fact that there's an article about this tends to agree with me that most people think: wtf happened?

sharikone 5y ago

Not to mention that a big part of the confusion is not even the compilation - it's not beyond possibilities to inspect the machine code - but the processor itself still does a lot of unintuitive stuff, even more so with concurrency. I wish the tooling you mention existed, but also some microcode simulation/implementation to look beyond that.

kaba0 5y ago
I’m not sure what are you thinking of, but CPUs are guaranteed to run as if the given instructions were executed one after another, even though in practice it is free to reorder them slightly.
- astrange 5y ago
  Memory operations aren't required to behave like that, although on x86 they mostly do. ARM and Alpha certainly don't.
- chrisseaton 5y ago
  > CPUs are guaranteed to run as if the given instructions were executed one after another That’s just not true. Intel models will reorder reads and other processors will even reorder writes - and I’m talking about observably.
- bregma 5y ago
  I've dealt with many CPUs designed to run out of sequence and to guarantee that. I've dealt with multiprocessor computers where order is not guaranteed at all outside the use of hardware interlinks. I've dealt with embarrassingly parallel processors where instruction order is orthogonal. Perhaps you're thinking of a simple abstract CPUs used in "introduction to computers" courses instead of the real-life silicon used in most applications?

ip26 5y ago

This is more of a multi threading & ordering topic than a compiler topic. There were never any guarantees about another thread discretely observing those two observable operations in the first place, due to the memory model.

Someone 5y ago

You use a compiler to abstract away the CPU. Because of that, there is no “run as written”. Simple example: if you declare a structure that is 14-bytes in size, declare variables foo and bar of that type, and then write foo = bar; C must behave as if 14 bytes are copied from the memory where bar is stored to the memory where foo is stored, wit two observable states: foo either has its old contents or the new ones. Chances are very high that your CPU cannot do that. The compiler will instead do something different. It might call memcpy, it might copy 8 bytes first, then 4 more, then the last 2, it might copy the first 8, then the last 8 (copying the two center bytes twice), etc. Similarly, if you declare an int named i and do i += 1; There’s no guarantee that will translate to a single instruction (what if your CPU has 8-bit registers, but your compiler chose to have 16-bit ints?). Even if it does, does “as written” mean an increase instruction or an addition? Your CPU might have neither, but only subtraction and negation. Also, a literal interpretation of “as written” requires spilling the result of every statement back to memory; a strict literal interpretation even forbids using registers (if you don’t write “copy i into a register, add 1, write the result back to i”, the compiler can’t do that if it has to run your code “as written”) But yes, it would be awesome if somehow the tooling or the compiler could report some of these things in an easy to visualize not-super-verbose way. Unfortunately, that isn’t possible. Modern languages and their compilers are just too far abstracted away from modern hardware.

overgard 5y ago
> You use a compiler to abstract away the CPU. Because of that, there is no “run as written”. Um, no. I use a compiler to generate code that would otherwise be very tedious to write. I do not expect it in any way to abstract away the CPU. I do not want it to abstract away the CPU. If I did I would write Java or python. I'm using C++. I am very aware of the CPU. What I'm not aware of is whatever silly transformations the optimizer thinks it can do to my carefully crafted program because I didn't read page 783 of the spec.
- gizmo686 5y ago
  You want a macro assembler. C is not a macro assembler. C was never supposed to be a macro assembler. It was designed as a cross platform language, which fundamentally means you are programming to an abstract machine, which necessarily means that you cannot have an macro assembler [0]. I can, however, see an argument that people want a C like language which is a macro assembler. I am lost again when people start asking for a C++ like macro assembler. The features that C++ brings over C are not concepts that exist on any mainstream CPU. [0] Unless you decide to build a concrete machine (either in bare metal, or as a VM).

pornel 5y ago

"run as written" implies targeting an abstract machine that performs operations the way programmer imagines it does. This machine isn't specified anywhere. The hard part is that programmers don't imagine the code to be executed in a naive unoptimized way, but still expect it to be reasonably well optimized (converted to "obviously" equivalent assembly), except where optimizations combine into a result they haven't thought of.