4 ms·

The re-ordering you were seeing may have been due to a lack of sequence points between the ordered operations. For example: If you have two 32 bit registers whi

by magila 8y ago

The re-ordering you were seeing may have been due to a lack of sequence points between the ordered operations. For example: If you have two 32 bit registers which correspond to a single 64 bit value and which must be read in a specific order to ensure consistency, then you cannot write (code simplified for clarity): int64_t val = (*reg0 << 32) | *reg1; The compiler is allowed to re-order the register reads even if they are volatile pointers. To guarantee a specific order you need something like: int64_t val = reg0 << 32; val |= reg1; Sequence points are one of those dark corners of C/C++ which most people don't know about because they rarely matter unless you are dealing directly with hardware.

vvanders 8y ago
x86 + win32 also forces a bunch of strict ordering(esp w/ volatile) that doesn't hold true on lots of other platforms. If you're developing code for testing on desktop then deploying to a target platform it's very easy to get bit by nasty concurrency bugs in a variety of ways.
burfog 8y ago
No, a problem with sequence points is at the compiler level, just the same as "volatile". Assume the compiler doesn't reorder anything. Maybe you turn the optimization off. You even inspect the assembly code, and all the operations are there in the correct order. You can still hit the problem. In my case, the code ran fine until we got an upgraded CPU. (from MPC6xx series to MPC74xx series) Suppose you store to registers at 0xf0000ffc, then 0xf0000104, then 0xf0000ff8. You need the stores to happen in that order. The instructions execute in that order, creating 32-bit chunks of data headed out toward the memory bus. There are multiple write buffers however, so they can go in parallel, each getting a distinct write buffer. They then head out onto the memory bus in some randomish order determined by timing issues internal to the CPU. In my case, I had to add "eieio" instructions between each pair of accesses for which ordering mattered. FYI, that is a real instruction, supposedly meaning "Enforce In-Order Execution of I/O".
- comex 8y ago
  I found this surprising so I looked it up. According to the MPC7410 user manual[1], assuming the register bank is mapped as caching-inhibited, a sequence of stores is required to take effect in program order without needing `eieio` between them. However, a store followed by a load to a different address can be subject to reordering and does need `eieio`. The ARM architecture does this more sanely. Page table entries have a flag that lets you choose between regular "Device" memory or "Strongly-ordered" memory; the latter performs all memory accesses in order without needing any synchronization instructions, and is more convenient in simple situations. [1] https://www.nxp.com/docs/en/reference-manual/MPC7410UM.pdf https://www.nxp.com/docs/en/reference-manual/MPC7410UM.pdf - Table 3-8

by magila 8y ago

vvanders 8y ago

x86 + win32 also forces a bunch of strict ordering(esp w/ volatile) that doesn't hold true on lots of other platforms. If you're developing code for testing on desktop then deploying to a target platform it's very easy to get bit by nasty concurrency bugs in a variety of ways.

burfog 8y ago

No, a problem with sequence points is at the compiler level, just the same as "volatile". Assume the compiler doesn't reorder anything. Maybe you turn the optimization off. You even inspect the assembly code, and all the operations are there in the correct order. You can still hit the problem. In my case, the code ran fine until we got an upgraded CPU. (from MPC6xx series to MPC74xx series) Suppose you store to registers at 0xf0000ffc, then 0xf0000104, then 0xf0000ff8. You need the stores to happen in that order. The instructions execute in that order, creating 32-bit chunks of data headed out toward the memory bus. There are multiple write buffers however, so they can go in parallel, each getting a distinct write buffer. They then head out onto the memory bus in some randomish order determined by timing issues internal to the CPU. In my case, I had to add "eieio" instructions between each pair of accesses for which ordering mattered. FYI, that is a real instruction, supposedly meaning "Enforce In-Order Execution of I/O".

comex 8y ago
I found this surprising so I looked it up. According to the MPC7410 user manual[1], assuming the register bank is mapped as caching-inhibited, a sequence of stores is required to take effect in program order without needing `eieio` between them. However, a store followed by a load to a different address can be subject to reordering and does need `eieio`. The ARM architecture does this more sanely. Page table entries have a flag that lets you choose between regular "Device" memory or "Strongly-ordered" memory; the latter performs all memory accesses in order without needing any synchronization instructions, and is more convenient in simple situations. [1] https://www.nxp.com/docs/en/reference-manual/MPC7410UM.pdf https://www.nxp.com/docs/en/reference-manual/MPC7410UM.pdf - Table 3-8