6 ms·

Heh, 10 years ago I gave a presentation about how easy folks used to x86 can trip up when dealing with ARM's weaker memory model. My demonstration then was with

by anyfoo 5y ago

Heh, 10 years ago I gave a presentation about how easy folks used to x86 can trip up when dealing with ARM's weaker memory model. My demonstration then was with a naive implementation of Peterson's algorithm.[1] I have a feeling that we will see a sharp rise of stories like this, now that ARM finds itself in more places which were previously mostly occupied by x86, and all the subtle race conditions that x86's memory model forgave actually start failing, in equally subtle ways. [1] The conclusion for this particular audience was: Don't try to avoid synchronization primitives, or even invent your own. They were not system level nor high perf code programmers, so they had that luxury.

gpderetta 5y ago
But Peterson's algorithm requires explicit memory barriers even on x86, it doesn't seem the best example to show the difference.
- anyfoo 5y ago
  Here are my slides from back then: https://reinference.net/mp-talk.pdf https://reinference.net/mp-talk.pdf You made me wonder, because I definitely remember using Peterson's Algorithm, so I went back to my slides and turns out: I first showed the problem with x86, then indeed added an MFENCE at the right place, and then showed how that was not enough for ARM. So the point back then was to show how weaker memory models can bite you with the example of x86, and then to show how it can still bite you on ARM with its even weaker model (ARMv7 at that time, and C11 atomics aren't mentioned yet either, but their old OS-specific support is).
  - destring 5y ago
    Damn, my knowledge of ARM and x86 memory models was limited to "x86 provides a stronger model" but that was it. So much to learn, thanks for the slides.
  - gpderetta 5y ago
    Oh, right, yes, ARM additionally needs a release barrier on the unlock path.
mwcampbell 5y ago
> Don't try to avoid synchronization primitives, or even invent your own. Makes me wonder if it's really a good idea in most cases to use, for example, the Rust parking_lot crate, which reimplements mutexes and RW locks. Besides the speed boost for uncontended RW locks, particularly on x64 Linux, what I really like about parking_lot is that a write lock can be downgraded to a read lock without letting go of the lock. But maybe I'd be better off sticking with the tried-and-true OS-provided lock implementations and finding another way to work around the lack of a downgrade option.
- cesarb 5y ago
  Unless you're the maintainer of the parking_lot crate, you're not "inventing your own". And since parking_lot is AFAIK the second most popular implementation of mutexes and RW locks in Rust (the most popular one being obviously the one in the Rust standard library, which wraps the OS-provided lock implementations), you can assume it's well tested.
  - codeflo 5y ago
    Everything about which people tell you to “not invent your own” must be invented by someone.
    - JulianMorrison 5y ago
      It's not "don't invent it". It's "be competent before you invent it, because it's hard". And if you aren't, then let someone who is do the inventing.
      - convolvatron 5y ago
        the best way to _get_ competent is to try so yes - invent your own synchronization primitives. please. just dont believe they are correct without being serious about trying to prove they are. and dont hold up your whole project for self-enrichment. but try to layer as much in as you can. developers these days are so productive, until they fall down and cant get up. and then they are completely useless.
stjohnswarts 5y ago
I doubt that. The number of ARM processors is far greater in reality than in x86 if we clarify it by saying “in operation” rather than historically and these stories will become more common but certainly won't see a “sharp increase”.
- codeflo 5y ago
  This sort of bug only happens when running a multithreaded program (with shared memory) on a multicore processor. You do need both for the problem to happen: Without shared memory, there’s nothing to exploit. And with a single core only, you get time-sliced multithreading, which orders all operations. My point is, that combination was a lot rarer in ARM land before people started doing serious server or desktop computing with those chips.
- retrac 5y ago
  Of course. Any such flaws in the Linux kernel or any library used by Android should have been found by now, for example. But the number of ARM processors running developer/server/desktop stacks has been tiny until recently. In my experience, quite a lot of Linux on desktop software fails to even build on non x86_64 machines.
  - stjohnswarts 5y ago
    Are you kidding? Arm computers are by far the most common over the past 10 years. Computers are everywhere and servers and home computers account for at most 10% of the market for cpus and microcontrollers.
    - masklinn 5y ago
      The vast majority of those are not running multitheaded workloads written by complete randos.
- SavantIdiot 5y ago
  The dominant Arm core in the world is a Cortex-M (or Cortex-R) which are single-core. They are 99% of the time on a die with far less <512K SRAM, and run an RToS or baremetal. These outnumber x86+Cortex-A by probably a factor of 1,000.

by anyfoo 5y ago

gpderetta 5y ago

But Peterson's algorithm requires explicit memory barriers even on x86, it doesn't seem the best example to show the difference.

anyfoo 5y ago
Here are my slides from back then: https://reinference.net/mp-talk.pdf https://reinference.net/mp-talk.pdf You made me wonder, because I definitely remember using Peterson's Algorithm, so I went back to my slides and turns out: I first showed the problem with x86, then indeed added an MFENCE at the right place, and then showed how that was not enough for ARM. So the point back then was to show how weaker memory models can bite you with the example of x86, and then to show how it can still bite you on ARM with its even weaker model (ARMv7 at that time, and C11 atomics aren't mentioned yet either, but their old OS-specific support is).
- destring 5y ago
  Damn, my knowledge of ARM and x86 memory models was limited to "x86 provides a stronger model" but that was it. So much to learn, thanks for the slides.
- gpderetta 5y ago
  Oh, right, yes, ARM additionally needs a release barrier on the unlock path.

mwcampbell 5y ago

> Don't try to avoid synchronization primitives, or even invent your own. Makes me wonder if it's really a good idea in most cases to use, for example, the Rust parking_lot crate, which reimplements mutexes and RW locks. Besides the speed boost for uncontended RW locks, particularly on x64 Linux, what I really like about parking_lot is that a write lock can be downgraded to a read lock without letting go of the lock. But maybe I'd be better off sticking with the tried-and-true OS-provided lock implementations and finding another way to work around the lack of a downgrade option.

cesarb 5y ago
Unless you're the maintainer of the parking_lot crate, you're not "inventing your own". And since parking_lot is AFAIK the second most popular implementation of mutexes and RW locks in Rust (the most popular one being obviously the one in the Rust standard library, which wraps the OS-provided lock implementations), you can assume it's well tested.
- codeflo 5y ago
  Everything about which people tell you to “not invent your own” must be invented by someone.
  - JulianMorrison 5y ago
    It's not "don't invent it". It's "be competent before you invent it, because it's hard". And if you aren't, then let someone who is do the inventing.
    - convolvatron 5y ago
      the best way to _get_ competent is to try so yes - invent your own synchronization primitives. please. just dont believe they are correct without being serious about trying to prove they are. and dont hold up your whole project for self-enrichment. but try to layer as much in as you can. developers these days are so productive, until they fall down and cant get up. and then they are completely useless.

stjohnswarts 5y ago

I doubt that. The number of ARM processors is far greater in reality than in x86 if we clarify it by saying “in operation” rather than historically and these stories will become more common but certainly won't see a “sharp increase”.

codeflo 5y ago
This sort of bug only happens when running a multithreaded program (with shared memory) on a multicore processor. You do need both for the problem to happen: Without shared memory, there’s nothing to exploit. And with a single core only, you get time-sliced multithreading, which orders all operations. My point is, that combination was a lot rarer in ARM land before people started doing serious server or desktop computing with those chips.
retrac 5y ago
Of course. Any such flaws in the Linux kernel or any library used by Android should have been found by now, for example. But the number of ARM processors running developer/server/desktop stacks has been tiny until recently. In my experience, quite a lot of Linux on desktop software fails to even build on non x86_64 machines.
- stjohnswarts 5y ago
  Are you kidding? Arm computers are by far the most common over the past 10 years. Computers are everywhere and servers and home computers account for at most 10% of the market for cpus and microcontrollers.
  - masklinn 5y ago
    The vast majority of those are not running multitheaded workloads written by complete randos.
SavantIdiot 5y ago
The dominant Arm core in the world is a Cortex-M (or Cortex-R) which are single-core. They are 99% of the time on a die with far less <512K SRAM, and run an RToS or baremetal. These outnumber x86+Cortex-A by probably a factor of 1,000.