5 ms·

Most multithreaded software has concurrency bugs and many of those bugs have been covered up for years by the relatively friendlier x86 memory model vs weaker m

Most multithreaded software has concurrency bugs and many of those bugs have been covered up for years by the relatively friendlier x86 memory model vs weaker models like ARM. It is not surprising at all that supported products are going to be slow to support other architecture.

aidos 6y ago
Ooooh interesting. Any more info about the difference in models and how it causes this effect?
- jeffbee 6y ago
  Quick explanation: https://www.cs.utexas.edu/~bornholt/post/memory-models.html https://www.cs.utexas.edu/~bornholt/post/memory-models.html Basically, although I'm sure someone will shout me down for this, x86 provides something close to sequential consistency among loads and stores even for concurrent programs, whereas ARM and POWER provide almost no kind of order for loads and stores without barriers. High-level languages like C++ do not really hide this problem. It is very easy to write a concurrent C++ program on x86 that appears to work in practice but never works on ARM.
  - my123 6y ago
    You need proper memory model handling for the program to be compliant even for x86. If you do it otherwise and you see that it works for you, keep in mind that the compiler is allowed to reorder in that case, which can result in very odd breakage. On x86, sequential consistency isn't the implemented memory model, but rather total store ordering. Stores are buffered inside the core and the guarantee provided is that they'll appear to other cores in the proper order. On SC designs, all the cores have the same view of coherent memory all the time. In such a design, barriers aren't required and are lowered down to NOPs. (only general-purpose processors that I know of that implement sequential consistency are NVIDIA's Tegra K1 (64-bit variant) and Tegra Xavier)
gopalv 6y ago
> many of those bugs have been covered up for years by the relatively friendlier x86 memory model vs weaker models like ARM I had to take a peek at a friend's Cavium's ThunderX2 kernel issue a while back, which stops having concurrency when release build is swapped with a debug build. So the specific part that drives you crazy is "Atomic can be reordered with stores - Y" for ARM and N for X86_64. And changing the compiler optimization flags can turn on/off this bug.
- foobiekr 6y ago
  We previously did a product based on the Octeon3. I feel sympathy for anyone who has to deal with Cavium's SDKs.
etaioinshrdlu 6y ago
It is not just ARM instances. Parallel Query, Multi-Master, and Performance Insights, are all locked to specific classes of x86 instances. It's like Amazon has narrowly tested specific features on specific hardware sizes.
- ashtonkem 6y ago
  On one hand, it's incredibly annoying. On the other hand part of why you'd choose RDS over deploying MariaDB yourself to an EC2 instance is to shift some of the responsibility of testing & configuration over to AWS. If AWS is disabling features on instances that don't work, this is actually a benefit that they're providing to their users, as you can better trust that any feature enabled on a given product & instance type will "just work".
  - acdha 6y ago
    Also, this is what you're paying for. I have a couple of legacy RDS databases which have been running for a decade. When you're making a promise to a large number of customers that you're going to support something for the long term it's understandable that you're going to be conservative and probably run some sort of closed beta for long enough to get solid operational experience.
- sandinmyjoints 6y ago
  Is there somewhere that lists which features are enabled on which instances? We were thinking of switching to Graviton2 RDS instances but not being able to use Performance Insights would be an important factor in making or not making that switch.

aidos 6y ago

Ooooh interesting. Any more info about the difference in models and how it causes this effect?

jeffbee 6y ago
Quick explanation: https://www.cs.utexas.edu/~bornholt/post/memory-models.html https://www.cs.utexas.edu/~bornholt/post/memory-models.html Basically, although I'm sure someone will shout me down for this, x86 provides something close to sequential consistency among loads and stores even for concurrent programs, whereas ARM and POWER provide almost no kind of order for loads and stores without barriers. High-level languages like C++ do not really hide this problem. It is very easy to write a concurrent C++ program on x86 that appears to work in practice but never works on ARM.
- my123 6y ago
  You need proper memory model handling for the program to be compliant even for x86. If you do it otherwise and you see that it works for you, keep in mind that the compiler is allowed to reorder in that case, which can result in very odd breakage. On x86, sequential consistency isn't the implemented memory model, but rather total store ordering. Stores are buffered inside the core and the guarantee provided is that they'll appear to other cores in the proper order. On SC designs, all the cores have the same view of coherent memory all the time. In such a design, barriers aren't required and are lowered down to NOPs. (only general-purpose processors that I know of that implement sequential consistency are NVIDIA's Tegra K1 (64-bit variant) and Tegra Xavier)

gopalv 6y ago

> many of those bugs have been covered up for years by the relatively friendlier x86 memory model vs weaker models like ARM I had to take a peek at a friend's Cavium's ThunderX2 kernel issue a while back, which stops having concurrency when release build is swapped with a debug build. So the specific part that drives you crazy is "Atomic can be reordered with stores - Y" for ARM and N for X86_64. And changing the compiler optimization flags can turn on/off this bug.

foobiekr 6y ago
We previously did a product based on the Octeon3. I feel sympathy for anyone who has to deal with Cavium's SDKs.

etaioinshrdlu 6y ago

It is not just ARM instances. Parallel Query, Multi-Master, and Performance Insights, are all locked to specific classes of x86 instances. It's like Amazon has narrowly tested specific features on specific hardware sizes.

ashtonkem 6y ago
On one hand, it's incredibly annoying. On the other hand part of why you'd choose RDS over deploying MariaDB yourself to an EC2 instance is to shift some of the responsibility of testing & configuration over to AWS. If AWS is disabling features on instances that don't work, this is actually a benefit that they're providing to their users, as you can better trust that any feature enabled on a given product & instance type will "just work".
- acdha 6y ago
  Also, this is what you're paying for. I have a couple of legacy RDS databases which have been running for a decade. When you're making a promise to a large number of customers that you're going to support something for the long term it's understandable that you're going to be conservative and probably run some sort of closed beta for long enough to get solid operational experience.
sandinmyjoints 6y ago
Is there somewhere that lists which features are enabled on which instances? We were thinking of switching to Graviton2 RDS instances but not being able to use Performance Insights would be an important factor in making or not making that switch.