2 ms·

A recent study seemed to support Fabian's well written article: https://dl.acm.org/doi/epdf/10.1145/3779212.3790129 https://dl.acm.org/doi/epdf/10.1145/3779212.

by Aissen 13d ago

A recent study seemed to support Fabian's well written article: https://dl.acm.org/doi/epdf/10.1145/3779212.3790129 https://dl.acm.org/doi/epdf/10.1145/3779212.3790129

wat10000 13d ago
Is this just a question of what one considers to be “significant”? I’d consider 3% to be significant but the authors apparently don’t.
- deater 13d ago
  well it depends on your error bars. On many modern system you can have +/- 5% variation or more run to run just due to the non-determinism present in modern CPU architectures and operating systems (even things like room temperature, time of day, the number of environment variables, etc, can affect this). While you could maybe run a set of careful experiments to characterize and remove this, in my experience most researchers don't bother. So something as small as 3% would need a lot of convincing to me to make the argument that it is significant.
  - wat10000 13d ago
    Multiple separate questions here. First, is a measured improvement actually real or just an artifact of noise? Second, if it is real, is 3% anywhere close to the true value? Third, if 3% is real, is it an important difference? I'm just commenting on the third one. If 3% is real, it's important. I'm inclined to believe there's a real improvement. They made a lot of different measurements. If the measured improvement was a result of noise, you'd expect a lot of variation an a lot of measurements where TSA was actually faster, and then 3% was the average of that variation. There was a lot of variation (expected, because they were measuring different things) but nearly all of them had TSO being either neutral or slower. Looking at their benchmark graphs, I see two (out of dozens) where TSO was faster. As far as being close to the true value, these results suggest there is no single true value, as it depends on the workload. No surprise there.
kccqzy 13d ago
It’s a question of how much resources to allocate to the hardware team, and how much resources to be distributed diffusely to the software engineers but especially to the compiler team. Even your linked paper contends that the actual observed slowdown is as much as 22% in the Geekbench example, but the thesis is that the slowdown is not inherent to TSO, but merely to the specific hardware implementation. Is it worthwhile for a company to optimize its TSO to chase the final gains, or is it better not to have this feature in the first place and just change the compiler? Indeed my instinct is that it is better to do this in software, where the programmer clearly communicates which stores are ordered, and which may happen in arbitrary order.

by Aissen 13d ago

A recent study seemed to support Fabian's well written article: https://dl.acm.org/doi/epdf/10.1145/3779212.3790129 https://dl.acm.org/doi/epdf/10.1145/3779212.3790129

wat10000 13d ago

Is this just a question of what one considers to be “significant”? I’d consider 3% to be significant but the authors apparently don’t.

deater 13d ago
well it depends on your error bars. On many modern system you can have +/- 5% variation or more run to run just due to the non-determinism present in modern CPU architectures and operating systems (even things like room temperature, time of day, the number of environment variables, etc, can affect this). While you could maybe run a set of careful experiments to characterize and remove this, in my experience most researchers don't bother. So something as small as 3% would need a lot of convincing to me to make the argument that it is significant.
- wat10000 13d ago
  Multiple separate questions here. First, is a measured improvement actually real or just an artifact of noise? Second, if it is real, is 3% anywhere close to the true value? Third, if 3% is real, is it an important difference? I'm just commenting on the third one. If 3% is real, it's important. I'm inclined to believe there's a real improvement. They made a lot of different measurements. If the measured improvement was a result of noise, you'd expect a lot of variation an a lot of measurements where TSA was actually faster, and then 3% was the average of that variation. There was a lot of variation (expected, because they were measuring different things) but nearly all of them had TSO being either neutral or slower. Looking at their benchmark graphs, I see two (out of dozens) where TSO was faster. As far as being close to the true value, these results suggest there is no single true value, as it depends on the workload. No surprise there.

kccqzy 13d ago

It’s a question of how much resources to allocate to the hardware team, and how much resources to be distributed diffusely to the software engineers but especially to the compiler team. Even your linked paper contends that the actual observed slowdown is as much as 22% in the Geekbench example, but the thesis is that the slowdown is not inherent to TSO, but merely to the specific hardware implementation. Is it worthwhile for a company to optimize its TSO to chase the final gains, or is it better not to have this feature in the first place and just change the compiler? Indeed my instinct is that it is better to do this in software, where the programmer clearly communicates which stores are ordered, and which may happen in arbitrary order.