4 ms·

This depends heavily on what concurrency optimizations your processor implements (and unfortunately this is the sort of thing that doesn't get doccumented and i

This depends heavily on what concurrency optimizations your processor implements (and unfortunately this is the sort of thing that doesn't get doccumented and is somewhat hard to test).

anematode 1y ago
I did a little unscientific test here on an Apple M4 Pro with n threads spamming atomic operations with pseudorandom values on one memory location (the worst case). Used inline asm to make sure there was no funny business going on. atomic adds n = 1 -> 333e6 adds/second n = 2 -> 174e6 n = 4 -> 95e6 n = 8 -> 63e6 atomic maxs n = 1 -> 161e6 maxs/second n = 2 -> 59e6 n = 4 -> 39e6 n = 8 -> 27e6 atomic maxs with preceding check n = 1 -> 929e6 maxs/second n = 2 -> 1541e6 n = 4 -> 3494e6 n = 8 -> 5985e6 So evidently the M4 doesn't do this optimization. Of course if your distribution is different you'd get different results, and this level of contention is unrealistic, but I don't see why you'd EVER not do a check before running atomic max. I also find it interesting that atomic max is significantly slower than atomic add
- thequux 1y ago
  I think that this can change the semantics though; with the preceding check you can miss the shared variable being decremented from another thread. In some cases, such as if the shared value is monotonic, this is done, but not in the general case.
  - anematode 1y ago
    With a relaxed ordering I'm not sure if that's right, since the ldumax would have no imposed ordering relation with the (atomic) decrement on another thread and so could very well have operated on the old value obtained by the non-atomic load
    - ibraheemdev 1y ago
      It does make a difference of course if you're running fetch_max from multiple threads, adding a load fast-path introduces a race condition.
      - deleted 1y ago
        [deleted]
      - masklinn 1y ago
        Does it tho? Assuming no torn reads/writes at those sizes, given the location should be strictly increasing are there situations where you could read a higher-than-stored value which would cause skipping a necessary update? Afaik on all of x86, arm, and riscv an atomic load of a word sized datum is just a regular load.
        gpderetta 1y ago
        It doesn't need to be strictly increasing some other thread could be making other arbitrary operations. Still even in that case, as Dylan16807 pointed out, it likely doesn't matter.
        masklinn 1y ago
        > It doesn't need to be strictly increasing some other thread could be making other arbitrary operations We're talking about collating a maximum, by definition every write to that is an increase.
        gpderetta 1y ago
        If you are implementing a library function atomic<T>::fetch_max, you cannot assume that every other thread is also performing a fetch_max on that object. There might be little reason for it, but other operations are allowed so the the sequence of modifications might not be strictly increasing (but then again, it doesn't matter for this specific optimization).
    - 1y ago

This depends heavily on what concurrency optimizations your processor implements (and unfortunately this is the sort of thing that doesn't get doccumented and is somewhat hard to test).

anematode 1y ago

I did a little unscientific test here on an Apple M4 Pro with n threads spamming atomic operations with pseudorandom values on one memory location (the worst case). Used inline asm to make sure there was no funny business going on. atomic adds n = 1 -> 333e6 adds/second n = 2 -> 174e6 n = 4 -> 95e6 n = 8 -> 63e6 atomic maxs n = 1 -> 161e6 maxs/second n = 2 -> 59e6 n = 4 -> 39e6 n = 8 -> 27e6 atomic maxs with preceding check n = 1 -> 929e6 maxs/second n = 2 -> 1541e6 n = 4 -> 3494e6 n = 8 -> 5985e6 So evidently the M4 doesn't do this optimization. Of course if your distribution is different you'd get different results, and this level of contention is unrealistic, but I don't see why you'd EVER not do a check before running atomic max. I also find it interesting that atomic max is significantly slower than atomic add

thequux 1y ago
I think that this can change the semantics though; with the preceding check you can miss the shared variable being decremented from another thread. In some cases, such as if the shared value is monotonic, this is done, but not in the general case.
- anematode 1y ago
  With a relaxed ordering I'm not sure if that's right, since the ldumax would have no imposed ordering relation with the (atomic) decrement on another thread and so could very well have operated on the old value obtained by the non-atomic load
  - ibraheemdev 1y ago
    It does make a difference of course if you're running fetch_max from multiple threads, adding a load fast-path introduces a race condition.
    - deleted 1y ago
      [deleted]
    - masklinn 1y ago
      Does it tho? Assuming no torn reads/writes at those sizes, given the location should be strictly increasing are there situations where you could read a higher-than-stored value which would cause skipping a necessary update? Afaik on all of x86, arm, and riscv an atomic load of a word sized datum is just a regular load.
      - gpderetta 1y ago
        It doesn't need to be strictly increasing some other thread could be making other arbitrary operations. Still even in that case, as Dylan16807 pointed out, it likely doesn't matter.
        masklinn 1y ago
        > It doesn't need to be strictly increasing some other thread could be making other arbitrary operations We're talking about collating a maximum, by definition every write to that is an increase.
        gpderetta 1y ago
        If you are implementing a library function atomic<T>::fetch_max, you cannot assume that every other thread is also performing a fetch_max on that object. There might be little reason for it, but other operations are allowed so the the sequence of modifications might not be strictly increasing (but then again, it doesn't matter for this specific optimization).
  - 1y ago