4 ms·

Relying heavily on a check for an atomic bool is prone to race conditions. I think it's cleaner to structure the event loop as a message queue and have a queued

Relying heavily on a check for an atomic bool is prone to race conditions. I think it's cleaner to structure the event loop as a message queue and have a queued message that indicates it's time to stop.

PaulDavisThe1st 1y ago
Every event loop is subject to the blocked-due-to-long-running-computation issue. It bites ...
- asveikau 1y ago
  The same is true if you're repeatedly polling an atomic boolean in an event loop.
  - CamperBob2 1y ago
    How so? It takes only a couple of machine cycles to poll a boolean. (And what other kind of boolean is there, besides atomic? It's either true or it's false, and if nothing can set it back to false once it goes true, I don't see the hazard. It's a CPU, not an FPGA.)
    - kbolino 1y ago
      The type is named atomic, but atomicity is not its only useful property. The atomic types also give control over the memory ordering, defaulting to sequentially consistent (seq_cst, the strongest). Without memory order guarantees enforced by memory barriers, a write to the boolean in thread A is not guaranteed to be observed by thread B. That matters both after initialization--where thread A sets the boolean to false but thread B may observe true, false, or invalid--and also after the transition--where thread B may fail to observe that the boolean has flipped from false to true. [edit: I'm not sure the above reasoning actually matters; as stated already by parent, "It's a CPU, not an FPGA"; modern multicore shared-memory CPUs have coherent caches]
      - CamperBob2 1y ago
        Well, it'll be observed the next time through the loop. If that matters, then it's true that this technique isn't desirable.
        manwe150 1y ago
        Without atomic, the compiler won’t bother with there being a next time and just infinite loop (in the old days, you’d mark it volatile instead)
        CamperBob2 1y ago
        True enough, but volatile still works, of course.
      - adwn 1y ago
        > Without memory order guarantees enforced by memory barriers, a write to the boolean in thread A is not guaranteed to be observed by thread B. No, that's not correct. Memory ordering doesn't influence how fast a write is propagated to other cores, that's what cache coherency is for. Memory ordering of an access only matters in relation to accesses on other memory locations. There's a great introduction to this topic by Mara Bos: https://marabos.nl/atomics/memory-ordering.html https://marabos.nl/atomics/memory-ordering.html
        kbolino 1y ago
        Indeed. I started to figure this out, hence my edit. Thanks for the link. There are hypothetical, historical, and special-purpose architectures which don't have cache coherency (or implement it differently enough to matter here), but for all practical purposes, it seems that all modern, general-purpose architectures implement it.
    - mattgreenrocks 1y ago
      It’s free on x86 for relaxed ordering, which is sufficient for this use case.
deleted 1y ago
[deleted]
a-dub 1y ago
disagree. i think then it's too tempting down the line for someone to add a message with blocking processing. a simple clear loop that looks for a requested stop flag with a confirmed stop flag works pretty well. this can be built into a synchronous "stop" function for the caller that sets the flag and then does a timed wait on the confirmation (using condition variables and pthread_cond_timedwait or waitforxxxobject if you're on windows).
- asveikau 1y ago
  Making your check less stable doesn't prevent this. The examples in this article IIRC were something like this. while (check_flag()) { do_stuff(); sleep_like_a_moron_instead_of_proper_blocking_mechanism(1second); } You're still going to be arbitrarily delayed if do_stuff() (or one one of its callees, maybe deep inside the stack) delays, or the sleep call. If you can't accept this, maybe don't play with threads, they are dangerous.
  - a-dub 1y ago
    that's the point. use nonblocking io and an event polling mechanism with a timeout to keep an eye on an exit flag- that's all you need to handle clean shutdowns. i think on windows you can wait on both the sockets/file descriptors and condition variables with the same waitforxxxobject blocking mechanism. on linux you can do libevent, epoll, select or pthread_cond_timedwait. all of these have "release on event or after timeout" semantics. you can use eventfd to combine them. i would not ever recommend relying on signals and writing custom cleanup handlers for them (!). unless they're blocked waiting for an external event, most system calls tend to return in a reasonable amount of time. handle the external event blocking scenario (stuff that select waits for) and you're basically there. moreover, if you're looking to exit cleanly, you probably don't want to take your chances interrupting syscalls with signals (!) anyway. > If you can't accept this, maybe don't play with threads, they are dangerous. too late. when i first started playing with threads, linux didn't really support them.
    - asveikau 1y ago
toast0 1y ago
Queuing a stop means you have to process the queue before stopping. Which certainly is stopping cleanly, but if you wanted to stop the thread because its queue was too long and the work requests were stale, it doesn't help much. You could maybe allow a queue skipping feature to be used for stop messages... But if it's only for stop messages, set an atomic bool stop, then send a stop message. If the thread just misses the stop bool and waits for messages, you'll get the stop message; if the queue is large, you'll get the stop bool. ps, hi
loeg 1y ago
> Relying heavily on a check for an atomic bool is prone to race conditions. It is not, actually. This extremely simple protocol is race-free.
- krackers 1y ago
  Calling pthread_cond_signal without acquiring mutex can lead to lost wakeup. And of course you can't really acquire a mutex in an async signal safe function like interrupt handler. Without the signaling thread acquiring a mutex, you might end up signaling after T2 has checked the boolean, but before it has called cond_wait. But this can be solved by processing the async signal in a deferred manner from some other watcher thread

PaulDavisThe1st 1y ago

Every event loop is subject to the blocked-due-to-long-running-computation issue. It bites ...

asveikau 1y ago
The same is true if you're repeatedly polling an atomic boolean in an event loop.
- CamperBob2 1y ago
  How so? It takes only a couple of machine cycles to poll a boolean. (And what other kind of boolean is there, besides atomic? It's either true or it's false, and if nothing can set it back to false once it goes true, I don't see the hazard. It's a CPU, not an FPGA.)
  - kbolino 1y ago
    The type is named atomic, but atomicity is not its only useful property. The atomic types also give control over the memory ordering, defaulting to sequentially consistent (seq_cst, the strongest). Without memory order guarantees enforced by memory barriers, a write to the boolean in thread A is not guaranteed to be observed by thread B. That matters both after initialization--where thread A sets the boolean to false but thread B may observe true, false, or invalid--and also after the transition--where thread B may fail to observe that the boolean has flipped from false to true. [edit: I'm not sure the above reasoning actually matters; as stated already by parent, "It's a CPU, not an FPGA"; modern multicore shared-memory CPUs have coherent caches]
    - CamperBob2 1y ago
      Well, it'll be observed the next time through the loop. If that matters, then it's true that this technique isn't desirable.
      - manwe150 1y ago
        Without atomic, the compiler won’t bother with there being a next time and just infinite loop (in the old days, you’d mark it volatile instead)
        CamperBob2 1y ago
        True enough, but volatile still works, of course.
    - adwn 1y ago
      > Without memory order guarantees enforced by memory barriers, a write to the boolean in thread A is not guaranteed to be observed by thread B. No, that's not correct. Memory ordering doesn't influence how fast a write is propagated to other cores, that's what cache coherency is for. Memory ordering of an access only matters in relation to accesses on other memory locations. There's a great introduction to this topic by Mara Bos: https://marabos.nl/atomics/memory-ordering.html https://marabos.nl/atomics/memory-ordering.html
      - kbolino 1y ago
        Indeed. I started to figure this out, hence my edit. Thanks for the link. There are hypothetical, historical, and special-purpose architectures which don't have cache coherency (or implement it differently enough to matter here), but for all practical purposes, it seems that all modern, general-purpose architectures implement it.
  - mattgreenrocks 1y ago
    It’s free on x86 for relaxed ordering, which is sufficient for this use case.

deleted 1y ago

[deleted]

a-dub 1y ago

disagree. i think then it's too tempting down the line for someone to add a message with blocking processing. a simple clear loop that looks for a requested stop flag with a confirmed stop flag works pretty well. this can be built into a synchronous "stop" function for the caller that sets the flag and then does a timed wait on the confirmation (using condition variables and pthread_cond_timedwait or waitforxxxobject if you're on windows).

asveikau 1y ago
Making your check less stable doesn't prevent this. The examples in this article IIRC were something like this. while (check_flag()) { do_stuff(); sleep_like_a_moron_instead_of_proper_blocking_mechanism(1second); } You're still going to be arbitrarily delayed if do_stuff() (or one one of its callees, maybe deep inside the stack) delays, or the sleep call. If you can't accept this, maybe don't play with threads, they are dangerous.
- a-dub 1y ago
  that's the point. use nonblocking io and an event polling mechanism with a timeout to keep an eye on an exit flag- that's all you need to handle clean shutdowns. i think on windows you can wait on both the sockets/file descriptors and condition variables with the same waitforxxxobject blocking mechanism. on linux you can do libevent, epoll, select or pthread_cond_timedwait. all of these have "release on event or after timeout" semantics. you can use eventfd to combine them. i would not ever recommend relying on signals and writing custom cleanup handlers for them (!). unless they're blocked waiting for an external event, most system calls tend to return in a reasonable amount of time. handle the external event blocking scenario (stuff that select waits for) and you're basically there. moreover, if you're looking to exit cleanly, you probably don't want to take your chances interrupting syscalls with signals (!) anyway. > If you can't accept this, maybe don't play with threads, they are dangerous. too late. when i first started playing with threads, linux didn't really support them.
  - asveikau 1y ago

toast0 1y ago

Queuing a stop means you have to process the queue before stopping. Which certainly is stopping cleanly, but if you wanted to stop the thread because its queue was too long and the work requests were stale, it doesn't help much. You could maybe allow a queue skipping feature to be used for stop messages... But if it's only for stop messages, set an atomic bool stop, then send a stop message. If the thread just misses the stop bool and waits for messages, you'll get the stop message; if the queue is large, you'll get the stop bool. ps, hi

loeg 1y ago

> Relying heavily on a check for an atomic bool is prone to race conditions. It is not, actually. This extremely simple protocol is race-free.

krackers 1y ago
Calling pthread_cond_signal without acquiring mutex can lead to lost wakeup. And of course you can't really acquire a mutex in an async signal safe function like interrupt handler. Without the signaling thread acquiring a mutex, you might end up signaling after T2 has checked the boolean, but before it has called cond_wait. But this can be solved by processing the async signal in a deferred manner from some other watcher thread