4 ms·

> The C++ loop is multithread safe. The C version is not. Neither of these loops are multi-thread safe. That's assuming that's an std::list, and even if it isn

by exDM69 11y ago

> The C++ loop is multithread safe. The C version is not. Neither of these loops are multi-thread safe. That's assuming that's an std::list, and even if it isn't you can't implement that C++ API in a fast and thread safe manner. Yes, the ZMQ list is awkward (why did they put the "cursor" or the iterator inside the list object)? But that's not an inherent property of the C language. If you want an example of how to implement a linked list in C elegantly and efficiently, look at the intrusive linked list implementation in the Linux kernel (also available separately in the CCAN project).

jasode 11y ago
>Neither of these loops are multi-thread safe. Yes, as a general rule, all STL containers in C++ are not thread safe. (E.g. Write operations need locks. Read ops also need locks if other threads can write to the container.) However, I was looking at the specific loop example and only saw .begin(), .end(), and a pointer dereference which are all read operations. These should all be thread safe (assuming no other threads modifying the list) and I wasn't aware of any documentation that contradicts that.
- Aeolos 11y ago
  What if a background thread deletes a node from the list while the iterator is running? From the cpp reference: "Concurrently accessing or modifying other elements is safe, although iterating through the container is not." That's the problem with multithreaded code. You can't just see a read loop in isolation and say "this is thread-safe" - it's not.
- exDM69 11y ago
  > These should all be thread safe (assuming no other threads modifying the list) and I wasn't aware of any documentation that contradicts that. In other words: this code would be thread safe if protected by a readers-writers lock or similar synchronization mechanism :)
  - jasode 11y ago
    >In other words: this code would be thread safe if protected by a readers-writers lock or similar synchronization mechanism The outer context can affect whether or not an STL begin()/end() loop needs locks. For example, many programs can be coded to work in phases: main() { setup_lookup_tables(); // phase 1: fill up (write) to lists/vectors/maps analyze(); // phase 2: spawn multiple threads to read the lists/vectors/maps } The code in the analyze() doesn't need to pay for millions of expensive locks+unlocks that slow things down. The programmer knows there are no writers to STL containers in the analyze() phase. With the assumption of no concurrent writers, the programmer can safely spawn a dozen threads with for(list.begin();list.end();) and get 100% determinism. Explicit sequential ordering can let one avoid unnecessary locks. In this limited sense, the loop is multithread safe for multiple readers. However, even with the assumption of no writers to zcmq's "list", you still cannot spawn multiple reading threads of zlist_next() because the cursor itself is mutating inside the list. The result will not be deterministic. That's the type of comparison I was making. You also have a valid point about general multi-threaded code covering all cases (implied concurrent writers).
PieterH 11y ago
The CZMQ list embeds the cursor because (a) it is sufficient for all cases we've encountered and (b) it makes calling code simpler. This iteration pattern is widely used in CZMQ and Scalable C applications like Zyre and Malamute.

by exDM69 11y ago

jasode 11y ago

>Neither of these loops are multi-thread safe. Yes, as a general rule, all STL containers in C++ are not thread safe. (E.g. Write operations need locks. Read ops also need locks if other threads can write to the container.) However, I was looking at the specific loop example and only saw .begin(), .end(), and a pointer dereference which are all read operations. These should all be thread safe (assuming no other threads modifying the list) and I wasn't aware of any documentation that contradicts that.

Aeolos 11y ago
What if a background thread deletes a node from the list while the iterator is running? From the cpp reference: "Concurrently accessing or modifying other elements is safe, although iterating through the container is not." That's the problem with multithreaded code. You can't just see a read loop in isolation and say "this is thread-safe" - it's not.
exDM69 11y ago
> These should all be thread safe (assuming no other threads modifying the list) and I wasn't aware of any documentation that contradicts that. In other words: this code would be thread safe if protected by a readers-writers lock or similar synchronization mechanism :)
- jasode 11y ago
  >In other words: this code would be thread safe if protected by a readers-writers lock or similar synchronization mechanism The outer context can affect whether or not an STL begin()/end() loop needs locks. For example, many programs can be coded to work in phases: main() { setup_lookup_tables(); // phase 1: fill up (write) to lists/vectors/maps analyze(); // phase 2: spawn multiple threads to read the lists/vectors/maps } The code in the analyze() doesn't need to pay for millions of expensive locks+unlocks that slow things down. The programmer knows there are no writers to STL containers in the analyze() phase. With the assumption of no concurrent writers, the programmer can safely spawn a dozen threads with for(list.begin();list.end();) and get 100% determinism. Explicit sequential ordering can let one avoid unnecessary locks. In this limited sense, the loop is multithread safe for multiple readers. However, even with the assumption of no writers to zcmq's "list", you still cannot spawn multiple reading threads of zlist_next() because the cursor itself is mutating inside the list. The result will not be deterministic. That's the type of comparison I was making. You also have a valid point about general multi-threaded code covering all cases (implied concurrent writers).

PieterH 11y ago

The CZMQ list embeds the cursor because (a) it is sufficient for all cases we've encountered and (b) it makes calling code simpler. This iteration pattern is widely used in CZMQ and Scalable C applications like Zyre and Malamute.