11 ms·

>The more C++ you know the worse you become at working with others. First, because your particular dialects of C++ tend to isolate you. So why is using somethi

by jasode 11y ago

>The more C++ you know the worse you become at working with others. First, because your particular dialects of C++ tend to isolate you. So why is using something non-standard and non-universal like <czmq.h> not perceived as a "dialect" of C?[1] Any non-trivial source codebase beyond "printf("hello world")" will be a "dialect" of the programmer(s). When looking at the C source files of Linux kernel, Redis, and SQLite, etc, the syntax patterns, helper macros, string manipulations, etc do not look the same. Also the author's example of, for (i = List.begin (); i != List.end (); ++i) cout << *i << " "; is not the same semantics as: char *fruit = (char *) zlist_first (list); while (fruit) { printf ("%s ", fruit); fruit = (char *) zlist_next (list); } The C++ loop is multithread safe. The C version is not. For the zlist_next() to work, the "list" data structure needs to maintain mutable state in between subsequent calls. (Think of how something like strtok() works by mutating the string). [1]possibly because Pieter Hintjens is the programmer & CEO behind ZeroMQ and czmq.h. Therefore, it doesn't feel like a dialect to him?

exDM69 11y ago
> The C++ loop is multithread safe. The C version is not. Neither of these loops are multi-thread safe. That's assuming that's an std::list, and even if it isn't you can't implement that C++ API in a fast and thread safe manner. Yes, the ZMQ list is awkward (why did they put the "cursor" or the iterator inside the list object)? But that's not an inherent property of the C language. If you want an example of how to implement a linked list in C elegantly and efficiently, look at the intrusive linked list implementation in the Linux kernel (also available separately in the CCAN project).
- jasode 11y ago
  >Neither of these loops are multi-thread safe. Yes, as a general rule, all STL containers in C++ are not thread safe. (E.g. Write operations need locks. Read ops also need locks if other threads can write to the container.) However, I was looking at the specific loop example and only saw .begin(), .end(), and a pointer dereference which are all read operations. These should all be thread safe (assuming no other threads modifying the list) and I wasn't aware of any documentation that contradicts that.
  - Aeolos 11y ago
    What if a background thread deletes a node from the list while the iterator is running? From the cpp reference: "Concurrently accessing or modifying other elements is safe, although iterating through the container is not." That's the problem with multithreaded code. You can't just see a read loop in isolation and say "this is thread-safe" - it's not.
  - exDM69 11y ago
    > These should all be thread safe (assuming no other threads modifying the list) and I wasn't aware of any documentation that contradicts that. In other words: this code would be thread safe if protected by a readers-writers lock or similar synchronization mechanism :)
    - jasode 11y ago
      >In other words: this code would be thread safe if protected by a readers-writers lock or similar synchronization mechanism The outer context can affect whether or not an STL begin()/end() loop needs locks. For example, many programs can be coded to work in phases: main() { setup_lookup_tables(); // phase 1: fill up (write) to lists/vectors/maps analyze(); // phase 2: spawn multiple threads to read the lists/vectors/maps } The code in the analyze() doesn't need to pay for millions of expensive locks+unlocks that slow things down. The programmer knows there are no writers to STL containers in the analyze() phase. With the assumption of no concurrent writers, the programmer can safely spawn a dozen threads with for(list.begin();list.end();) and get 100% determinism. Explicit sequential ordering can let one avoid unnecessary locks. In this limited sense, the loop is multithread safe for multiple readers. However, even with the assumption of no writers to zcmq's "list", you still cannot spawn multiple reading threads of zlist_next() because the cursor itself is mutating inside the list. The result will not be deterministic. That's the type of comparison I was making. You also have a valid point about general multi-threaded code covering all cases (implied concurrent writers).
- PieterH 11y ago
  The CZMQ list embeds the cursor because (a) it is sufficient for all cases we've encountered and (b) it makes calling code simpler. This iteration pattern is widely used in CZMQ and Scalable C applications like Zyre and Malamute.
jokoon 11y ago
Also, it's not always a problem of being able to work with other. Having a standard also means that other compilers will compile your C++ code. We're not talking about a language, we're talking about a programming language, and standards and compilers are the only metrics which make that language "talked" by others.
- michaelmior 11y ago
  Isn't a programming language both? Sure the goal is arguably to communicate some process for the computer to perform in a machine-readable way. But I can easily write two valid semantically equivalent programs which produce equivalent code when fed to a standards compliant compiler. However, one of these programs may be much more readable to a human because it follows an established set of coding practices. Perhaps a better word that standards would be "conventions" but either way, most programs also need to be read by humans at some point.
- Aeolos 11y ago
  > Having a standard also means that other compilers will compile your C++ code. As someone maintaining code that must run on VS2012, VS2013, VS2015, GCC 4.7+, clang 3.5+, on both x86 and x86_64, I chuckled hard at your comment. Let's just say that I'm looking forward to moving on with my career in a few months. Hopefully to something that doesn't involve maintaining C++ code.
vidarh 11y ago
The C loop is not multithread safe because of this right in "why this book exists": > The standard approach for concurrency is POSIX threads that share their state. This is complex and fragile. We know how to do this better, using message passing between threads. In other words, the code he's presenting is not expected to need to be thread safe because the structures are not expected to be accessed from more than one thread.
- jasode 11y ago
  >is not expected to need to be thread safe I'm not "expecting" his example code to be multithread safe either. (By default, I expect all code to not be multithread safe unless it is explicitly engineered to be so.) I'm pointing out that the code does not work the same way under the covers. The author would know there's mutable state hidden in that "list" object but I don't think it would be obvious to others at first glance. If it was, programmers wouldn't get bitten by bugs because of misunderstanding ctime() and strtok() in single-threaded programs. Likewise, the zlist_next() can also be inadvertently misused to cause bugs in single-threaded programs. If the author knows how czmq internally works and how it's supposed to be used (don't multi-thread or misuse in single-thread), but outsiders haven't grokked those guidelines yet, can that coding paradigm be labeled as a "dialect"? That's the interesting philosophical question to me. Words like "dialect", "explicit", "syntactic tricks" are relative to the programmer and I think the author's code examples illustrate those subjective biases rather than advise us of objective truth.
  - vidarh 11y ago
    I think there is a fundamental difference there in that as you say: You by default expect all code to not be multithread safe. That's a default assumption for most C/C++ developers when encountering a class or library they don't know because it is very common. To the extent that quite a few people argue that libraries should not take extra effort to be thread safe, as is the case here. Besides, it is obvious from the API in this case that there is hidden state to all but extreme beginners. I think his argument regarding dialects in C++ is sound to an extent. C is quite boring and settled when it comes to structure and interop at the API level. APIs may vary wildly, but e.g. <libraryidentifier>_<classname>_<methodname> or <libraryidentifier>_<functionname> with pointer to an "object" or struct holding generic state is very well established, for example. A lot of the simplicity there comes from the simplicity of the language. The internals of how the code is written varies more, and things like build systems etc. varies wildly. But at the end of it you are usually left with some headers and a bunch of exposed symbols that follows a relatively limited set of styles. C++ has far trickier issues, in that you are dealing with people who wrap everything in templates; people who try to use functors all over the place; people who want exception safety everywhere; people who ban exceptions in their codebases; smart pointers vs. bare pointers etc. C library code can usually interoperate. C++ code written in different "cultures" can sometimes be like trying to make LEGO and PlayMobil fit toys fit together. Many C++ developers have gone through "phases" - for my own part, before I mostly ditched C++ I'd written codes in styles that appeared totally alien to each other...
twoodfin 11y ago
Is the zlist version even re-entrant?
- jasode 11y ago
  That's also a good point. You can't use zlist* functions in recursive algorithms in single-threaded code. Whether such recursive code would be an artificial contrivance or a real-world misuse is something left to the reader's imagination. I notice that the zlist_copy() function was deprecated so you'd have to do something else if you wanted to pass a copy of a new "list" to subsequent recursive calls. I didn't dig deeper into the czmq documentation to find out what the recommended guidelines would be. (Although we'd guess that any copy operation would be O(n) and therefore thinking in terms of recursion would be an antipattern for czmq.)
  - bluski 11y ago
    You don't need recursion to hit the wall - how about comparing all pairs of elements? I.e. inner loop that iterates over all elements once again.
  - PieterH 11y ago
    It's zlist_dup() and you would not use recursion to process these types of lists. The API is designed for iteration. One case for recursion is when you use lists of lists of lists... you can then process such structures recursively. See zconfig for examples of that.
kazinator 11y ago
zlist_first and zlist_next can be made thread safe by using a thread-specific variable for the iteration cursor, rather than a simple member of the list structure. Of course, that's a big, inefficient hack, and tied to a particular threading model (not generally reentrant). Anyway, it's probably largely moot because neither example is safe if the list is treated as mutable (there is at least one writer concurrent with the reader).
- deleted 11y ago
  [deleted]
pjmlp 11y ago
> So why is using something non-standard and non-universal like <czmq.h> not perceived as a "dialect" of C? Somehow many fail to acknowledge that many write in compiler dialects, and think "my compiler == C language". Yet commercial compiler vendors are always criticized for their language extensions or having other interpretations for the parts not defined in ANSI C.
PieterH 11y ago
CZMQ is a library of around 30K lines of code. The Scalable C book is in many ways a guide to using that library in real projects. Of course every programmer develops their own dialects, just as every writer has their own voice. C is however a small language and if you stay away from weird macro magic, any well-written C code is mutually intelligible to other C programmers. Whereas with C++, dialects can have so little overlap they are not mutually intelligible. FWIW neither of the fragments is safe if you are sharing state between threads. In practice we do not share objects between threads, and each object holds its own state, and thus our C code is 100% thread safe and reentrant.
- jasode 11y ago
  >The Scalable C book is in many ways a guide to using that library in real projects. I'm not criticizing the book or czmq. I'm sure it's a fine library. I just found your characterizations of C++ to be strange. For example: ">, any well-written C code is mutually intelligible to other C programmers." If you're going to qualify C code with "well-written" to help make your point then can't the same qualification be applied to C++? If so, it means you believe that there is "well-written C++ code" that is simultaneously unintelligible to other programmers. In your opinion, what would be an example of that? (E.g. If you can point to github of unintelligible but well-written C++ source code.) And your other statement: >, because your particular dialects of C++ tend to isolate you. I don't know if CryEngine and Unreal game engines are well-written C++ but they seem attract developers. There's also the scuba diving app[1] written in C++ that also has Linus (who you already know hates C++) modifying the source code. I contend there are bigger factors than C++ dialects causing isolation. The C++ ABI may be more isolating than the C ABI because of C++ name mangling incompatibilities. But I don't see how the C++ dialects (of well-written code) are isolating. [1]https://github.com/torvalds/subsurface https://github.com/torvalds/subsurface
  - microcolonel 11y ago
    Given the sheer breadth of approaches and features in C++, any one codebase will only touch some of it. Inevitably this means that if you get used to C++ in one project, you will have more difficulty moving to projects using a different subset.
abcd_f 11y ago
> The C++ loop is multithread safe. Of course, it's not. If another thread removes the item that i refers to while i's thread is in its cout part, then this loop will go out in flames. Either functionally or semantically. And if it will not (meaning that List explicitly supports this sort of re-entrant case), then you basically get here not just a list, but some multi-purpose construct that can withstand a nuclear strike and the fallout - not something you'd actually expect when looking for a basic list container. However this is exactly the kind of thing you'd typically get in C++. In contrast, with C you get a basic minimum that does what it says on a tin - links things into a list. You want it "multithread safe", you arrange for that externally. You happen to need that frequently, you wrap it into mt_safe_list container. That's the key difference between C and C++ mentality. C++ does things universally and you can pick a sliver that you want to actually work with (leading to dialects), while C generally favors doing things in small functional units and combining them as needed. The Unix way. You can, technically, call the result of such combining a dialect, but that's quite a bit different dialects arising from the C++ usage.

by jasode 11y ago

exDM69 11y ago

> The C++ loop is multithread safe. The C version is not. Neither of these loops are multi-thread safe. That's assuming that's an std::list, and even if it isn't you can't implement that C++ API in a fast and thread safe manner. Yes, the ZMQ list is awkward (why did they put the "cursor" or the iterator inside the list object)? But that's not an inherent property of the C language. If you want an example of how to implement a linked list in C elegantly and efficiently, look at the intrusive linked list implementation in the Linux kernel (also available separately in the CCAN project).

jasode 11y ago
>Neither of these loops are multi-thread safe. Yes, as a general rule, all STL containers in C++ are not thread safe. (E.g. Write operations need locks. Read ops also need locks if other threads can write to the container.) However, I was looking at the specific loop example and only saw .begin(), .end(), and a pointer dereference which are all read operations. These should all be thread safe (assuming no other threads modifying the list) and I wasn't aware of any documentation that contradicts that.
- Aeolos 11y ago
  What if a background thread deletes a node from the list while the iterator is running? From the cpp reference: "Concurrently accessing or modifying other elements is safe, although iterating through the container is not." That's the problem with multithreaded code. You can't just see a read loop in isolation and say "this is thread-safe" - it's not.
- exDM69 11y ago
  > These should all be thread safe (assuming no other threads modifying the list) and I wasn't aware of any documentation that contradicts that. In other words: this code would be thread safe if protected by a readers-writers lock or similar synchronization mechanism :)
  - jasode 11y ago
    >In other words: this code would be thread safe if protected by a readers-writers lock or similar synchronization mechanism The outer context can affect whether or not an STL begin()/end() loop needs locks. For example, many programs can be coded to work in phases: main() { setup_lookup_tables(); // phase 1: fill up (write) to lists/vectors/maps analyze(); // phase 2: spawn multiple threads to read the lists/vectors/maps } The code in the analyze() doesn't need to pay for millions of expensive locks+unlocks that slow things down. The programmer knows there are no writers to STL containers in the analyze() phase. With the assumption of no concurrent writers, the programmer can safely spawn a dozen threads with for(list.begin();list.end();) and get 100% determinism. Explicit sequential ordering can let one avoid unnecessary locks. In this limited sense, the loop is multithread safe for multiple readers. However, even with the assumption of no writers to zcmq's "list", you still cannot spawn multiple reading threads of zlist_next() because the cursor itself is mutating inside the list. The result will not be deterministic. That's the type of comparison I was making. You also have a valid point about general multi-threaded code covering all cases (implied concurrent writers).
PieterH 11y ago
The CZMQ list embeds the cursor because (a) it is sufficient for all cases we've encountered and (b) it makes calling code simpler. This iteration pattern is widely used in CZMQ and Scalable C applications like Zyre and Malamute.

jokoon 11y ago

Also, it's not always a problem of being able to work with other. Having a standard also means that other compilers will compile your C++ code. We're not talking about a language, we're talking about a programming language, and standards and compilers are the only metrics which make that language "talked" by others.

michaelmior 11y ago
Isn't a programming language both? Sure the goal is arguably to communicate some process for the computer to perform in a machine-readable way. But I can easily write two valid semantically equivalent programs which produce equivalent code when fed to a standards compliant compiler. However, one of these programs may be much more readable to a human because it follows an established set of coding practices. Perhaps a better word that standards would be "conventions" but either way, most programs also need to be read by humans at some point.
Aeolos 11y ago
> Having a standard also means that other compilers will compile your C++ code. As someone maintaining code that must run on VS2012, VS2013, VS2015, GCC 4.7+, clang 3.5+, on both x86 and x86_64, I chuckled hard at your comment. Let's just say that I'm looking forward to moving on with my career in a few months. Hopefully to something that doesn't involve maintaining C++ code.

vidarh 11y ago

The C loop is not multithread safe because of this right in "why this book exists": > The standard approach for concurrency is POSIX threads that share their state. This is complex and fragile. We know how to do this better, using message passing between threads. In other words, the code he's presenting is not expected to need to be thread safe because the structures are not expected to be accessed from more than one thread.

jasode 11y ago
>is not expected to need to be thread safe I'm not "expecting" his example code to be multithread safe either. (By default, I expect all code to not be multithread safe unless it is explicitly engineered to be so.) I'm pointing out that the code does not work the same way under the covers. The author would know there's mutable state hidden in that "list" object but I don't think it would be obvious to others at first glance. If it was, programmers wouldn't get bitten by bugs because of misunderstanding ctime() and strtok() in single-threaded programs. Likewise, the zlist_next() can also be inadvertently misused to cause bugs in single-threaded programs. If the author knows how czmq internally works and how it's supposed to be used (don't multi-thread or misuse in single-thread), but outsiders haven't grokked those guidelines yet, can that coding paradigm be labeled as a "dialect"? That's the interesting philosophical question to me. Words like "dialect", "explicit", "syntactic tricks" are relative to the programmer and I think the author's code examples illustrate those subjective biases rather than advise us of objective truth.
- vidarh 11y ago
  I think there is a fundamental difference there in that as you say: You by default expect all code to not be multithread safe. That's a default assumption for most C/C++ developers when encountering a class or library they don't know because it is very common. To the extent that quite a few people argue that libraries should not take extra effort to be thread safe, as is the case here. Besides, it is obvious from the API in this case that there is hidden state to all but extreme beginners. I think his argument regarding dialects in C++ is sound to an extent. C is quite boring and settled when it comes to structure and interop at the API level. APIs may vary wildly, but e.g. <libraryidentifier>_<classname>_<methodname> or <libraryidentifier>_<functionname> with pointer to an "object" or struct holding generic state is very well established, for example. A lot of the simplicity there comes from the simplicity of the language. The internals of how the code is written varies more, and things like build systems etc. varies wildly. But at the end of it you are usually left with some headers and a bunch of exposed symbols that follows a relatively limited set of styles. C++ has far trickier issues, in that you are dealing with people who wrap everything in templates; people who try to use functors all over the place; people who want exception safety everywhere; people who ban exceptions in their codebases; smart pointers vs. bare pointers etc. C library code can usually interoperate. C++ code written in different "cultures" can sometimes be like trying to make LEGO and PlayMobil fit toys fit together. Many C++ developers have gone through "phases" - for my own part, before I mostly ditched C++ I'd written codes in styles that appeared totally alien to each other...

twoodfin 11y ago

Is the zlist version even re-entrant?

jasode 11y ago
That's also a good point. You can't use zlist* functions in recursive algorithms in single-threaded code. Whether such recursive code would be an artificial contrivance or a real-world misuse is something left to the reader's imagination. I notice that the zlist_copy() function was deprecated so you'd have to do something else if you wanted to pass a copy of a new "list" to subsequent recursive calls. I didn't dig deeper into the czmq documentation to find out what the recommended guidelines would be. (Although we'd guess that any copy operation would be O(n) and therefore thinking in terms of recursion would be an antipattern for czmq.)
- bluski 11y ago
  You don't need recursion to hit the wall - how about comparing all pairs of elements? I.e. inner loop that iterates over all elements once again.
- PieterH 11y ago
  It's zlist_dup() and you would not use recursion to process these types of lists. The API is designed for iteration. One case for recursion is when you use lists of lists of lists... you can then process such structures recursively. See zconfig for examples of that.

kazinator 11y ago

zlist_first and zlist_next can be made thread safe by using a thread-specific variable for the iteration cursor, rather than a simple member of the list structure. Of course, that's a big, inefficient hack, and tied to a particular threading model (not generally reentrant). Anyway, it's probably largely moot because neither example is safe if the list is treated as mutable (there is at least one writer concurrent with the reader).

deleted 11y ago
[deleted]

pjmlp 11y ago

> So why is using something non-standard and non-universal like <czmq.h> not perceived as a "dialect" of C? Somehow many fail to acknowledge that many write in compiler dialects, and think "my compiler == C language". Yet commercial compiler vendors are always criticized for their language extensions or having other interpretations for the parts not defined in ANSI C.

PieterH 11y ago

CZMQ is a library of around 30K lines of code. The Scalable C book is in many ways a guide to using that library in real projects. Of course every programmer develops their own dialects, just as every writer has their own voice. C is however a small language and if you stay away from weird macro magic, any well-written C code is mutually intelligible to other C programmers. Whereas with C++, dialects can have so little overlap they are not mutually intelligible. FWIW neither of the fragments is safe if you are sharing state between threads. In practice we do not share objects between threads, and each object holds its own state, and thus our C code is 100% thread safe and reentrant.

jasode 11y ago
>The Scalable C book is in many ways a guide to using that library in real projects. I'm not criticizing the book or czmq. I'm sure it's a fine library. I just found your characterizations of C++ to be strange. For example: ">, any well-written C code is mutually intelligible to other C programmers." If you're going to qualify C code with "well-written" to help make your point then can't the same qualification be applied to C++? If so, it means you believe that there is "well-written C++ code" that is simultaneously unintelligible to other programmers. In your opinion, what would be an example of that? (E.g. If you can point to github of unintelligible but well-written C++ source code.) And your other statement: >, because your particular dialects of C++ tend to isolate you. I don't know if CryEngine and Unreal game engines are well-written C++ but they seem attract developers. There's also the scuba diving app[1] written in C++ that also has Linus (who you already know hates C++) modifying the source code. I contend there are bigger factors than C++ dialects causing isolation. The C++ ABI may be more isolating than the C ABI because of C++ name mangling incompatibilities. But I don't see how the C++ dialects (of well-written code) are isolating. [1]https://github.com/torvalds/subsurface https://github.com/torvalds/subsurface
- microcolonel 11y ago
  Given the sheer breadth of approaches and features in C++, any one codebase will only touch some of it. Inevitably this means that if you get used to C++ in one project, you will have more difficulty moving to projects using a different subset.

abcd_f 11y ago

> The C++ loop is multithread safe. Of course, it's not. If another thread removes the item that i refers to while i's thread is in its cout part, then this loop will go out in flames. Either functionally or semantically. And if it will not (meaning that List explicitly supports this sort of re-entrant case), then you basically get here not just a list, but some multi-purpose construct that can withstand a nuclear strike and the fallout - not something you'd actually expect when looking for a basic list container. However this is exactly the kind of thing you'd typically get in C++. In contrast, with C you get a basic minimum that does what it says on a tin - links things into a list. You want it "multithread safe", you arrange for that externally. You happen to need that frequently, you wrap it into mt_safe_list container. That's the key difference between C and C++ mentality. C++ does things universally and you can pick a sliver that you want to actually work with (leading to dialects), while C generally favors doing things in small functional units and combining them as needed. The Unix way. You can, technically, call the result of such combining a dialect, but that's quite a bit different dialects arising from the C++ usage.