6 ms·

I have to admit that I have an extremely visceral, negative feeling whenever I see a mutex, simply because I've had to debug enough code written by engineers wh

• aidenn0 2y ago
  I feel the similarly about C"s "volatile" (when used in multithreaded code rather than device drivers). I've seen people scatter volatile around randomly until the problem goes away. Given that volatile significantly disturbs the timing of a program, any timing sensitive bugs can be masked by adding it around randomly.
  • cogman10 2y ago
    There seems to be a lot of voodoo beliefs around concurrent programming that lead to really bad things. One of the best books I've read on it is Java concurrency in practice [1]. It does an excellent job of dispelling these occultic beliefs and letting the reader know exactly when and how concurrency should be implemented. It is applicable to more languages than just java, especially since many have adopted large parts of the java memory model. The worst things I usually find when reviewing concurrent code is people either not using locks when they should, using locks when they shouldn't, and having inconsistent data guards. I've seen people throw in random locks to guard local non-shared state which is just crazy town but "Multiple threads are running this code, so I'm adding a lock". I certainly prefer message passing over shared state. However, it's a little baffling to me why it's so hard for devs to grasp how to properly maintain shared state. Instead of just learning the basic rules, it gets couched in "It's just too hard to understand so keep adding things until it works". [1] https://www.amazon.com/Java-Concurrency-Practice-Brian-Goetz/dp/0321349601 https://www.amazon.com/Java-Concurrency-Practice-Brian-Goetz...
    • deleted 2y ago
      [deleted]
    • LegionMammal978 2y ago
      > However, it's a little baffling to me why it's so hard for devs to grasp how to properly maintain shared state. Instead of just learning the basic rules, it gets couched in "It's just too hard to understand so keep adding things until it works". Probably because most people aren't aware that there are basic rules to be learned. I'd imagine the typical experience is, you're very familiar with single-threaded code, and now you're trying to let other threads work with your data. You have heard that there are many pitfalls, and that there are special-purpose tools like mutexes to avoid those, but you look at the examples and find them mostly baffling. "Why do they perform these incantations for this data but not that data, or in this place but not that place?" So you come up with some weird mental model and move on with your life, never aware that there are underlying principles for maintaining shared state. Personally, I didn't understand mutexes very well at all, until I started looking into what the atomic memory orderings from C++ et al. were supposed to mean.
      • Maxatar 2y ago
        Not too sure what the basic rules are and I'm not able to find any list of such rules. For me the biggest challenge when sharing state is that the only benefit I can see for parallelism is performance, so if I'm not gaining performance there is no reason to use parallelism. If I use coarse-grained mutexes then I end up with straight forward to reason about code but I lose the performance benefit and in fact can end up with slower than single threaded code. If I use very fine grained mutexes then I end up with faster code that has very hard to find bugs that happen on very rare occasion. And then on top of that even if you do write correct fine grained locking, you can still end up with slow code due to cache behavior such as false sharing and cache coherence. So ultimately I disagree that writing parallel code is simple unless you're willing to give up performance in which case you may as well just stick to single threaded code or use parallelism among independent data. Writing correct parallel software that shares state and actually delivers substantial performance benefits is incredibly difficult, and I am skeptical that there is a set of simple rules that one can simply read about.
  • tialaramex 2y ago
    In most cases (in a C or C++ compiler, not Java) it's just straight up incorrect to use volatile for something other than memory mapped I/O. Yes, POSIX guarantees that in a specific case (signal handling IIRC) it'll do what you meant if you use volatile int. That's nice, but more generally this is not the right choice. Unfortunately Microsoft enshrined the situation (on Windows, on their compiler, on x86 and x86-64 only) that volatile primitive types are effectively atomics with Acquire-Release ordering. This is of course awkward when Microsoft tries to bring people to a non-x86 architecture and it can't just give them this because it would suck really hard, so finally they have to grow up and teach their developers about actual atomics. !! Edited to fix: Previously this said Relaxed ordering, the ordering guaranteed by Microsoft is in fact Acquire-Release, hence it's expensive to provide for architectures where that's not the default.
    • hinkley 2y ago
      When Java implemented volatile it didn’t do anything. Later when they fixed the memory model to deal with out of order execution they made it part of the fence semantics, and then it actually made some sense.
  • zozbot234 2y ago
    The "volatile" keyword should never be used for C/C++ multithreaded code. It's specifically intended for access to device-mapped addresses and does not account for any specific memory model, so using it for multithreading will lead to breakage. Please use the C/C++ memory model facilities instead. (As a contrast, note that in Java the "volatile" keyword can be used for multithreading, but again this does not apply to C/C++.)
    • hinkley 2y ago
      I’m surprised that’s true. C borrowed very heavily from Java when fixing the NUMA situations that were creeping into modern processors.
      • jcranmer 2y ago
        The C/C++ memory model is directly derived from the Java 5 memory model. However, the decision was made that volatile in C/C++ specifically referred to memory-mapped I/O stuff, and the extra machinery needed to effect the sequential consistency guarantees was undesirable. As a result, what is volatile in Java is _Atomic in C and std::atomic in C++. C/C++ also went further and adopted a few different notions of atomic variables, so you can choose between a sequentially-consistent atomic variable, a release/acquire atomic variable, a release/consume atomic variable (which ended up going unimplemented for reasons), and a fully relaxed atomic variable (whose specification turned out to be unexpectedly tortuous).
        • tialaramex 2y ago
          Importantly these aren't types they're operations. So it's not that you have a "release/acquire atomic variable" but you have an atomic variable and it so happens you choose to do a Release store to that variable, in other code maybe you do a Relaxed fetch from the same variable, elsewhere you have a compare exchange with different ordering rules Since we're talking about Mutex here, here's the entirety of Rust's "try_lock" for Mutex on a Linux-like platform: self.futex.compare_exchange(UNLOCKED, LOCKED, Acquire, Relaxed).is_ok() That's a single atomic operation, in which we hope the futex is UNLOCKED, if it is we store LOCKED to it with Acquire ordering, but, if it wasn't we use a Relaxed load to find out what it was instead of UNLOCKED. We actually don't do anything with that load, but the Ordering for both operations is specified here, not when the variable was typed.
  • senderista 2y ago
    If you only use volatile in C without any atomic operations or fences, then your multithreaded code is certainly incorrect.
• jart 2y ago
  What are some examples of people using mutexes wrong? I know one gotcha is you need to maintain a consistent hierarchy. Usually the easiest way to not get snagged by that, is to have critical sections be small and pure. Java's whole MO of letting people add a synchronized keyword to an entire method was probably not the greatest idea.
  • cogman10 2y ago
    When, how, and why. The biggest part of mutexes and how to properly use them is thinking of the consistency of the data that you are working with. Here's a really common bug (psuedocode) if (lock {data.size()} > 0) { value = lock { data.pop() } lock { foo.add(value) } } The issue here is size can change, pop can change, and foo can change in unexpected ways between each of the acquired locks. The right way to write this code is lock { if (data.size() > 0) { value = data.pop() foo.add(value) } } That ensures the data is all in a consistent state while you are mutating it. Now, what does make this tricky is someone well-meaning might have decided to push the lock down a method. Imagine, for example, you have a `Foo` where all methods operate within a mutex. This code is also (likely) incorrect. value = foo.bar() if (value.bat()) { foo.baz(value) } The problem here is exactly the same problem as above. Between `foo.bar()` and `foo.baz()` the state of foo may have changed such that running `foo.baz(value)` is now a mistake. That's why the right thing to do is likely to have a `foo.barBaz()` method that encapsulates the `if` logic to avoid inconsistency (or to add another mutex). In java, the most common manifestation (that I see) of this looks like this var map = new ConcurrentHashMap(); if (map.get(foo) == null) map.put(foo, new Value()); Because now, you have a situation where the value of `foo` in the map could be 2 or more values depending on who gets it. So, if someone is mutating `Value` concurrently you have a weird hard to track down data race. The solution to this problem in java is map.computeIfAbsent(foo, (unused)->new Value());
    • hinkley 2y ago
      Composing locks is where Java usually blows up. And computeIfAbsent can end up holding the lock for too long if the function is slow.
• deleted 2y ago
  [deleted]
• bob1029 2y ago
  > some kind of queue or messaging abstraction Agreed. I find things like LMAX Disruptor much easier to reason about.
  • tombert 2y ago
    Even within Java, something like BlockingQueue will get you pretty far, and that's built into the runtime. If I am allowed to use libraries, I end up using Vert.x for nearly everything. I think that their eventbus abstraction is easy enough to reason about, and even without using it simply using the non-blocking stuff it provides ends up being pretty handy.
• jeffbee 2y ago
  Message passing is just outsourcing the lock, right? For example a Go channel is internally synchronized, nothing magic about it. Most of the mutex tragedies I have seen in my career have been in C, a useless language without effective scopes. In C++ it's pretty easy to use a scoped lock. In fact I'd say I have had more trouble with people who are trying to avoid locks than with people who use them. The avoiders either think their program order is obviously correct (totally wrong on modern CPUs) or that their atomics are faster (wrong again on many CPUs).
  • loeg 2y ago
    > Message passing is just outsourcing the lock, right? More or less, yeah. You can write an MPSC queue that doesn't explicitly use a lock (or even anything that looks like a lock).
  • tombert 2y ago
    It's definitely doing synchronization behind the scenes, no argument here. BlockingQueues in Java seem to use ReentrantLocks everywhere. It's outsourcing the lock to people who understand locks better. It just abstracts this detail away for me, and I personally trust the libraries implementing these abstractions to be more correct than some ad hoc thing I write. It's an abstraction that I personally find a lot easier to reason about, and so my thinking is this: if my reasoning is more likely to be correct because of the easier abstraction, and the internal synchronization is more likely to be correct, then it's more likely that my code will be correct. I don't do super low-level stuff at all, most of my stuff ends up touching a network, so the small differences between the built-in synchronized structures vs the regular ones really don't matter since any small gains I'd get on that will be eaten the first time I hit the network, so a considerably higher ROI for me is almost always figuring out how to reduce latency. If I did C or C++, I'd probably have different opinions on this stuff.
  • foobazgt 2y ago
    Every abstraction is about outsourcing the thing it's abstracting away. If using a queue solves your problem, you no longer have to deal with all the headaches that you can run into using a bare mutex.
  • 2y ago
• another-acct 2y ago
  > remove locks from code and replace with some kind of queue or messaging abstraction Shared-nothing message passing reflects the underlying (modern) computer architecture more closely, so I'd call the above a good move. Shared memory / symmetric multiprocessing is an abstraction that leaks like a sieve; it no longer reflects how modern computers are built (multiple levels of CPU caches, cores, sockets, NUMA, etc).
  • tombert 2y ago
    That's good to hear! I am pretty removed from underlying hardware now, so it makes me happy to hear that better way of doing things is catching on even in low-level land.
  • gpderetta 2y ago
    If you are doing pure shared nothing message passing, you do not need coherent caches; in fact cache coherency gets in the way of pure message passing. Viceversa if you do pure message passing you are not benefitting from hardware accelerated cache coherency and leaving performance (and usability) on the floor.
• intelVISA 2y ago
  Shared-nothing is typically The Right Choice in my experience as well. Maybe the odd atomic...