6 ms·

It seems the answer is “Not available right now, let’s work on an API”? I think that matches my perspective on this, in that it isn’t worth a “mortal” develope

It seems the answer is “Not available right now, let’s work on an API”? I think that matches my perspective on this, in that it isn’t worth a “mortal” developer’s time right now. Rarely have I hit a scenario where a lock is the primary source of inefficiency and crossing thread boundaries. The Go approach of “Do not communicate by sharing memory” holds well, because CPU and memory do not always compose as one would expect (As the author of the post notes). Passing messages simplifies a lot of things because you learn to now rely on a specific thread or order, or program defensively against that possibility.

jiggawatts 6y ago
My experience is that I've often seen locks being an unseen source of performance woes in enterprise systems. So when you say that "rarely have I hit a scenario", I'd say that there's an even chance that you have, it's just that you didn't know it. Here's the acid test: Imagine if you got given a two-socket AMD EPYC server with 128 cores to run your software on. No virtualisation, local NVMe SSD for storage, 400 Gbps Ethernet for networking. No hardware bottlenecks of any type! Could your software utilise all 256 hardware threads of this computer? If not, why not? Maybe you got lucky and you really haven't had issues with locks, but certainly other developers have. The most common issue I see is with implicit locks in things like logging frameworks. E.g.: Sending output to a text file can be a bottleneck for larger servers like in the example above. Similarly, console output in multi-threaded software also requires locks, and also often has contention issues. Even innocent-looking code that simply uses dynamic memory allocation ("new", "malloc", etc...) can be bottlenecked by cross-thread contention on the heap data structures and the associated locks. Some modern allocators have thread-local pools, but this doesn't always work. E.g.: often large allocations go to a shared pool. Code that over-allocates large buffers and rapidly frees them can hit this issue all too easily. Web application servers will often hit the wall on something like a shared cache or the session-state store. Unless 100% of the shared data uses efficient lock-free algorithms, then given enough threads eventually some mutex somewhere will be the limit. There was a study done recently that showed that no modern database engine can scale past 64 cores properly, let alone 128. Not Oracle, not SAP, not SQL Server. At the rate TSMC is advancing with chip technology, they'll hit 300 million transistors per square millimeter in just a couple years. I fully expect AMD to release 128-core CPUs once they're on that process. A quad-socket server with those will have 512 cores or 1,024 threads. Completely lock-free algorithms will be the only way to scale up to just one server at that scale! PS: I remember reading the content on http://www.1024cores.net/ http://www.1024cores.net/ a few years back and thinking to myself that this guy has the right idea, but he's thinking too far ahead. Now... not so much. Now I think the author of that site is a visionary that more people should have paid attention to.
- DenisM 6y ago
  I feel it would be more productive to let databases handle any shared state and only write my own code that uses local state. If the databases are falling short I am sure they will catch up in a hurry, and will do a better job than I will have time for.
  - codr7 6y ago
    Many languages/standard libraries offer some kind of concurrent map that might be worth considering depending on context, sync.Map in Golang for example.
- wahern 6y ago
  Lock-free is not the same as wait-free. > Web application servers will often hit the wall on something like a shared cache or the session-state store. Unless 100% of the shared data uses efficient lock-free algorithms, then given enough threads eventually some mutex somewhere will be the limit. Without strong LL/SC (which exists nowhere for useful block sizes), it's difficult if not impossible to implement complex data structures, particularly compound data structures (lockless doesn't compose well), that can't block somewhere. Many sophisticated lock-free and even nominally wait-free data structures implicitly rely on dynamic memory and complex garbage collection strategies, which just hides the ball. If you have shared resource contention, usually something is going to block somewhere. And the more you scale up, the more your optimization hacks to minimize real-world blocking breakdown, thus the long history of multithreaded software hitting limits at 2, 4, 8, 16, 32, 64, 128, etc CPUs. The way around these is to architect your application at a higher-level to avoid or mitigate resource contention. And how you do that is extremely context dependent, though "share memory by communicating" is a rule of thumb intended to steer you in a less failure-prone direction, keeping your options open. If you go in planning to just add lockless implementations of basic data structures everywhere, you're going to fail hard. Often because the hit to pipelining, etc, for using atomic operations in your critical path impose huge costs upfront.
  - alextheparrot 6y ago
    > lockless doesn't compose well Are you familiar with any resources on this?I've noticed this when interviewing candidates and asking follow-ups related to how to make an LRU cache thread-safe. Many, many candidates usually reach for converting their HashMap into a ConcurrentHashMap, which effectively buys them nothing due to the point you raised. Seems like a missing abstraction that makes it hard to compose these structures, and therefore construct complex structures. Another thing to the research list, irrespective. Or maybe it is naturally the case that the implementations of composition for lockless doesn't scale well? I still need to read GP's article from 1024cores which seems to potential get into this as well.
necubi 6y ago
It depends on what you're doing. For real-time applications you simply cannot use locks in a real-time thread, because there is no guarantee that you will unlock within your deadline. This is common in audio programming, where missing your deadline causes glitches in the audio stream. It's definitely niche, but when you need them, you really need them.
- arakageeta 6y ago
  That's not strictly true. There are dozens of real-time sleep-based locking protocols that support bounds on worst-case blocking time. Now, even if you have a suitable protocol available to you, your deadlines may be so tight that you can't afford the overhead of context switching. Then yes, you may need a lock-free or wait-free synchronization mechanism.
  - necubi 6y ago
    I'm not sure if we're disagreeing here, but in the context of this article "a lock" is a kernel lock. What that implies will obviously depend on your kernel, but in Linux it means your thread may wait for an unbounded amount of time.
masklinn 6y ago
> The Go approach of “Do not communicate by sharing memory” Calling it an approach is hardly true given Go programs are generally full of shared mutable memory, even more so when performance are of any concern because channels are rather slow (multiple locks are involved) but not only: any time you're sending a pointer through a channel (including sending pointer-like structures e.g. a map), you're sharing memory.

jiggawatts 6y ago

My experience is that I've often seen locks being an unseen source of performance woes in enterprise systems. So when you say that "rarely have I hit a scenario", I'd say that there's an even chance that you have, it's just that you didn't know it. Here's the acid test: Imagine if you got given a two-socket AMD EPYC server with 128 cores to run your software on. No virtualisation, local NVMe SSD for storage, 400 Gbps Ethernet for networking. No hardware bottlenecks of any type! Could your software utilise all 256 hardware threads of this computer? If not, why not? Maybe you got lucky and you really haven't had issues with locks, but certainly other developers have. The most common issue I see is with implicit locks in things like logging frameworks. E.g.: Sending output to a text file can be a bottleneck for larger servers like in the example above. Similarly, console output in multi-threaded software also requires locks, and also often has contention issues. Even innocent-looking code that simply uses dynamic memory allocation ("new", "malloc", etc...) can be bottlenecked by cross-thread contention on the heap data structures and the associated locks. Some modern allocators have thread-local pools, but this doesn't always work. E.g.: often large allocations go to a shared pool. Code that over-allocates large buffers and rapidly frees them can hit this issue all too easily. Web application servers will often hit the wall on something like a shared cache or the session-state store. Unless 100% of the shared data uses efficient lock-free algorithms, then given enough threads eventually some mutex somewhere will be the limit. There was a study done recently that showed that no modern database engine can scale past 64 cores properly, let alone 128. Not Oracle, not SAP, not SQL Server. At the rate TSMC is advancing with chip technology, they'll hit 300 million transistors per square millimeter in just a couple years. I fully expect AMD to release 128-core CPUs once they're on that process. A quad-socket server with those will have 512 cores or 1,024 threads. Completely lock-free algorithms will be the only way to scale up to just one server at that scale! PS: I remember reading the content on http://www.1024cores.net/ http://www.1024cores.net/ a few years back and thinking to myself that this guy has the right idea, but he's thinking too far ahead. Now... not so much. Now I think the author of that site is a visionary that more people should have paid attention to.

DenisM 6y ago
I feel it would be more productive to let databases handle any shared state and only write my own code that uses local state. If the databases are falling short I am sure they will catch up in a hurry, and will do a better job than I will have time for.
- codr7 6y ago
  Many languages/standard libraries offer some kind of concurrent map that might be worth considering depending on context, sync.Map in Golang for example.
wahern 6y ago
Lock-free is not the same as wait-free. > Web application servers will often hit the wall on something like a shared cache or the session-state store. Unless 100% of the shared data uses efficient lock-free algorithms, then given enough threads eventually some mutex somewhere will be the limit. Without strong LL/SC (which exists nowhere for useful block sizes), it's difficult if not impossible to implement complex data structures, particularly compound data structures (lockless doesn't compose well), that can't block somewhere. Many sophisticated lock-free and even nominally wait-free data structures implicitly rely on dynamic memory and complex garbage collection strategies, which just hides the ball. If you have shared resource contention, usually something is going to block somewhere. And the more you scale up, the more your optimization hacks to minimize real-world blocking breakdown, thus the long history of multithreaded software hitting limits at 2, 4, 8, 16, 32, 64, 128, etc CPUs. The way around these is to architect your application at a higher-level to avoid or mitigate resource contention. And how you do that is extremely context dependent, though "share memory by communicating" is a rule of thumb intended to steer you in a less failure-prone direction, keeping your options open. If you go in planning to just add lockless implementations of basic data structures everywhere, you're going to fail hard. Often because the hit to pipelining, etc, for using atomic operations in your critical path impose huge costs upfront.
- alextheparrot 6y ago
  > lockless doesn't compose well Are you familiar with any resources on this?I've noticed this when interviewing candidates and asking follow-ups related to how to make an LRU cache thread-safe. Many, many candidates usually reach for converting their HashMap into a ConcurrentHashMap, which effectively buys them nothing due to the point you raised. Seems like a missing abstraction that makes it hard to compose these structures, and therefore construct complex structures. Another thing to the research list, irrespective. Or maybe it is naturally the case that the implementations of composition for lockless doesn't scale well? I still need to read GP's article from 1024cores which seems to potential get into this as well.

necubi 6y ago

It depends on what you're doing. For real-time applications you simply cannot use locks in a real-time thread, because there is no guarantee that you will unlock within your deadline. This is common in audio programming, where missing your deadline causes glitches in the audio stream. It's definitely niche, but when you need them, you really need them.

arakageeta 6y ago
That's not strictly true. There are dozens of real-time sleep-based locking protocols that support bounds on worst-case blocking time. Now, even if you have a suitable protocol available to you, your deadlines may be so tight that you can't afford the overhead of context switching. Then yes, you may need a lock-free or wait-free synchronization mechanism.
- necubi 6y ago
  I'm not sure if we're disagreeing here, but in the context of this article "a lock" is a kernel lock. What that implies will obviously depend on your kernel, but in Linux it means your thread may wait for an unbounded amount of time.

masklinn 6y ago

> The Go approach of “Do not communicate by sharing memory” Calling it an approach is hardly true given Go programs are generally full of shared mutable memory, even more so when performance are of any concern because channels are rather slow (multiple locks are involved) but not only: any time you're sending a pointer through a channel (including sending pointer-like structures e.g. a map), you're sharing memory.