8 ms·

Why thread-based application parallelism is trumped in the multicore era

46 points by anupj 14y ago

acqq 14y ago
It seems to me that the article doesn't contain anything to support the claim from the title or its own conclusion. It just recycles some elementary topics from the freshman year of some computer course.
- wmf 14y ago
  Unfortunately that's par for the course for developerWorks.
16s 14y ago
I use a large Platform computational compute cluster often. We use individual processes (not threads) to crunch big data and in general that approach works well. Each process has its own memory space and its own little bit of the work load to complete. One process can crash or throw and exception and the others keep on going. Having no shared streams or shared data containers to worry with (mutexes, locks, etc) is just wonderful. We call it poor man's parallelism and some guys who have done a lot of threading make light of it. It's so simple (compared to threads) that it seems like a naive approach. But it performs so well that it's hard to argue with the results.
- th0ma5 14y ago
  I've dabbled in larger than usual parallelism and this has been my experience as well. On *nix systems processes are cheap, while on M$ threads are cheap, but regardless of those, having a shared-nothing lock free approach has me feeling limitless.
- eblume 14y ago
  We do the same doing some terabyte-scale data processing. We try and load all of the shared state in to a read-only structure ahead of time, then fork off the workers. This way, copy-on-write memory architectures get the advantage of only persisting the data once, yet we get multiple processes working from a very very large shared set of data, all without having to do any sort of IPC or locking. The only locks we used are around streams used for a very basic sort of IPC used to communicate to shared stream controllers like logfiles - these do perform very poorly compared to 'proper' parallelism but they make up for a very tiny fraction of the total workload. We've found that this will beat a 'better' parallelism even if we need to do a lot of extra or redundant work due to the lack of IPC or reduction due to the no-overhead approach, although this will of course be highly dependent on the problem at hand.
- KirinDave 14y ago
  Actually, it's only "simpler" in that you are familiar with it and perhaps your programming environment of choice lacks good support for multi-threaded programming. It's certainly not "simpler" from the perspective of resource efficiency (e.g., it's so easy to spoil Copy-on-Write's benefits) and usually it asks some other process to do the hard work of coordination. From the perspective of correctness, introducing multiple processes to your system greatly increases the number of failure states and vastly complicates error recovery. I think you're dismissing the state of the art on multiple fronts, as is the article. From single-variable STM (of which Haskell and Clojure both have excellent implementations) to battle-tested and well-understood concurrency primitives in the Java standard concurrency library, multi-threaded programming is more approachable and performant now than it has ever been. But the article is oddest in that it seems to hold up Erlang as a way forward, but Erlang is just a different model built on top of thread-based concurrency. If the argument is the old pthread-ish model of "1 thread per call" and very primitive synchronization tools are antiquated... then who is he arguing with? Erlang uses actors, Haskell uses sorcery (really, it's fancy; they turn normal-looking threaded code into erlang-ish sliced execution under the covers), Go uses fancy structures along with coroutines, Java uses Executors to implement higher-level work off patterns, and everyone is using Futures and Promises now.
  - slurgfest 14y ago
    It is gratuitous to take the word "simpler" as some kind of proof that the person you are responding to is an ignorant who uses objectively bad tools. Reality is less clear. "Simpler" clearly referred to the complexity and the mess created by synchronization in a shared-everything environment, which is where most languages are at with threading (Haskell, Clojure and Erlang are not most languages). This is a valid criticism. You may lose Copy-on-Write, but sharing everything by default, at a low level, does raise the need for complex synchronization, which has its own performance issues. And this is very good reason to explore and use other concurrency models. This is something that you and the article and the person you are responding to all seem to agree on. Except that you seem to take any interesting concurrency interface to be threading (no, goroutines are NOT threads) and you draw the strange moral that only the complex problems of threads can be addressed with nice tools and new ideas, but somehow not the problems of other basic concurrency models. Assuming we have available helpful interfaces for using processes, threads and greenlets - all of which have been produced somewhere - the argument should be about the performance of the foolproofed backends. Preferably based on numbers.
zwieback 14y ago
I think there's a big difference between designing an application using threads and using an existing thread-based API or system, like the article describes. If you write your own application-specific threads you can pick and choose from any concurrent design pattern you want. Using pre-existing multithreaded systems typically forces the programmer to use specific policies to interact with the system.
- sparkie 14y ago
  This is true, but the term "threads" has been abused to refer exclusively to preemptive multitasking, omitting other solutions like coroutines. This is why we keep inventing new silly terms like "fibers" to refer to alternative solutions.
  - zwieback 14y ago
    Yes, when I hear thread I have a specific thing in mind and that thing is not like a coroutine. The difference between process and thread is probably pretty clear to everyone but other concurrent programming models are not very clearly differentiated. Also, because the call stack is such a fundamentally accepted concept, which the article does a good job explaining and illustrating, it's hard for most of us to jump out of our traditional concepts of execution flow.
chj 14y ago
Another useless paper. The deep call stack problem exists in any kind of parallelism. And use multiple processes can be safer but the complexity is actually the same.
- usefulcomment 14y ago
  > The deep call stack problem exists in any kind of parallelism. Consider an event-based server that returns to an event loop whenever it makes a blocking call. At any point in time, most of the active requests will have a call stack depth of zero, and the rest will have just the frames since they were last unblocked. Contrast that with a threaded server where a request occupies a single, deeper stack for its lifetime. Or consider an actor-based system where an actor has just a few responsibilities and communicates with other actors with messages. Contrast with a threaded system where the server's submodules have to communicate with function calls. I don't see how stack depth matters much, though. There are way more important considerations.
  - zwieback 14y ago
    > I don't see how stack depth matters much, though. You knew someone would say it so I will: it matters on RAM-constrained systems. I'm currently working on a CPU with 384K flash (a lot) and 48K RAM (not quite enough). Many systems in this class use preemptive multi-tasking OSes in a shared, non-virtual memory space. The question how to pre-allocate stack space is tricky. One non-traditional model is to use run-to-completion threads without stack switching so the problem reduces to estimating your overall worst-case stack usage. Another popular model is main-loop + interrupt handlers. In a lot of ways that's more like an actor model.
    - usefulcomment 14y ago
      Fair enough, I can see how it matters on low memory systems. The article says it's even worse than just stack memory, too, as the live stack frames screw up your dead object detection for up to 25% extra memory overhead. My thoughts were: 1) As we enter the massively multicore era, there will be much bigger fish to fry than a little memory overhead. Clock speeds have topped out while transistor counts are still growing exponentially. Exponential means soon application programmers will be tasked with keeping 100s, then 1000s of cores busy. It's gonna be a giant challenge for PL guys, systems guys, and app guys to make that happen. If an improved concurrency model can get us there, then 25% memory overhead one way or the other will be relatively insignificant. 2) The object liveness issue is an implementation detail of one particular VM. There's nothing but CPU cost preventing VM writers from being a little more clever and marking objects unreachable if they are only referenced by dead local vars in otherwise live stack frames. I don't know enough about it to know whether the CPU cost would be prohibitive, but the issue seems more nuanced than just "threads => deep stacks => +25% mem".
jamesaguilar 14y ago
I dunno, I constrain my multithreaded processes to a single cpu sometimes and it does seem like multiple threads and multiple CPUs get me performance benefits. Maybe IBM knows something I don't.
- sp332 14y ago
  Multithreading is better than nothing. This article points some reasons that it's not optimal and a few possible alternatives.
ww520 14y ago
Why does a thread-based application have to use shared objects? Any other parallel models can use shared objects and have the same problem.

46 points by anupj 14y ago

acqq 14y ago

It seems to me that the article doesn't contain anything to support the claim from the title or its own conclusion. It just recycles some elementary topics from the freshman year of some computer course.

wmf 14y ago
Unfortunately that's par for the course for developerWorks.

16s 14y ago

I use a large Platform computational compute cluster often. We use individual processes (not threads) to crunch big data and in general that approach works well. Each process has its own memory space and its own little bit of the work load to complete. One process can crash or throw and exception and the others keep on going. Having no shared streams or shared data containers to worry with (mutexes, locks, etc) is just wonderful. We call it poor man's parallelism and some guys who have done a lot of threading make light of it. It's so simple (compared to threads) that it seems like a naive approach. But it performs so well that it's hard to argue with the results.

th0ma5 14y ago
I've dabbled in larger than usual parallelism and this has been my experience as well. On *nix systems processes are cheap, while on M$ threads are cheap, but regardless of those, having a shared-nothing lock free approach has me feeling limitless.
eblume 14y ago
We do the same doing some terabyte-scale data processing. We try and load all of the shared state in to a read-only structure ahead of time, then fork off the workers. This way, copy-on-write memory architectures get the advantage of only persisting the data once, yet we get multiple processes working from a very very large shared set of data, all without having to do any sort of IPC or locking. The only locks we used are around streams used for a very basic sort of IPC used to communicate to shared stream controllers like logfiles - these do perform very poorly compared to 'proper' parallelism but they make up for a very tiny fraction of the total workload. We've found that this will beat a 'better' parallelism even if we need to do a lot of extra or redundant work due to the lack of IPC or reduction due to the no-overhead approach, although this will of course be highly dependent on the problem at hand.
KirinDave 14y ago
Actually, it's only "simpler" in that you are familiar with it and perhaps your programming environment of choice lacks good support for multi-threaded programming. It's certainly not "simpler" from the perspective of resource efficiency (e.g., it's so easy to spoil Copy-on-Write's benefits) and usually it asks some other process to do the hard work of coordination. From the perspective of correctness, introducing multiple processes to your system greatly increases the number of failure states and vastly complicates error recovery. I think you're dismissing the state of the art on multiple fronts, as is the article. From single-variable STM (of which Haskell and Clojure both have excellent implementations) to battle-tested and well-understood concurrency primitives in the Java standard concurrency library, multi-threaded programming is more approachable and performant now than it has ever been. But the article is oddest in that it seems to hold up Erlang as a way forward, but Erlang is just a different model built on top of thread-based concurrency. If the argument is the old pthread-ish model of "1 thread per call" and very primitive synchronization tools are antiquated... then who is he arguing with? Erlang uses actors, Haskell uses sorcery (really, it's fancy; they turn normal-looking threaded code into erlang-ish sliced execution under the covers), Go uses fancy structures along with coroutines, Java uses Executors to implement higher-level work off patterns, and everyone is using Futures and Promises now.
- slurgfest 14y ago
  It is gratuitous to take the word "simpler" as some kind of proof that the person you are responding to is an ignorant who uses objectively bad tools. Reality is less clear. "Simpler" clearly referred to the complexity and the mess created by synchronization in a shared-everything environment, which is where most languages are at with threading (Haskell, Clojure and Erlang are not most languages). This is a valid criticism. You may lose Copy-on-Write, but sharing everything by default, at a low level, does raise the need for complex synchronization, which has its own performance issues. And this is very good reason to explore and use other concurrency models. This is something that you and the article and the person you are responding to all seem to agree on. Except that you seem to take any interesting concurrency interface to be threading (no, goroutines are NOT threads) and you draw the strange moral that only the complex problems of threads can be addressed with nice tools and new ideas, but somehow not the problems of other basic concurrency models. Assuming we have available helpful interfaces for using processes, threads and greenlets - all of which have been produced somewhere - the argument should be about the performance of the foolproofed backends. Preferably based on numbers.

zwieback 14y ago

I think there's a big difference between designing an application using threads and using an existing thread-based API or system, like the article describes. If you write your own application-specific threads you can pick and choose from any concurrent design pattern you want. Using pre-existing multithreaded systems typically forces the programmer to use specific policies to interact with the system.

sparkie 14y ago
This is true, but the term "threads" has been abused to refer exclusively to preemptive multitasking, omitting other solutions like coroutines. This is why we keep inventing new silly terms like "fibers" to refer to alternative solutions.
- zwieback 14y ago
  Yes, when I hear thread I have a specific thing in mind and that thing is not like a coroutine. The difference between process and thread is probably pretty clear to everyone but other concurrent programming models are not very clearly differentiated. Also, because the call stack is such a fundamentally accepted concept, which the article does a good job explaining and illustrating, it's hard for most of us to jump out of our traditional concepts of execution flow.

chj 14y ago

Another useless paper. The deep call stack problem exists in any kind of parallelism. And use multiple processes can be safer but the complexity is actually the same.

usefulcomment 14y ago
> The deep call stack problem exists in any kind of parallelism. Consider an event-based server that returns to an event loop whenever it makes a blocking call. At any point in time, most of the active requests will have a call stack depth of zero, and the rest will have just the frames since they were last unblocked. Contrast that with a threaded server where a request occupies a single, deeper stack for its lifetime. Or consider an actor-based system where an actor has just a few responsibilities and communicates with other actors with messages. Contrast with a threaded system where the server's submodules have to communicate with function calls. I don't see how stack depth matters much, though. There are way more important considerations.
- zwieback 14y ago
  > I don't see how stack depth matters much, though. You knew someone would say it so I will: it matters on RAM-constrained systems. I'm currently working on a CPU with 384K flash (a lot) and 48K RAM (not quite enough). Many systems in this class use preemptive multi-tasking OSes in a shared, non-virtual memory space. The question how to pre-allocate stack space is tricky. One non-traditional model is to use run-to-completion threads without stack switching so the problem reduces to estimating your overall worst-case stack usage. Another popular model is main-loop + interrupt handlers. In a lot of ways that's more like an actor model.
  - usefulcomment 14y ago
    Fair enough, I can see how it matters on low memory systems. The article says it's even worse than just stack memory, too, as the live stack frames screw up your dead object detection for up to 25% extra memory overhead. My thoughts were: 1) As we enter the massively multicore era, there will be much bigger fish to fry than a little memory overhead. Clock speeds have topped out while transistor counts are still growing exponentially. Exponential means soon application programmers will be tasked with keeping 100s, then 1000s of cores busy. It's gonna be a giant challenge for PL guys, systems guys, and app guys to make that happen. If an improved concurrency model can get us there, then 25% memory overhead one way or the other will be relatively insignificant. 2) The object liveness issue is an implementation detail of one particular VM. There's nothing but CPU cost preventing VM writers from being a little more clever and marking objects unreachable if they are only referenced by dead local vars in otherwise live stack frames. I don't know enough about it to know whether the CPU cost would be prohibitive, but the issue seems more nuanced than just "threads => deep stacks => +25% mem".

jamesaguilar 14y ago

I dunno, I constrain my multithreaded processes to a single cpu sometimes and it does seem like multiple threads and multiple CPUs get me performance benefits. Maybe IBM knows something I don't.

sp332 14y ago
Multithreading is better than nothing. This article points some reasons that it's not optimal and a few possible alternatives.

ww520 14y ago

Why does a thread-based application have to use shared objects? Any other parallel models can use shared objects and have the same problem.