4 ms·

On being stateful (2008)

38 points by kornish 10y ago

richard_todd 10y ago
I'm not optimistic about the kinds of approaches mentioned fostering "ubiquitous parallelism." Just as a better compiler, language, or implementation technique won't make a bubble-sort into a quicksort, I don't expect them to turn essentially sequential programs into parallel ones. We need new kinds of algorithms, not better programming techniques. Being explicit and telling the compiler all about isolated pockets of dependencies does not ultimately defeat their constraints on the overall program. I remember a statement by Knuth in an interview[1]: "Let me put it this way: During the past 50 years, I’ve written well over a thousand programs, many of which have substantial size. I can’t think of even five of those programs that would have been enhanced noticeably by parallelism or multithreading." [1]: http://www.informit.com/articles/article.aspx?p=1193856 http://www.informit.com/articles/article.aspx?p=1193856
- jerf 10y ago
  A lot has happened in that space since that 2008 blog post. Unfortunately for us all, what has happened is that automatic parallelism has basically been disproved. It effectively isn't possible. At best, it requires a language so profoundly different than anything we have to prod developers into using it in a way that can be automatically parallelized that it's arguably not really automatic anymore, but a new restricted DSL for the task. Similarly for STM; it's only practical in a language with enough mutation control built into it that nobody will want to use it, i.e., Haskell. It can't be added to conventional languages. IIRC it was already looking pretty grim by 2008, but now it's certain; you can't bodge STM onto a conventional imperative language.
sqeaky 10y ago
I think this guy was too kind to STM. Otherwise I largely agree. Despite being a C++ programmer I like using high level abstraction. I like being able to write a few lines and knowing what the result will be and having the computer just figure it out the details. Despite that I really want those abstractions to be as close to zero cost as possible. I also want to precisely know how to work with the contract the spec demands and the implementation reality requires, because performance is all too often a real concern for me. STM has never had a role in any of that because it tries to be a panacea. It tries to fix everything for everybody. There is no way it can be an optimal (or even reasonable) solution for most problems). I think me and the author would agree that naked mutexes are the gotos of threaded algorithms. Just as we replaced gotos with several kinds of loops, several kinds of functions, classes, lambdas and countless other more expressive tools, we will replace naked mutexes with a host of other tools. Have something you just want to run on some other hardware and get the answer from, try a future/promise. Have N units or identical work, try a parallel for or a one of the many GPU executors. Have a complex dependency chain of heterogeneous work, build a DAG of actors or other automata. Need something else, perhaps your threading primitive hasn't been invented yet.
- jhartmann 10y ago
  I think you should also mention work queues and producer / consumer thread pools. These are very powerful concepts you don't mention here, and they are often the first tool I use. You very often don't need anything else.

38 points by kornish 10y ago

I'm not optimistic about the kinds of approaches mentioned fostering "ubiquitous parallelism." Just as a better compiler, language, or implementation technique won't make a bubble-sort into a quicksort, I don't expect them to turn essentially sequential programs into parallel ones. We need new kinds of algorithms, not better programming techniques. Being explicit and telling the compiler all about isolated pockets of dependencies does not ultimately defeat their constraints on the overall program. I remember a statement by Knuth in an interview[1]: "Let me put it this way: During the past 50 years, I’ve written well over a thousand programs, many of which have substantial size. I can’t think of even five of those programs that would have been enhanced noticeably by parallelism or multithreading." [1]: http://www.informit.com/articles/article.aspx?p=1193856 http://www.informit.com/articles/article.aspx?p=1193856

jerf 10y ago
A lot has happened in that space since that 2008 blog post. Unfortunately for us all, what has happened is that automatic parallelism has basically been disproved. It effectively isn't possible. At best, it requires a language so profoundly different than anything we have to prod developers into using it in a way that can be automatically parallelized that it's arguably not really automatic anymore, but a new restricted DSL for the task. Similarly for STM; it's only practical in a language with enough mutation control built into it that nobody will want to use it, i.e., Haskell. It can't be added to conventional languages. IIRC it was already looking pretty grim by 2008, but now it's certain; you can't bodge STM onto a conventional imperative language.

sqeaky 10y ago

I think this guy was too kind to STM. Otherwise I largely agree. Despite being a C++ programmer I like using high level abstraction. I like being able to write a few lines and knowing what the result will be and having the computer just figure it out the details. Despite that I really want those abstractions to be as close to zero cost as possible. I also want to precisely know how to work with the contract the spec demands and the implementation reality requires, because performance is all too often a real concern for me. STM has never had a role in any of that because it tries to be a panacea. It tries to fix everything for everybody. There is no way it can be an optimal (or even reasonable) solution for most problems). I think me and the author would agree that naked mutexes are the gotos of threaded algorithms. Just as we replaced gotos with several kinds of loops, several kinds of functions, classes, lambdas and countless other more expressive tools, we will replace naked mutexes with a host of other tools. Have something you just want to run on some other hardware and get the answer from, try a future/promise. Have N units or identical work, try a parallel for or a one of the many GPU executors. Have a complex dependency chain of heterogeneous work, build a DAG of actors or other automata. Need something else, perhaps your threading primitive hasn't been invented yet.

jhartmann 10y ago
I think you should also mention work queues and producer / consumer thread pools. These are very powerful concepts you don't mention here, and they are often the first tool I use. You very often don't need anything else.