5 ms·

One thing I find myself asking a lot is if it's really hard to make a program take advantage of the cores or if the prevalent tooling and paradigms of today mak

One thing I find myself asking a lot is if it's really hard to make a program take advantage of the cores or if the prevalent tooling and paradigms of today make it hard to take advantage of the cores. While most programs are not embarassingly parallel in nature, I suspect the latter plays a bigger role than most of us tend to admit.

CyberDildonics 9y ago
I think you are exactly right. I've done a lot with concurrency, and the fact is that there is no life vest so you better know how to swim. I think an architecture made specifically for programs to take advantage of concurrency would help an enormous amount. Right now we are two steps removed. There aren't even many good data structure implementations out there. There is moodycamel::concurrent_queue and different memory allocators like jemalloc and the new rpmalloc, but the vast majority of programmers likely don't even realize that malloc locks and kills concurrency. Maybe they grab something from boost that just surrounds a data structure with a mutex, which is lazy bullshit. I actually think the majority of software that runs slow can be made to be sped up nearly linearly using dozens of threads, but the few niche libraries that help are overly complicated to compile and use, bloated with enormous dependencies and end up being a minefield of usage. At the moment many people's idea of multi-threaded is mostly fork-join techniques like openMP and that will never scale to using all cores throughout the whole program, since is one very narrow technique.
- wrcwill 9y ago
  As someone with only a basic understanding of threads, what is a better idea than fork/joins for scaling? Are you talking about thread pools or something else?
  - marcosdumay 9y ago
    Yes for thread pools instead of forking (could be process pools too). But that's only scratching the surface. It's how you communicate with those threads and allocate their work that makes the difference.
  - CyberDildonics 9y ago
    I think the quick and dirty answer is yes, although I doubt I would structure things the same way as a normal thread pool. If you think about the heart of the problem, it is really synchronization, and that only needs to happen in certain places in programs. Anything you can break up into pieces that can be transformed independently can be leveraged for concurrency, which also implies that independent stages in a pipeline can be concurrent as well.
- pythonaut_16 9y ago
  The particular implementation I had in mind is Erlang. Due to the green thread implementation, immutable data, and message passing, Erlang is able to offer a simple, robust concurrency platform. Of course it's still possible to write very sequential code but generally someone writing idiomatic Erlang won't have to go out of their way to get concurrency.
  - CyberDildonics 9y ago
    It is also far from ideal when it comes to performance from what I've seen. Maybe its overall design is good, but I don't believe that adding so much overhead over native programs is necessary.
    - pythonaut_16 9y ago
      From my somewhat limited reading it seems that the only inherent performance hit in Erlang's implementation is that all its processes can be interrupted externally, I think the term is cooperative multitasking? Otherwise I think the rest of Erlang's performance is because it's optimized for reliability and distribution rather than raw performance.
- paulmd 9y ago
  I really think there's only two rays of hope for parallel programming. One is promises (async/await). This takes the burden off the programmer for explicitly managing concurrency, and potentially allows lazy evaluation in many cases (if non-side-effecting). I've never worked in Lisps but it seems like a viable model in many ways. The one I wanted to bring up was probabilistic programming. Imagine a tree where there's not just one place to put something, there's many, possibly distributed. As such you greatly reduce the chances of contention from multiple threads. They would also probably (almost necessarily) reduce the amount of rebalancing work. GPUs seem like the perfect model here in multiple ways, not only do you have the massive degree of threading that surfaces any problems in such things but you also have the degree of threading to deterministically search these kind of structures. GPUs also highly discourage things like CAS or mutexing that we need to move away from. (PS nice username)
  - CyberDildonics 9y ago
    I'm not sure how the tree you are describing helps anything. Reading can always be done concurrently, it isn't the problem - all threads can read from the same memory at the same time. The problem is writes, which means any alterations. GPUs are useful for the kinds of parallelism that has always been known to be easy - fork join with all threads writing to new memory. There is no mystery to how to do this. > CAS or mutexing that we need to move away from compare and swap is the backbone of lock free programming. Mutexes to me seem to rarely be ideal, but there is no getting away from compare and swap as far as I know.

CyberDildonics 9y ago

I think you are exactly right. I've done a lot with concurrency, and the fact is that there is no life vest so you better know how to swim. I think an architecture made specifically for programs to take advantage of concurrency would help an enormous amount. Right now we are two steps removed. There aren't even many good data structure implementations out there. There is moodycamel::concurrent_queue and different memory allocators like jemalloc and the new rpmalloc, but the vast majority of programmers likely don't even realize that malloc locks and kills concurrency. Maybe they grab something from boost that just surrounds a data structure with a mutex, which is lazy bullshit. I actually think the majority of software that runs slow can be made to be sped up nearly linearly using dozens of threads, but the few niche libraries that help are overly complicated to compile and use, bloated with enormous dependencies and end up being a minefield of usage. At the moment many people's idea of multi-threaded is mostly fork-join techniques like openMP and that will never scale to using all cores throughout the whole program, since is one very narrow technique.

wrcwill 9y ago
As someone with only a basic understanding of threads, what is a better idea than fork/joins for scaling? Are you talking about thread pools or something else?
- marcosdumay 9y ago
  Yes for thread pools instead of forking (could be process pools too). But that's only scratching the surface. It's how you communicate with those threads and allocate their work that makes the difference.
- CyberDildonics 9y ago
  I think the quick and dirty answer is yes, although I doubt I would structure things the same way as a normal thread pool. If you think about the heart of the problem, it is really synchronization, and that only needs to happen in certain places in programs. Anything you can break up into pieces that can be transformed independently can be leveraged for concurrency, which also implies that independent stages in a pipeline can be concurrent as well.
pythonaut_16 9y ago
The particular implementation I had in mind is Erlang. Due to the green thread implementation, immutable data, and message passing, Erlang is able to offer a simple, robust concurrency platform. Of course it's still possible to write very sequential code but generally someone writing idiomatic Erlang won't have to go out of their way to get concurrency.
- CyberDildonics 9y ago
  It is also far from ideal when it comes to performance from what I've seen. Maybe its overall design is good, but I don't believe that adding so much overhead over native programs is necessary.
  - pythonaut_16 9y ago
    From my somewhat limited reading it seems that the only inherent performance hit in Erlang's implementation is that all its processes can be interrupted externally, I think the term is cooperative multitasking? Otherwise I think the rest of Erlang's performance is because it's optimized for reliability and distribution rather than raw performance.
paulmd 9y ago
I really think there's only two rays of hope for parallel programming. One is promises (async/await). This takes the burden off the programmer for explicitly managing concurrency, and potentially allows lazy evaluation in many cases (if non-side-effecting). I've never worked in Lisps but it seems like a viable model in many ways. The one I wanted to bring up was probabilistic programming. Imagine a tree where there's not just one place to put something, there's many, possibly distributed. As such you greatly reduce the chances of contention from multiple threads. They would also probably (almost necessarily) reduce the amount of rebalancing work. GPUs seem like the perfect model here in multiple ways, not only do you have the massive degree of threading that surfaces any problems in such things but you also have the degree of threading to deterministically search these kind of structures. GPUs also highly discourage things like CAS or mutexing that we need to move away from. (PS nice username)
- CyberDildonics 9y ago
  I'm not sure how the tree you are describing helps anything. Reading can always be done concurrently, it isn't the problem - all threads can read from the same memory at the same time. The problem is writes, which means any alterations. GPUs are useful for the kinds of parallelism that has always been known to be easy - fork join with all threads writing to new memory. There is no mystery to how to do this. > CAS or mutexing that we need to move away from compare and swap is the backbone of lock free programming. Mutexes to me seem to rarely be ideal, but there is no getting away from compare and swap as far as I know.