6 ms·
Nuts and Bolts of Multithreaded Programming (2011)
- fspacef 9y agoThis is required reading as part of ME344: Introduction to High Performance Computing at Stanford. I think it gives a great high level overview of how parallel programming works. Great starting point for those interested in learning more about HPC systems!
- fizixer 9y agoTitle mentions multithreaded programming, while the first line of the text (as well as major emphasis through the article) is on parallel programming. And surprisingly, there is no mention of GPU. Almost as if they want you to think parallelism can only be achieved through CPUs (cores, and threads) but don't want to admit it in the title.
- nine_k 9y agoCan a SIMD system be really considered "multi-threaded"?
- deleted 9y ago[deleted]
- fizixer 9y agoI wouldn't think so. Not sure why you asked though.
- nine_k 9y agoThere's a question upthread asking why GPUs are not mentioned in "Nuts and bolts of multi-threaded programming". Hence my wonderment.
- mockery 9y agoBut GPUs aren't SIMD. (In some sense they're almost the opposite.)
- vilya 9y agoIn an incorrect sense, yes. :-) SIMD = Single Instruction Multiple Data, meaning the same instruction being applied to multiple different values simultaneously. That's exactly what GPUs do.
- telperion 9y agoThat's exactly what GPUs do in a single thread, but GPUs are also about threading, usually sporting tens, if not hundreds of small cores to effectively allow them to compute that many pixels (or rather shader outputs) in parallel. Otherwise, it wouldn't scale much. That being said, I understand you only wanted to point out the error in the upper post.
- telperion 9y agoactually they are.
- fizixer 9y agoThen you didn't read the comment upthread fully. It did acknowledge the title, and contrasted it with the tone of the article (the first half at least) which is way broader minus a mention of GPUs (the basics of parallel algorithms, parallel APIs).
- chrisseaton 9y agoSIMD is the classic example of how you don't need concurrency to have parallelism.
- deleted 9y ago[deleted]
- deepnotderp 9y agoTechnically, most modern GPUs are SIMT meaning that they aggregate lockstep threads into vector instructions, so yes.
- deleted 9y ago[deleted]
- fspacef 9y agoOf course, every article has bias but I still think it does the title justice as it talks about exposing concurrency via processes/threads. Also note that it was written in 2011, not sure if GPU based HPC was as common then but I could be wrong.
- zurn 9y agoI think the article structure is quite standard. It first introduces the more general concepts (parallel programming, zooming in on MIMD on shared-memory machines) and then focuses on a subset (multithreaded) while also alluding to other parallel programming techniques (MPI etc).
- hamandcheese 9y agoAlthough it was updated in 2011, the HN title is a bit misleading. > As Intel puts two cores on a single piece of silicon and as multi-socket systems continue to grow, shared memory systems are going to become the norm. Clearly this was originally published quite a bit earlier.
- willtim 9y agoGiven that the sustainability of Intel's profit margins are dependent on it, I am surprised they do not put more effort into making parallel programming easier. For example, GHC Haskell has a very promising parallelism/concurrency story with its many available programming models, including software transactional memory and lightweight threads, yet it is Microsoft funding it and not Intel.
- Athas 9y agoIntel Labs did do some work on functional language implementation, including a Haskell compiler (backend): https://github.com/IntelLabs/flrc https://github.com/IntelLabs/flrc I think the focus was on vectorisation rather than task parallelism, though. And it's been abandoned for years. I'm not sure Intel's sustainability is uniquely dependent on parallel programming. In fact, given that Intel processors have the best sequential performance, one could even argue that it's to their advantage that parallel programming remains awkward.
- vilya 9y agoMaybe the effort just isn't where you're looking? Intel provides the Threading Building Blocks library for C++; some top notch debugging & profiling tools with multi-threading support; very good documentation; and they do quite a bit of community outreach to educate about and promote multi-threaded programming. (I'm not affiliated with them, just grateful for all of that stuff!)
- willtim 9y agoI'm aware of TBB, but IMHO C++ is not the future of parallel and concurrent programming, or at least not the future I am hoping for.
- CyberDildonics 9y agoWhat is one example of haskell being used effectively for parallel execution in a real world scenario?
- kulu2002 9y agoGood introductory article for cores, threads, shared memories and all. Would have been good if there were few lines about debugging/ troubleshooting multi-threaded systems
- rapidsslonline 9y agoit was published earlier in 2011. Putting now old updates will make confuse user High-Performance Computing.