4 ms·

Not to offend, but yeah, if you've already partitioned the work to be done, then of course it's not hard. I'm pretty sure that anyone who's come into even vagu

Not to offend, but yeah, if you've already partitioned the work to be done, then of course it's not hard. I'm pretty sure that anyone who's come into even vague contact with parallel programming understands that the hardest part is communication and synchronization. Your post feels incredibly pedantic.

dragontamer 6y ago
I think you underestimate the number of real-world problems that can be solved by a bash script executing "./some_task &" in a for loop, and using "jobs", "fg", and "bg" as needed. For many problems, that's literally good enough parallelism. -------- Case in point: I once needed to render an animation in Blender, and I noticed that Blender's built-in parallelism was kind of crap (it was an older version of Blender at the time. My 32-thread / 16-core system barely reached 400% utilization... I know I could do better). So I just spawned 16x Blenders in headless mode, each handling a different frame of the animation, and achieved far higher utilization than the innate Blender parallelism. Yes, literally: #!/bin/bash for i in `seq 1 16`; do blender (params) & # Yeah, just run a bunch of blenders in headless + background mode done Bonus points: I tweaked the script to take advantage of NUMA-locality, by automatically pinning blender to one NUMA-node. All without ever touching a single line of C code. ------- Communication / synchronization is hard. But parallelism (even high-performance NUMA-locality with thread affinity pinned to particular CPU nodes) doesn't have to be hard. Just think a bit smarter. -------- The equivalent to doing "./some_job &" is fork() and wait() in POSIX C / C++. That's good enough for a ton of problems (hell, its how us older programmers achieved parallelism back in the CGI-BIN days). Don't do complicated high-performance techniques if you only need a bit of parallelism. Don't overthink things, stick to simple parallelism when simple solutions solve simple problem. Don't premature optimize into complicated producer/consumer patterns unless you actually need the performance. Definitely don't premature optimize into high-performance lockless programming techniques unless you really need to.
- npsimons 6y ago
  I mean in the example you gave, the work was already partitioned for you. That we have high level tools that make it easy doesn't change the fact that the work of rendering frames in an animation are very clear chunks of work that lend themselves naturally to parallelization. We can quibble about the semantics of "parallel" programming all day, but I think that either too broad or too narrow a scope for the term misses the point: that data coherency is what makes parallel programming hard. In the ultimate reductio ad absurdum, you can just say it falls into one of the two classes of hard computer science problems, that being "cache invalidation", where the "cache" is some form of shared data. In this sense, I agree with you: it's not the parallelism per se that's hard, but I think anyone talking about the difficulty of it quite clearly is referring to the shared data problems.
  - dragontamer 6y ago
    > I mean in the example you gave, the work was already partitioned for you. In my experience, I keep running into problems *already partitioned*. Make -j. 3d-animation frames. SAXY arrays. (Seriously, do you know how many for loops I've come across that can become #pragma omp parallel for and just automatically benefit from parallelism??). Multiple connections to a server (ex: Fork/join Apache model / CGI-bin / PHP / etc. etc.). Multiple connections to other servers (SMTP, IRC, etc. etc.) A ton of problems are "easy parallelism" in my experience. Probably most problems. Yeah, there's some hard problems out there, but beginners should learn how to parallelize the easy stuff first.
    - dagss 6y ago
      The point is that these easy cases may just referred to as "work to do". Using more than one CPU core at the time to process through a backlog of work is just...the normal case. Doing work simultanously is not what people refer to when they say "parallel programming is hard"; and the pedantery is in insisting that people word themselves to exclude trivially parallel work when what they want to talk about are the cases where parallelism is not trivial. The conversation above is like this: "Solving cubic equations is hard" "Actually, solving x ^ 3 = 27 is easy, just take the cubic root.." "Well that is not what we meant when we said cubic equations, we were interested in the general case..." "I see equations like x^3 = y all the time, it is not that often you have a linear term too..."
phkahler 6y ago
I think a lot of people don't know how easy it can and should be to handle the easy cases. C++ has some parallel constructs, but they are not as simple as OpenMP because they're meant to handle the harder stuff. I think it's important to show and use the easy ways before getting into the complex things.

dragontamer 6y ago

I think you underestimate the number of real-world problems that can be solved by a bash script executing "./some_task &" in a for loop, and using "jobs", "fg", and "bg" as needed. For many problems, that's literally good enough parallelism. -------- Case in point: I once needed to render an animation in Blender, and I noticed that Blender's built-in parallelism was kind of crap (it was an older version of Blender at the time. My 32-thread / 16-core system barely reached 400% utilization... I know I could do better). So I just spawned 16x Blenders in headless mode, each handling a different frame of the animation, and achieved far higher utilization than the innate Blender parallelism. Yes, literally: #!/bin/bash for i in `seq 1 16`; do blender (params) & # Yeah, just run a bunch of blenders in headless + background mode done Bonus points: I tweaked the script to take advantage of NUMA-locality, by automatically pinning blender to one NUMA-node. All without ever touching a single line of C code. ------- Communication / synchronization is hard. But parallelism (even high-performance NUMA-locality with thread affinity pinned to particular CPU nodes) doesn't have to be hard. Just think a bit smarter. -------- The equivalent to doing "./some_job &" is fork() and wait() in POSIX C / C++. That's good enough for a ton of problems (hell, its how us older programmers achieved parallelism back in the CGI-BIN days). Don't do complicated high-performance techniques if you only need a bit of parallelism. Don't overthink things, stick to simple parallelism when simple solutions solve simple problem. Don't premature optimize into complicated producer/consumer patterns unless you actually need the performance. Definitely don't premature optimize into high-performance lockless programming techniques unless you really need to.

npsimons 6y ago
I mean in the example you gave, the work was already partitioned for you. That we have high level tools that make it easy doesn't change the fact that the work of rendering frames in an animation are very clear chunks of work that lend themselves naturally to parallelization. We can quibble about the semantics of "parallel" programming all day, but I think that either too broad or too narrow a scope for the term misses the point: that data coherency is what makes parallel programming hard. In the ultimate reductio ad absurdum, you can just say it falls into one of the two classes of hard computer science problems, that being "cache invalidation", where the "cache" is some form of shared data. In this sense, I agree with you: it's not the parallelism per se that's hard, but I think anyone talking about the difficulty of it quite clearly is referring to the shared data problems.
- dragontamer 6y ago
  > I mean in the example you gave, the work was already partitioned for you. In my experience, I keep running into problems *already partitioned*. Make -j. 3d-animation frames. SAXY arrays. (Seriously, do you know how many for loops I've come across that can become #pragma omp parallel for and just automatically benefit from parallelism??). Multiple connections to a server (ex: Fork/join Apache model / CGI-bin / PHP / etc. etc.). Multiple connections to other servers (SMTP, IRC, etc. etc.) A ton of problems are "easy parallelism" in my experience. Probably most problems. Yeah, there's some hard problems out there, but beginners should learn how to parallelize the easy stuff first.
  - dagss 6y ago
    The point is that these easy cases may just referred to as "work to do". Using more than one CPU core at the time to process through a backlog of work is just...the normal case. Doing work simultanously is not what people refer to when they say "parallel programming is hard"; and the pedantery is in insisting that people word themselves to exclude trivially parallel work when what they want to talk about are the cases where parallelism is not trivial. The conversation above is like this: "Solving cubic equations is hard" "Actually, solving x ^ 3 = 27 is easy, just take the cubic root.." "Well that is not what we meant when we said cubic equations, we were interested in the general case..." "I see equations like x^3 = y all the time, it is not that often you have a linear term too..."

phkahler 6y ago

I think a lot of people don't know how easy it can and should be to handle the easy cases. C++ has some parallel constructs, but they are not as simple as OpenMP because they're meant to handle the harder stuff. I think it's important to show and use the easy ways before getting into the complex things.