3 ms·
One of the easiest ways to create a parallel program is to split your data, or split your processing. When you split your data, you can just apply the same algo
by keeperofdakeys 15y ago
One of the easiest ways to create a parallel program is to split your data, or split your processing. When you split your data, you can just apply the same algorithm to both sets, and re-integrate the processed data. This necessitates that the processing can be done apart, however. Graphics cards already do this. They have many, many cores that are used for 3d graphics. You can easily split the work done on the screen into small chunks, process these individually, then put it back together. The other way is to have one core do one set of work, and another core do another set. The algorithms should process a similar amount of data though, otherwise you are wasting processing power.
The other problem is representation of data in the programming language. Objects, which are used in a lot of programming languages, can be hard to use when it comes to parallelism. An object can only be operated on by one algorithm at a time, and to achieve parallelism, you must use concurrency (locking). You can switch to a different paradigm away from OOP (object orientated programming), to something like FP (functional programming). Since everything is a function, with data just being passed around, you can abstract a program to multiple cores in a more natural way. FP comes with its own difficulties though. This isn't to say parallel programs can't be written in OOP programs successfully, it just requires a different mindset to normal programming.
- scott_s 15y agoThe high-performance computing community calls what you described "data parallelism": http://en.wikipedia.org/wiki/Data_parallelism http://en.wikipedia.org/wiki/Data_parallelism
- dkersten 15y agoAnd data parallelism is the type of thing that gets called "embarrassingly parallel" a lot, if the data is independent enough to be easy to partition into lots of parallel bits.
- scott_s 15y agoNot in my experience. I've seen "embarrassingly parallel" used only to describe when an entire application can be divided up and they never need to communicate again until the end. That is obviously data parallel, but "data parallelism" is also used for finer granularities such as SIMD operators, or parallelizing independent loop iterations. Such things are not considered embarrassingly parallel because the granularity can be fine enough that the synchronization costs can still dominate.
- dkersten 15y agoI don't disagree. That's sort of what I said: often data parallelism is what is called "embarrassingly parallel", but there are certainly plenty of cases of data parallelism which are not embarrassingly parallel. I would say that "embarrassingly parallel" problems are a subset of "data parallel" problems.