5 ms·

> Your odds of being able to apply parallel-computing techniques to a problem are inversely proportional to the degree of irreducible semantic nonlocality in yo

> Your odds of being able to apply parallel-computing techniques to a problem are inversely proportional to the degree of irreducible semantic nonlocality in your input data. That's not right. GPUs do pretty well at tasks that have poor locality compared to CPUs, as long as there's a lot of parallelism to be exploited -- see for example ray tracing. The reason for this is that CPUs depend on their large caches, which are useless when there is little locality, while GPUs simply switch to another warp while data is being loaded from memory, and they have much higher memory bandwidth anyway. Also, GPUs are distinctively not systolic arrays. Hennesy's "Computer Architecture: a quantitative approach" has a good primer on Single instruction Multiple Threads (SIMT) architectures typically seen in GPUs.

michelpp 8y ago
> Also, GPUs are distinctively not systolic arrays. Hennesy's "Computer Architecture: a quantitative approach" has a good primer on Single instruction Multiple Threads (SIMT) architectures typically seen in GPUs. Good catch, as pointed out by wikipedia: "They are sometimes classified as multiple-instruction single-data (MISD) architectures under Flynn's taxonomy, but this classification is questionable because a strong argument can be made to distinguish systolic arrays from any of Flynn's four categories: SISD, SIMD, MISD, MIMD" https://en.wikipedia.org/wiki/Systolic_array https://en.wikipedia.org/wiki/Systolic_array
wtracy 8y ago
SIMT architecture is why GPUs are actually not that great for ray tracing. In a nontrivial scene, some rays will bounce more times than others, and that means that the whole parallel execution unit is tied up until every single ray terminates. (The other "threads" essentially execute NOP instructions in the meantime.) Despite what Nvidia would have you believe, a lot of VFX rendering is done on CPUs, not GPUs, and this is one of the reasons why. (For example, last I heard, Pixar does all their render work on Intel Xeons.)

michelpp 8y ago

> Also, GPUs are distinctively not systolic arrays. Hennesy's "Computer Architecture: a quantitative approach" has a good primer on Single instruction Multiple Threads (SIMT) architectures typically seen in GPUs. Good catch, as pointed out by wikipedia: "They are sometimes classified as multiple-instruction single-data (MISD) architectures under Flynn's taxonomy, but this classification is questionable because a strong argument can be made to distinguish systolic arrays from any of Flynn's four categories: SISD, SIMD, MISD, MIMD" https://en.wikipedia.org/wiki/Systolic_array https://en.wikipedia.org/wiki/Systolic_array

wtracy 8y ago

SIMT architecture is why GPUs are actually not that great for ray tracing. In a nontrivial scene, some rays will bounce more times than others, and that means that the whole parallel execution unit is tied up until every single ray terminates. (The other "threads" essentially execute NOP instructions in the meantime.) Despite what Nvidia would have you believe, a lot of VFX rendering is done on CPUs, not GPUs, and this is one of the reasons why. (For example, last I heard, Pixar does all their render work on Intel Xeons.)