5 ms·

The thing with GPUs is that for many problems really dumb and simple algorithms (think bubble sort equivalent) are many times faster than very fancy CPU algorit

The thing with GPUs is that for many problems really dumb and simple algorithms (think bubble sort equivalent) are many times faster than very fancy CPU algorithms (think quick sort equivalent). Your typical non-neural-network GPU algorithm is rarely using more than 50% of it's power, yet still outperforms carefully written CPU algorithms.

pjmlp 1y ago
Except it is only worth doing, if when taking into account loading data into the GPU and getting the results back, is still faster than total execution on the CPU. It doesn't help that GPU beats the CPU in compute, if a plain SIMD approach outperforms the total execution time.
- woooooo 1y ago
  Especially if you're saving watts in the process. And not utilizing a capital-intensive asset.
- formerly_proven 1y ago
  The "GPU as accelerator" vs. "GPU-native software" split. The former usually results in or from poor, generic architectures.
sumtechguy 1y ago
That is application of the formula Pre-work time + pack up time + send time + unpack time + work time + pack up time + send time + unpack time + post-work time. All remote work has these properties. Even something 'simple' like a remote REST call. If 'remote work time' plus all that other stuff is less than your local calls then it is time wise worth sending it remote. If not local CPU would win. That in many cases right now the GPU is 'winning' that race.
- EraYaN 1y ago
  There are some neat tricks to remove almost all the pack and unpack time. Apache Arrow can help a ton there (uses the same data format on both CPU and GPU or other accelerator). And on some unified memory systems even the send time can be very low.
  - sumtechguy 1y ago
    oh absolutely reducing any of those spots can change the dynamics of the formula. 'zero copy memory' is another local item many drivers can use. It is just one of those things that also change as tech marches on. What used to be the gold standard on speed is suddenly rendered moot because one of those variables changes. Or suddenly something new will become possible. It is kinda cool but you need to keep an eye on it.

pjmlp 1y ago

Except it is only worth doing, if when taking into account loading data into the GPU and getting the results back, is still faster than total execution on the CPU. It doesn't help that GPU beats the CPU in compute, if a plain SIMD approach outperforms the total execution time.

woooooo 1y ago
Especially if you're saving watts in the process. And not utilizing a capital-intensive asset.
formerly_proven 1y ago
The "GPU as accelerator" vs. "GPU-native software" split. The former usually results in or from poor, generic architectures.

sumtechguy 1y ago

That is application of the formula Pre-work time + pack up time + send time + unpack time + work time + pack up time + send time + unpack time + post-work time. All remote work has these properties. Even something 'simple' like a remote REST call. If 'remote work time' plus all that other stuff is less than your local calls then it is time wise worth sending it remote. If not local CPU would win. That in many cases right now the GPU is 'winning' that race.

EraYaN 1y ago
There are some neat tricks to remove almost all the pack and unpack time. Apache Arrow can help a ton there (uses the same data format on both CPU and GPU or other accelerator). And on some unified memory systems even the send time can be very low.
- sumtechguy 1y ago
  oh absolutely reducing any of those spots can change the dynamics of the formula. 'zero copy memory' is another local item many drivers can use. It is just one of those things that also change as tech marches on. What used to be the gold standard on speed is suddenly rendered moot because one of those variables changes. Or suddenly something new will become possible. It is kinda cool but you need to keep an eye on it.