CUDA is pretty neat; I worked with an early beta (0.7?) and have tracked it on and off over time. It's not magic, though. The point is that GPGPU is really grea

CUDA is pretty neat; I worked with an early beta (0.7?) and have tracked it on and off over time. It's not magic, though. The point is that GPGPU is really great for some things and pretty ordinary for others. 'Embarrassingly parallel' algorithms do well, as do things that are heavy consumers of memory bandwidth. Like you say, anyone who needs frequent transfers to/from CPU may not do as well. Highly optimized C and CUDA are, to me, the same concept - to get the best results, you have to understand the underlying machine to get the best results. While your resources are different, it's a very similar skill - to codesign an algorithm with the deep knowledge of how either a modern CPU or a GPGPU actually works. I'm no longer current as a GPGPU programmer but it wouldn't surprise me that the same 10-20x that an expert can get with astute design for modern architecture is available on GPGPU as well (i.e. the difference between a naive and a sophisticated design on the same platform). Personally I regard that kind of algorithmic codesign as one of the most fun things you can do with your clothes on, but that's probably just me.