I've worked with CUDA a pretty good amount, so I feel obliged to point out that most of initial effort put into porting an algorithm to CUDA is not spent learni

I've worked with CUDA a pretty good amount, so I feel obliged to point out that most of initial effort put into porting an algorithm to CUDA is not spent learning the architecture, but parallelizing the algorithm (which is something you would have to deal with even if just porting your code to multi-core CPU). Once you have figured out how to parallelize the algorithm, then you need to worry about the architecture, but only if you need to eek out every last bit of performance. In most viable cases, you should be able to get a pretty decent speedup without tearing your hair out. In any case, OpenCL is based on CUDA (the driver API), and they are practically the same (compare the reference manuals and you'll see what I mean). Also, GPU computing is much, much more efficient than CPU computing (in terms of FLOPS/Watt). A high-end GPU (say, a Tesla C1060) can probably pull between 200-300 watts (~2x what a high-end Xeon uses), but can do over 1TFLOP with sufficiently optimized code.