OpenCL was pretty good in this a few years ago. I had some kernels which ran across AMD and Nvidia GPUs as well as many core Xeon systems without changing anyth

OpenCL was pretty good in this a few years ago. I had some kernels which ran across AMD and Nvidia GPUs as well as many core Xeon systems without changing anything. To squeeze the highest performance out, you still had to tune for the device, but it was a great start. Now, all vendors seem to have dropped support for OpenCL, though it will live on nicely in PortableCL, which implements it atop LLVM, allowing it to target anything LLVM does, which finally goes further to achieve the open-ness the original standard aspired to, than the vendors themselves at the time.