I think there may be solutions to some of these things. Firstly, it sounds they are envisioning pipelined algorithms where stages are run on the best compute d

I think there may be solutions to some of these things. Firstly, it sounds they are envisioning pipelined algorithms where stages are run on the best compute device. The CPU pipeline would presumably handle all asynchronous Apis and only call the pipelined algorithms once the data was available. It would be much easier to parallelize the code that can be easily be parallelized if the gpu context is the same as the cpu context. The second part I think is up to compilers. If we can make better compilers for gpu code which integrate more nicely with cpu code then the super low level programming won't be as necessary. GPUs haven't been in the limelight nearly as long as CPUs, so it's understandable that most modern languages are made for CPU code. I'd personally like to see LLVM and PTX playing together nicely in Rust. Humans will still have to consider the differences in optimal memory access / branching logic patterns, but that's understandable. We already do that - with the CPU in mind - so it will just take a bit of extra learning to consider the GPU.