I'm guessing it will probably be more like large complicated libraries, just like now, that have 8 different implementations for different chips, heuristics for

I'm guessing it will probably be more like large complicated libraries, just like now, that have 8 different implementations for different chips, heuristics for what is worth doing on GPU, etc.