That's not much different than a modern CPU with an OS on top; where you have the OS doing some of the scheduling then the CPU is splitting up the instructions

That's not much different than a modern CPU with an OS on top; where you have the OS doing some of the scheduling then the CPU is splitting up the instructions into microinstructions and then scheduling them again in finer detail (hyperthreading and such). Seems to me there must be a C-level syntax and compiler so you're not manually splitting up individual adds and such and is still capable of optimizing the math effectively. But if that were true, we wouldn't have AAA game studios going to NVidia to optimize their game engines for each individual game.