Because every GPU on the market has a very different internal organization and instruction set. They are able to execute any shader using standardized programm

Because every GPU on the market has a very different internal organization and instruction set. They are able to execute any shader using standardized programming constructs (Vulkan, DirectX, Metal) that both the OS and driver understand. While the API and OS manage device/app context, the driver manages the device itself and the job of compiling shaders or standard calls to their GPUs specific instructions or layout.