Hardware decoding has its own module, commonly called "VDEC" in most architectures. It is mostly functionally divorced from the rest of the GPU. GPUs themselve

Hardware decoding has its own module, commonly called "VDEC" in most architectures. It is mostly functionally divorced from the rest of the GPU. GPUs themselves are mostly SIMD vector processors for their "shader cores" with a bunch of custom fixed-function hardware for the more special blocks in the pipeline. This is a completely separate unit from the CPU. There was an attempt from Intel known as "Larabee" to try and build a GPU style pipeline on top of an expanded Intel CPU. The consumer product was canned but the expanded CPU went on to be known as Intel's Xeon Phi line.