Inference (not training) is bottlenecked by memory access speed, not compute. Having special hardware wouldn't make it faster unless you somehow found a faster

Inference (not training) is bottlenecked by memory access speed, not compute. Having special hardware wouldn't make it faster unless you somehow found a faster memory controller than the GPU has.