Both in terms of hardware and software, they need to be able to support the small set of operations that their training or inference uses, so the problem on bot

Both in terms of hardware and software, they need to be able to support the small set of operations that their training or inference uses, so the problem on both sides is much smaller than both a full GPU and a full CUDA replacement.