CUDA, OpenCL, Vulkan, and Metal are all different graphics APIs provided by GPU vendors. GPU vendors can implement all four of them if they like, but in practic

CUDA, OpenCL, Vulkan, and Metal are all different graphics APIs provided by GPU vendors. GPU vendors can implement all four of them if they like, but in practice, CUDA is an NVidia exclusive, Metal is a Apple exclusive and vice versa, and OpenCL is kind of old and unused (and also used to be an Apple technology as well). There's also other APIs provided by non-GPU hardware vendors. PyTorch and TensorFlow are training and inference libraries. In theory, PyTorch at least (probably also TF) is properly abstracted and can work with all APIs. In practice CUDA gets the most testing and has all the operator fusion kernels you need for good performance, so everyone trains on CUDA. Inference is less resource-intensive than training, and there's a lot of vendors that provide inference-only hardware (e.g. "Edge TPUs") that's better supported, so on the inference end there's more variety in API usage. You can of course train and inference on CPU but that means terrible performance.