3 ms·

I hope the development of all these accelerators drives developers and frameworks to move away from CUDA only and even better to move to a open standard of some

I hope the development of all these accelerators drives developers and frameworks to move away from CUDA only and even better to move to a open standard of some sort (eg. ROCM, if it can atain parity with CUDA in terms of usability). It would be such a waste for each framework to need to have a seperate backend for each hardware vendor. Then we will end up in the same desperate situation as game engines with Vulkan, Metal and DX12 with a side of OpenGL and Intel OneAPI (for things like eg. Blender). That sadly seems to be the way it's going right now.

joshhart 3y ago
Trainium and Inferentia sadly aren't going this way, they have their own approach https://github.com/aws-neuron/transformers-neuronx https://github.com/aws-neuron/transformers-neuronx . I think the best case scenario is some middleware like Triton https://openai.com/research/triton https://openai.com/research/triton becomes the standard and we can build adapters to all of these backends. It is disappointing though.