10 ms·
Knowledgable people, please explain. Vulkan allows executing code on GPU. Can it be used for typical ML tasks (e.g. large matrix multiplication)? Why do we need
by codedokode 3y ago
Knowledgable people, please explain. Vulkan allows executing code on GPU. Can it be used for typical ML tasks (e.g. large matrix multiplication)? Why do we need drivers like CUDA or ROCm then?
- raphlinus 3y agoI think we'll get there but it will take years to arrive at a robust solution, because of the complexity of the ecosystem. The cooperative matrix extension, the portable and standard way to do what is usually marketed as "tensor cores," just landed in the spec and drivers a couple months ago. It will take people time to figure out how to use them effectively, partly because you have to query which options are supported at runtime. There's still a driver and you're still at mercy of it (bugs and all), it's just much more likely to be installed on your Windows or Linux machine than a ROCm stack.
- boustrophedon 3y agoTo answer your second question directly: CUDA predates Vulkan. So if your question then becomes: Why did we need Vulkan if we have CUDA? OpenGL, DirectX, Metal, and Vulkan all have lots of graphics-specific concepts built in to their APIs. While you could probably implement most of this in CUDA, at some level you need hardware support for your graphics APIs because in the end the data in your framebuffer goes from the GPU's memory to your monitor via HDMI or DisplayPort. CUDA has no interface for that graphics-specific stuff (technically there are CUDA-OpenGL interop APIs inside CUDA) because it has to be able to run on AI acceleration hardware in datacenters (although that hardware didn't exist when CUDA was invented either, to be fair)