4 ms·
Sounds great! How far is it actually compatible right now? Are there any tests / benchmarks? Can this be used to run CUDA-accelerated LLMs?
by DrNosferatu 3y ago
Sounds great!
How far is it actually compatible right now?
Are there any tests / benchmarks?
Can this be used to run CUDA-accelerated LLMs?
- johndough 3y agoThe Vulkan spec does not enforce strict IEEE 754 floating point semantics, so perfect compatibility with CUDA is impossible. https://registry.khronos.org/vulkan/specs/1.3-khr-extensions/html/chap43.html#spirvenv-op-prec https://registry.khronos.org/vulkan/specs/1.3-khr-extensions... However, the deep learning field does currently not pay much attention to reproducibility, so this might not be a big issue.
- my123 3y agoMemory management in Vulkan is _very_ restricted. Nothing remotely like UVM. CUDA on Vulkan for these reasons will always stay a pet project at best, with no shot at usable quality whatsoever.
- syntaxers 3y ago> > VUDA only takes kernels in SPIR-V format. VUDA does not provide any support for compiling CUDA C kernels directly to SPIR-V (yet). However, it does not know or care how the SPIR-V source was created - may it be GLSL, HLSL, OpenCL. So the answer is no, it can't be used with kernels that use cublas or cudnn, which excludes almost all ML use-cases.