3 ms·
Looks cool! Does the Vulkan backend work on PC? MLC-LLM proves it can work well and it would be cool to have a cross platform, minimal runtime, GPU-agnostic bac
by modeless 3y ago
Looks cool! Does the Vulkan backend work on PC? MLC-LLM proves it can work well and it would be cool to have a cross platform, minimal runtime, GPU-agnostic backend for PC too, not just mobile.
- ssjia 3y agoThe Vulkan backend does work on PC, as the only requirement is that Vulkan drivers are present. However, it was developed with mobile use-cases in mind and we haven't validated/optimized performance for PC. As an aside, the Vulkan backend is tied to TorchScript at the moment, so it is not yet compatible with ExecuTorch. However, we are also planning to introduce a Vulkan delegate for ExecuTorch which will enable GPU delegation through ExecuTorch.