3 ms·
You probably can just use Vulkan. Why don't you think Vulkan is an alternative? Can it not handle BLAS and memory access? What is a graphics card supposed to be
by roenxi 2mo ago
You probably can just use Vulkan. Why don't you think Vulkan is an alternative? Can it not handle BLAS and memory access? What is a graphics card supposed to be doing that isn't efficient in Vulkan?
- binary132 2mo agoSometimes it’s just that a particular backend is poorly optimized or has a regression on a particular platform as compared to the “mainstream” backends. For example, whisper.cpp’s Vulkan backend performs 2-3x worse than CPU on my Snapdragon X2 laptop when using the ggerganov v3 turbo model. It’s probably a simple fix, but it does need to be fixed.