4 ms·
> mlc is the fastest Also noticed that mlc+Vulkan has near zero CPU use while Llama+opencl on gpu still end up with a fair bit of cpu use. So seems to be taki
by Havoc 3y ago
> mlc is the fastest
Also noticed that mlc+Vulkan has near zero CPU use while Llama+opencl on gpu still end up with a fair bit of cpu use. So seems to be taking a fundamentally different route
This was on a rock chip 3588 so not exactly conventional AI hardware