3 ms·
On Windows, llama.cpp has OpenCL support (CLBlast) and MLC LLM (https://mlc.ai/mlc-llm/docs/ https://mlc.ai/mlc-llm/docs/) has Vulkan acceleration. On Linux, E
by lhl 3y ago
On Windows, llama.cpp has OpenCL support (CLBlast) and MLC LLM (https://mlc.ai/mlc-llm/docs/ https://mlc.ai/mlc-llm/docs/) has Vulkan acceleration.
On Linux, ExLlama and MLC LLM have native ROCm support, and there is a HIPified fork of llama.cpp as well.