2 ms·
both llama.cpp and vllm support inference with rocm or vulkan. inference is the easiest thing to decouple from nvidia.
by buyucu 2y ago
both llama.cpp and vllm support inference with rocm or vulkan.
inference is the easiest thing to decouple from nvidia.