3 ms·
But can you run llms that easily with vllm? do you have to fiddle with formats to get it to run?
by jacooper 2y ago
But can you run llms that easily with vllm? do you have to fiddle with formats to get it to run?
- buyucu 2y agoI'm still in early stages of exploration, but vllm seems to be compatible with most models on huggingface.