3 ms·

I'm still in early stages of exploration, but vllm seems to be compatible with most models on huggingface.