4 ms·
Do inference frameworks like vllm support vision?
by gdiamos 2y ago
Do inference frameworks like vllm support vision?
- woodson 2y agoYes, vLLM does (though marked experimental): https://docs.vllm.ai/en/latest/models/vlm.html https://docs.vllm.ai/en/latest/models/vlm.html
- theaniketmaurya 2y agoYou can run with LitServe. here is the code - https://lightning.ai/lightning-ai/studios/deploy-llama-3-2-vision-with-litserve https://lightning.ai/lightning-ai/studios/deploy-llama-3-2-v...