2 ms·
Would it be possible to run something like vLLM or TensortRT-llm with tinfoil?
by tealcod 1y ago
Would it be possible to run something like vLLM or TensortRT-llm with tinfoil?
- FrasiertheLion 1y agoWe’re already using vllm as our inference server for our standard models. We can run whatever inference server for custom deployments.