3 ms·
Look into vllm and text-generation-inference Why not use an API though? Some to check out are: Replicate, Together AI, Fireworks AI, Deepinfra, Anyscale Endp
by tikkun 3y ago
Look into vllm and text-generation-inference
Why not use an API though?
Some to check out are:
Replicate, Together AI, Fireworks AI, Deepinfra, Anyscale Endpoints
- extasia 3y agoI can personally recommend deepinfra. Been using it the past few days and it's served me very well.