3 ms·
pip install vllm, boom you have an openapi-compatible webserver. No further action necessary.
by throwawaymaths 2y ago
pip install vllm, boom you have an openapi-compatible webserver. No further action necessary.
- JimDabell 2y agoNo, there is further action necessary. If you want any kind of decent performance, you need to run it with an appropriate GPU. This is not true of spaCy, which makes spaCy easier to deploy.