4 ms·
Is there any equivalent of TF Serving for PyTorch? We have been thrilled with how robust and easy it is to deploy our models to production on the TF stack, and
by sbrother 7y ago
Is there any equivalent of TF Serving for PyTorch? We have been thrilled with how robust and easy it is to deploy our models to production on the TF stack, and it worries me that the inertia in the deep learning community seems to be toward PyTorch.
- calebkaiser 7y agoHave you checked out Cortex? It's an open source platform for deploying PyTorch models easily. I wrote an article for the PyTorch blog about it: https://medium.com/pytorch/how-to-build-production-software-with-pytorch-9a8725382f2a https://medium.com/pytorch/how-to-build-production-software-... GitHub: https://github.com/cortexlabs/cortex https://github.com/cortexlabs/cortex Full disclosure/shameless plug: I work on Cortex
- sbrother 7y agoThanks! I was not aware of this and it looks fantastic. Is it in the roadmap to target GCP, or even just a generic Kubernetes cluster?
- calebkaiser 7y agoGCP is on the immediate short-term roadmap, and we're investigating on-premise, but don't have a firm timeline on it quite (we're still a small team).
- smhx 7y agoThere's also fastapi, which is well-respected in Python. Someone wrapped fastapi / rabbitRPC to serve PyTorch models (with auto-batching to increase serving efficiency) in https://github.com/catalyst-team/reaction https://github.com/catalyst-team/reaction
- MiroF 7y agoYou can convert the checkpoints over for deployment. I agree the serving infra on TF is much better.