I'm in the same situation. I found this cog project to dockerise ML https://github.com/replicate/cog https://github.com/replicate/cog : you write just one pyth

I'm in the same situation. I found this cog project to dockerise ML https://github.com/replicate/cog https://github.com/replicate/cog : you write just one python class and a yaml file, and it takes care of the "CUDA hell" and deps. It even creates a flask app in front of your model. That helps keep your system clean, but someone with big $s please rewrite pytorch to golang or rust or even nodejs / typescript.