Oh my! We were exactly like this many, many years ago. See reply to this thread[0]. The way I view it is frictions and impedance mismatch. People lived in seve

Oh my! We were exactly like this many, many years ago. See reply to this thread[0]. The way I view it is frictions and impedance mismatch. People lived in several universes and there were many "taps on shoulders". Data scientist tries to work on a project but the system upgrade messed up their compute environment and their GPU isn't working anymore. Data scientsits ssh'ing into a "powerful workstation" to have their notebooks run on more RAM or more powerful GPUs, having a certain convention to start their notebook servers with specific ports. Building models and then wanting to show results to the client and asking a colleague. Set up a VM on GCP, write a small application, scp the model to the machine, create an environment with the same dependencies to load the model, set up authentication on the machine. Email the client. Client doesn't reply in time. You have a bunch of VMs. Meanwhile the data scientist has produced another model with a notebook and they want the engineer to deploy it. Others want to reproduce it but have the same trouble with running the notebook (libraries, etc.). A complete mess. We ended up building our platform[0]. We wanted our PhDs to do what they were good at, and we wanted to handle a lot for them. In the same time, we wanted our more engineering inclined colleagues not to do that work themselves, and we let the platform do many of these things (building images, deploying, scheduling notebooks, etc). - [0]: https://iko.ai https://iko.ai