You should checkout Hopsworks (disclaimer: work on it). It does all of the above (including a Feature Store, notebooks as jobs, airflow for ML pipelines, model

You should checkout Hopsworks (disclaimer: work on it). It does all of the above (including a Feature Store, notebooks as jobs, airflow for ML pipelines, model serving (TensorFlowServing, Flask, and soon KFserving), experiments, a project-based multi-tenancy model that supports sensitive data on a shared cluster, and a UI. It does not have a labelling system - but you can pip install libraries. You don't need to learn Docker - each project has a conda environment, and we compile Docker images for projects transparently, so jobs are tied to Docker images, but you don't need to write a Dockerfile (this is a huge win for data scientists). You can run Python jobs (connect to a k8s cluster) or Spark/Flink jobs (on Hopsworks itself). Open-source: * https://github.com/logicalclocks/hopsworks https://github.com/logicalclocks/hopsworks Managed platform on AWS/Azure (with elastic compute/storage, integration with managed K8s, LDAP/AD): * https://hopsworks.ai https://hopsworks.ai