Here's one thing that you missed - transformations. Data engineers hear the word transformations and think they know what they mean - aggregations, binning, da

Here's one thing that you missed - transformations. Data engineers hear the word transformations and think they know what they mean - aggregations, binning, data reductions, cleansing, etc. Data scientists, however, think preparing features for models with transformations - encoding categorical variables (one-hot-encoding, LabelEncoding, OrdinalEncoding) and normalizing/standardizing/log-transforms for numerical features. These transformations are so different that some of them are "model-independent" - you can do the transformations and reuse the output feature across many models (aggregations, binning, feature-crosses, embeddings, etc), but the data scientists transformations are model-dependent (and not reusable across different models - e.g., a XGBoost model doesn't want normalized numerical features, but a DNN typically needs normalized or standardized numerical features. These differences are reflected in how we build our "ML pipelines" - we split them into feature/training/inference pipelines, enabling us to localize model-independent transformations to feature pipelines, while doing model-dependent transformations in training/inference pipelines (while ensuring no skew). In summary, the devil's in the details in pipelines for ML. I agree, however, that orchestrators like Airflow are good enough for orchestration. However, the ML Assets (mutable & reusable features, immutable models, immutable training/inference datasets) are different and tooling will ultimately reflect those differences.