This is absolutely correct! After spending a lot of time trying to understand why building machine learning models is so difficult, I came to the same conclusio

This is absolutely correct! After spending a lot of time trying to understand why building machine learning models is so difficult, I came to the same conclusion that "feature engineering" is the key to building high performing models. While feature engineering's importance is generally recognized [0], it's unfortunate that there aren't more tools and formal methods for applying it. Personally, I am a developer of an open source python library called Featuretools (https://github.com/featuretools/featuretools/ https://github.com/featuretools/featuretools/) that is trying to change this for tabular and multi-table datasets. We are working hard to make automated feature engineering available to everyone and have a list of demos for people to try here: https://www.featuretools.com/demos https://www.featuretools.com/demos. It's also worth noting that deep learning is changing the need for feature engineering. However, it primarily works in cases where you don't need interpretable features and you have plenty of data. This means that it's biggest success have been in images, audio, and text problems. For all other use cases feature engineering is still a necessary step for applying machine learning. [0] https://bit.ly/things_to_know_ml https://bit.ly/things_to_know_ml