3 ms·

Auto ML is great as long as there is some rational on why a particular model is developed. However, in real world applications, there are so many constraints an

Auto ML is great as long as there is some rational on why a particular model is developed. However, in real world applications, there are so many constraints and considerations needed besides just having the most accurate model. Furthermore, as a VP of data science, selling the business on the value and benefits of data science is non-trivial as many are not even aware of what's capable with data science so my personal opinion is that ML isn't quite suitable for mainstream work. Practitioners still need to have a strong grasp of the business context, data features and nuances, etc. which still is quite technical in nature. Asking a data analyst to build models with auto ML is something I consider to be a leap too far and is risky to put into production. The only way it could work is to have an experienced data scientist supervise.

mlthoughts2018 7y ago
Exactly this. The combination of selling a modeling strategy to higher ups, customizing a model for super weird deployment or resource limitations that exist for political or historical reasons, and all your standard trade-offs vs feature engineering and model selection just means that AutoML (and similar tools) has very little applicability in most product companies. If you work in Google & you can remove the political blockers & guarantee the model’s space of deployed resources is parametrizable with a clean set of parameters that AutoML can consider as part of the optimization, then by all means use it. That’s just decades away from being viable at any given product company.
eyeball 7y ago
>The only way it could work is to have an experienced data scientist supervise. That appears to be the business model of https://www.datarobot.com/ https://www.datarobot.com/ automated tools with consulting data scientists to help clients use them effectively

mlthoughts2018 7y ago

Exactly this. The combination of selling a modeling strategy to higher ups, customizing a model for super weird deployment or resource limitations that exist for political or historical reasons, and all your standard trade-offs vs feature engineering and model selection just means that AutoML (and similar tools) has very little applicability in most product companies. If you work in Google & you can remove the political blockers & guarantee the model’s space of deployed resources is parametrizable with a clean set of parameters that AutoML can consider as part of the optimization, then by all means use it. That’s just decades away from being viable at any given product company.

eyeball 7y ago

>The only way it could work is to have an experienced data scientist supervise. That appears to be the business model of https://www.datarobot.com/ https://www.datarobot.com/ automated tools with consulting data scientists to help clients use them effectively