3 ms·
That loop has to run over high quality, relevant data. The regression model needs to be retrained if the production data drifts from the original training datas
by pbronez 6y ago
That loop has to run over high quality, relevant data. The regression model needs to be retrained if the production data drifts from the original training dataset. The output of the regression model needs to be wired into a downstream application or decision support tool. That downstream application needs to know how to deal with error bars. A data scientist / statistician needs to re-architect the whole thing if the data or business requirement changes substantially.
As usual, the hardest problems are outside of the code.
- bigger_cheese 6y agoI agree with you here Data and measurement is the single most important part of the process. From my experience working in an industrial plant which has been involved in several machine learning trials a lot of the time there are attempts made to use complex modeling techniques to make up for a lack of measurements. Something I question is whether the outcome would have been better if the money which was invested into hiring AI consultants was spent on better plant instrumentation. Industrial Instruments are not cheap something like a PGNAA analyser (https://en.wikipedia.org/wiki/Prompt_gamma_neutron_activation_analysis https://en.wikipedia.org/wiki/Prompt_gamma_neutron_activatio...) is an expensive capital purchase and I suspect some people have unrealistic expectations that AI and machine learning can replace things like this. I think there is some middle ground where AI complements better sensors (maybe instrument companies should be pushing this). I've yet to see any of the data experts push back and say something like "actually you need to measure this better first before we can model it."