3 ms·
I'm surprised that nothing here has been written about human in the loop. All time series models are flawed in some capacity, and are going to have difficulties
by mcastillon 5y ago
I'm surprised that nothing here has been written about human in the loop. All time series models are flawed in some capacity, and are going to have difficulties in the current market given how it won't necessarily match historical seasonalities. Just having more human safeguards to flag wacky outputs seems to be a more reasonable explanation than not handling Prophet's shortcomings properly. Zillow would have likely faced similar issues with other time series packages as well
- mcastillon 5y agoReading some more, it does look like Zillow was in fact leveraging humans in the loop https://venturebeat.com/2019/12/11/why-explainable-ai-is-indispensable-to-zillows-business/ https://venturebeat.com/2019/12/11/why-explainable-ai-is-ind.... What seems like a likely explanation is that the proper balance between machine and human judgement was not calibrated to this current market. "Part of the challenge of Zillow Offers’ human-in-the-loop system — any such system, really — is finding the balance between humans and machines. “In order to optimize this human-in-the-loop system, we’d like to figure out when the human is best, when an assistive situation is best, and […] when a machine is best,” said Fagnan."