3 ms·
To build on what you've said, I've learned from experience that alternative data can be misleading in the presence of unusual tail events. It's the fundamental
by wenc 7y ago
To build on what you've said, I've learned from experience that alternative data can be misleading in the presence of unusual tail events. It's the fundamental bias-variance tradeoff. I was benchmarking one of our simple models (based on known causal variables) with another from a third-party that claims to be augmented with alternative data. When things were normal, both models behaved similarly. When things were out of left field, the alternative data model tended to be more wrong, whereas the simple model tended to be more robust and less wrong.
This is a surprising but well-known phenomenon in mathematical modeling [1, 2] -- simple models tend to outperform complex models in complex situations.
Complex models can give you accuracy refinements during normal situations but need to be detuned/weighted-reduced during abnormal situations.
[1] https://www.johndcook.com/blog/2012/09/17/robustness-of-simple-rules/ https://www.johndcook.com/blog/2012/09/17/robustness-of-simp...
[2] https://sloanreview.mit.edu/article/why-forecasts-fail-what-to-do-instead/ https://sloanreview.mit.edu/article/why-forecasts-fail-what-...