30 ms·

> The model makes use of “alternative data” – such as credit card purchase data, smartphone location data, satellite imagery and so on The problem with predict

by awb 7y ago

> The model makes use of “alternative data” – such as credit card purchase data, smartphone location data, satellite imagery and so on The problem with predicting markets is that they suck up information. When these predictions become public or anyone acts on them, the market automatically adjusts. Then the cycle repeats with people looking for even more leading indicators because the old ones are already priced in. So, while it's interesting that these "alternative" data points seem correlated with prices, I'm dubious that anyone will profit off them at above-market returns for any sustained period of time.

wenc 7y ago
To build on what you've said, I've learned from experience that alternative data can be misleading in the presence of unusual tail events. It's the fundamental bias-variance tradeoff. I was benchmarking one of our simple models (based on known causal variables) with another from a third-party that claims to be augmented with alternative data. When things were normal, both models behaved similarly. When things were out of left field, the alternative data model tended to be more wrong, whereas the simple model tended to be more robust and less wrong. This is a surprising but well-known phenomenon in mathematical modeling [1, 2] -- simple models tend to outperform complex models in complex situations. Complex models can give you accuracy refinements during normal situations but need to be detuned/weighted-reduced during abnormal situations. [1] https://www.johndcook.com/blog/2012/09/17/robustness-of-simple-rules/ https://www.johndcook.com/blog/2012/09/17/robustness-of-simp... [2] https://sloanreview.mit.edu/article/why-forecasts-fail-what-to-do-instead/ https://sloanreview.mit.edu/article/why-forecasts-fail-what-...
kevin_thibedeau 7y ago
This is precisely what all the quant based investment firms have been doing for 25+ years.

by awb 7y ago

wenc 7y ago

To build on what you've said, I've learned from experience that alternative data can be misleading in the presence of unusual tail events. It's the fundamental bias-variance tradeoff. I was benchmarking one of our simple models (based on known causal variables) with another from a third-party that claims to be augmented with alternative data. When things were normal, both models behaved similarly. When things were out of left field, the alternative data model tended to be more wrong, whereas the simple model tended to be more robust and less wrong. This is a surprising but well-known phenomenon in mathematical modeling [1, 2] -- simple models tend to outperform complex models in complex situations. Complex models can give you accuracy refinements during normal situations but need to be detuned/weighted-reduced during abnormal situations. [1] https://www.johndcook.com/blog/2012/09/17/robustness-of-simple-rules/ https://www.johndcook.com/blog/2012/09/17/robustness-of-simp... [2] https://sloanreview.mit.edu/article/why-forecasts-fail-what-to-do-instead/ https://sloanreview.mit.edu/article/why-forecasts-fail-what-...

kevin_thibedeau 7y ago

This is precisely what all the quant based investment firms have been doing for 25+ years.