8 ms·
The talk focuses for a bit on having pure data from before the given date. But it doesn't consider that the data available from before that time may be subject
by abeppu 11mo ago
The talk focuses for a bit on having pure data from before the given date. But it doesn't consider that the data available from before that time may be subject to strong selection bias, based on what's interesting to people doing scholarship or archival work after that date. E.g. have we disproportionately digitized the notes/letters/journals of figures whose ideas have gained traction after their death?
The article makes a comparison to financial backtesting. If you form a dataset of historical prices of stocks which are _currently_ in the S&P500, even if you only use price data before time t, models trained against your data will expect that prices go up and companies never die, because they've only seen the price history of successful firms.
- alalv 11mo agoIt mentions that problem in the first section
- malkia 11mo agoNot a financial person by any means, but doesn't the Black Swan Theory basically disproves such methods due to rarity of an event that might have huge impact without something to predict (in the past) that it might happen, or even if it can be predicted - the impact cannot? For example: Chernobyl, COVID, 2008 financial crisis and even 9/11
- ACCount37 11mo agoAll models are wrong, but some are useful. If you had a financial model that somehow predicted everything but black swan events, that would still be enough to make yourself rich beyond belief.
- dboon 11mo agoThe talk explicitly addresses this exact issue.