4 ms·
Because financial data generally means time-series, autocorrelations abound, and it becomes very easy to develop a model that underperforms naive baselines (e.g
by claytonjy 6y ago
Because financial data generally means time-series, autocorrelations abound, and it becomes very easy to develop a model that underperforms naive baselines (e.g. LOCF, arima) or peeks into the future through improper cross validation or feature engineering.
If the customer gives you a column that peeks into the future (e.g. "quarterly sales" when each row is a sale in that quarter), you'll build a model that looks great on metrics and to the customer, and might take months for the customer to realize was practically useless. Are you able to reliably prevent these kinds of issues at a technical level, or do you lean towards customer education ("don't give us quarterly sales") instead?
- avais 6y agoGreat question, and we've seen this happening actually, when some payments-based features used in scoring, data leakage. We are not doing "hard" constraints on this, but there are 2 ways for the user to identify this: first one is feature importance built automatically, so the user will see these features immediately second one is the process of building inference. The same data prep/transformation pipeline assumed to be used as API or for batch processing before model application, and at this point they will definitely realise that they just don't have these features yet for new customers!