3 ms·
I've worked on models trained on ultra-large tabular data. It still took substantial effort to beat tree models (custom architecture specifically for this parti
by Scene_Cast2 3y ago
I've worked on models trained on ultra-large tabular data. It still took substantial effort to beat tree models (custom architecture specifically for this particular domain, something I haven't seen elsewhere out in the open).
When tabular data is mentioned, one of the unspoken applications is finance. There, my guess is that one of the issues is that data is not very IID and thus latent "events" are fairly sparse. Combine that with the humongous amount of raw data, and you get models that overfit.
- TimPC 3y agoI think there are certain types of tabular data that lend themselves naturally to tree models. But when you're talking about tabular data for finance I guarantee you very few hedge funds are running tree models for trading strategies. When your scale of data is the past X quarters of all stock prices and trade volumes you have enough data that you can fit an NN and there are a number of techniques you can use to reduce overfitting (large amount of data, good regularization, dropout, etc.)
- Jensson 3y ago> But when you're talking about tabular data for finance I guarantee you very few hedge funds are running tree models for trading strategies What do you base this on? Having only neural nets on tabular data is mostly done due to laziness of the creator since neural nets are much easier to use, not because neural nets perform better even with large amounts of data. In general you want both since they are good at finding different kinds of patterns.
- richrichie 3y agoI thought neural nets are universal approximators that could also approximate whatever surface a tree model cooks up.