4 ms·
> Show that your approach works on real-world data, that you didn't have access to when training your model. Show that your models can predict the future, not t
by candiodari 4y ago
> Show that your approach works on real-world data, that you didn't have access to when training your model. Show that your models can predict the future, not the past. Otherwise, your experiments don't mean, what you think they mean.
Great, but ... that requires you have collection infrastructure and almost by definition means you can't make it work on a dataset. The way ML research works is like the Kaggle competitions.
You get a dataset. This can be anything, as long as it can be expressed as a tensor (which are vectors, but can have more dimensions). Then split into seen and unseen data. You split into 3 portions/partitions. One for training. One for testing, and one for blind validation (80% training and 10% each for test and validation sets).
In order to simulate what you're suggesting researchers can, and often do, take the most recent datapoints as validation and testing sets.
But extending the data set is an expensive process that generally can't be done by the machine learning researchers themselves. So it is not done.