3 ms·
Here is my guess. This is not the first iteration of the computer simulation. They keep tuning the simulation until it produces better and better results. Bu
by btilly 13y ago
Here is my guess.
This is not the first iteration of the computer simulation. They keep tuning the simulation until it produces better and better results. But are you getting better results because the simulation is right, or because it was implicitly fit to the historical data?
In machine learning terms, their training set and test set are the same, so odds are that the model is bunk.
- milesskorpen 13y agoCould you reasonably investigate this by creating a model for one time period, and then applying it to another time period? Alternatively, train it with one region (say Europe) and then test a second region which is mostly isolated (Americas, if we know enough, or Japan / SE Asia).
- icefox 13y agoIf your interested in this in more detail I just finished "guns germs and steel" which I would recommend for a enjoyable (to me) overview of old history.
- yk 13y agoIn the same genre Ian Morris Why the west rules - for now is also very good. ( And if you did not already read it, Collapse by Diamond.)
- btilly 13y agoYes. But if you're trying to improve the model, only take ideas from training in one region. Never, ever, look at how it failed in the test region because you'll quickly fall in the "overfitting for your data" trap.
- tomrod 13y agoThat was my first thought too. But it publishes, I guess.
- yk 13y agoDepends on the model, if they build a model 'from first principles,' whatever that means in history, then they can probably do something interesting. ( Or if they caution themselves enough.) But of course, without actually reading the paper ( ars did link the wrong doi), it is hard to tell what they actually tried to archive or how they build their model.
- teamonkey 13y agoYes, it seems to have exactly the same problems as back testing trading algorithms.