4 ms·
This appears to be Yet Another "we threw some code together and published the results without feeling obligated to check our model against the real world" study
by jerf 15y ago
This appears to be Yet Another "we threw some code together and published the results without feeling obligated to check our model against the real world" study. At least if they did any sort of real-word verification, the article doesn't mention it.
The assumptions strike me as unreasonable, the results worthless because the details matter a lot. Another prime candidate for "we tuned our unverified model until it did something publishable", too.
- Geee 15y agoThat passes publishable, it doesn't have to be final verdict on the subject. It should be taken as is. They mentioned observing the revolutions which happened really fast and would fit in the model.
- jerf 15y agoYes, the low bar of publishable is part of my point. I think I failed to communicate my main point, which is that you can make these iterative models do any damn thing you please. It's a fun sort of entertainment for programmers, kind of like those "evolve a cart that runs over random terrain" things that pop up every so often... but without proving a correlation to the real world, it's useless. I can make very small tweaks to their model and make the threshold 20%. I can make very small tweaks to their model and make the threshold 5%. How do you judge which of these three results is correct? Or how do you judge if a single numeric threshold is even meaningful in the real world? No, seriously, don't just blip over that. How do you verify this result? How do you tell which is correct? There is no evidence in the link that anybody has asked that. Maybe they did, but it's not a negligible detail, it's very important! Here, run one of them in your browser: http://www.openprocessing.org/visuals/?visualID=6599 http://www.openprocessing.org/visuals/?visualID=6599 Have fun yourself seeing how very small tweaks, well below the real-world noise threshold, can have profound effects, the source is right there. It's not an interesting result, without some sort of verification against the world, and cherry picking a few current events and making vague allusions to how it sort of acts like a system you threw together isn't verification in the slightest.
- Helianthus 15y agoThis may be a useless post since it just reiterates, but I feel strongly that I should re-emphasize that yes, _very small tweaks_ can make the threshold 20% or 5%, so the number is essentially pulled out of a hat.
- jacques_chester 15y agoFinance types refer to this as a "sensitive" model -- it's not robust in the face of small changes to parameters. A good way to explore these problems is to perform monte carlo simulations over the set of possible parameters and then derive general behaviour from that -- looking in particular for 'catastrophic' features in the phases space. Later you can narrow your data set down based on empirically observed parameters. As an undergraduate I wrote a standard grid-based model of epidemics which does this. So it's not as though it's rocket science.