3 ms·
> Trying to frame it as "they modelled it wrong" is nonsense. It's not that. It's that when the system you model responds to the existence of your model, it be
by TeMPOraL 6y ago
> Trying to frame it as "they modelled it wrong" is nonsense.
It's not that. It's that when the system you model responds to the existence of your model, it becomes anti-inductive. It's no longer like weather, but is now like the stock market[0]. Your model suddenly can't predict the system anymore[2], it can at best determine its operating envelope by estimating the degree to which the system can react to the existence of the model.
--
[0] - I use the term anti-inductive per LessWrong nomenclature[1], but I've also been reading "Sapiens" by Yuval Noah Harari, and there he uses terms "first order chaotic" for systems like weather, and "second order chaotic" for systems like the stock market.
[1] - Introduced in https://www.lesswrong.com/posts/h24JGbmweNpWZfBkM/markets-are-anti-inductive https://www.lesswrong.com/posts/h24JGbmweNpWZfBkM/markets-ar....
[2] - I think it becomes uncomputable in Turing sense, but I'm not smart enough to reduce this to the Halting Problem.
- miscorrection 6y agoThe issue is that the example nostromo gave (40m) was not intended to be predictive of what would actually happen. It was based on a worst case / left unchecked scenario (useful for establishing an upper bound), and therefore irrelevant w/r/t the system responding to the model.
- monkeydreams 6y agoAlso many early models were based on SARS and MERS because we had no comparable illnesses, and these were worst case respiratory diseases.
- zwaps 6y agoThis is also known as the Lucas critique, dating from 1976 https://en.wikipedia.org/wiki/Lucas_critique https://en.wikipedia.org/wiki/Lucas_critique and also https://en.wikipedia.org/wiki/Campbell%27s_law https://en.wikipedia.org/wiki/Campbell%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law Edit: I think this is not the first time the good people from lesswrong dug up some well known idea and gave it a new name. Good thing, too, giving this important concept more attention. Too often we forget how many people have dealt with the problem of modeling complex systems in the past. And while we can not read everything, it's often a good idea to have at least a glance at where they failed! Too often I read/review some new "revolutionary" paper based on the idea that hey, we can model this process (involving people) like XYZ from physics, where this stuff works great! Surely, this is better than the plebian approaches in the literature! And then, to the shock of all involved, it doesn't work great.... Also: https://xkcd.com/793/ https://xkcd.com/793/
- krzepah 6y agofun fact comment, Asimov thought about this, I'm quoting the short part that is relevant from this article on wikipedia :https://en.wikipedia.org/wiki/Foundation_series https://en.wikipedia.org/wiki/Foundation_series "One key feature of Seldon's theory, which has proved influential in real-world social science,[3] is the uncertainty principle: if a population gains knowledge of its predicted behavior, its self-aware collective actions become unpredictable."