3 ms·
It is a fantastic article, but I don't think excessive, obsessive nuance is what plagues most people here. In any event, don't take it too far. In dynamical sys
by pron 11y ago
It is a fantastic article, but I don't think excessive, obsessive nuance is what plagues most people here. In any event, don't take it too far. In dynamical system, your theory can be simple and predict observation 100% yet be completely wrong -- not just a little wrong -- and fail to model any of the actual dynamics (e.g. x = 0 and x = t are both completely predictive, yet entirely false, models of x" = 0). It is those simplified theories that are no more than empirical observation, and suffer from another kind of overfitting.
> In statistical terms, these dismissals are little more than a call for overfitting. can arbitrarily add more dimensions to a theory, I can make it fit any data set.
In order to fit any social data set, you may need dozens if not hundreds of variables. If your data set contains many millions of data points, telling you that there may be four variables instead than two is certainly not "a call for overfitting"; it is no more excessive nuance than the entirety of special+general relativity. Or, don't use this as an excuse to stop questioning your theory once it predicts behavior in one setting. You may have completely missed the mark by misjudging the order of the equation. Let alone, when your theory doesn't even fit the data we have -- just the data you have -- it's nothing more than amateurism; don't confuse amateur smart-assedness with any sort of profundity; it is often wrong in the most boring way: uninformed wrong.
Intuition that comes from years of study of human behavior may help a lot, but some believe they can come up with theories without spending the time to earn their intuition first. Whether overly nuanced or over-simplified, I think the chances that any of their theories are correct -- especially if they put little effort in forming them -- are low.
And one thing is certain: the less you know, the far more likely it is that you'll err on the side of over-simplification than over-nuance. This paper is a warning to experts, who know enough to err with over-care. Amateurs (of the non-humble kind) are indeed completely protected from this particular kind of error by their ignorance, especially if they don't have the basic tools to even assess the system's level of complexity.
This paper is a call against obsessive, exaggerated nuance. Not a call for oversimplification and sloppiness. And while one can err on both sides, historically, errors of over-simplification (when it comes to social theories) have been much more detrimental to the human race than errors of nuance. The latter kind may cause boring science; the former has led to atrocities. I think that many of the calls for more nuance are just warnings against careless errors of the first kind, simply because they are far more dangerous than errors of the second kind.
- yummyfajitas 11y agoIn any event, don't take it too far. In dynamical system, your theory can be simple and predict observation 100% yet be completely wrong -- not just a little wrong -- and fail to model any of the actual dynamics (e.g. x = 0 and x = t are both completely predictive, yet entirely false, models of x" = 0). It is those simplified theories that are no more than empirical observation, and suffer from another kind of overfitting. No, this is not overfitting at all. This is failing to fit the data. Based on your true model of x''=0, x=0.5t and x=-t are also valid data sets which can be observed but which are not predicted. And one thing is certain: the less you know, the far more likely it is that you'll err on the side of over-simplification than over-nuance. When this is the case, there is only one valid response: "your theory predicts X but experiment shows Y." "You don't have enough expertise", "you haven't been hazed with years of study", "you are an amateur", etc, are simply ad hominem attacks. So is blaming scientists with simple theories for "atrocities". The fact is that lots of highly successful social theories have very few variables - supply&demand, EMH, psychometrics (g) are all great examples. They make testable predictions which accurately predict the world. Lack of nuance only becomes an issue when you want to go from 85% to 95% accurate.
- pron 11y ago> This is failing to fit the data. This is assuming that the non-fitting data is not ignored, if it is known to the theorist at all. If anything, the bigger problem among social science amateurs is obliviousness to data or lack of intuition to understand how biased collected data is simply because they're unaware of historical data. Hence, they come up with theories like x = t. > there is only one valid response: "your theory predicts X but experiment shows Y." Sure, except that the falsifying experiment took place 890 years ago, you likely don't know about it, and if you did, you'd find excuses (possibly good ones) for dismissing it as irrelevant. The problem is, of course, that experiments are very hard to conduct, and all-but-impossible to conduct under lab conditions. The reason intuition is so important is that we know with absolute certainty that some things that seem so natural and stable now, used to be diametrically different, and they changed because people changed them. If you know from experience that 80% of coefficients are known to vary considerably over time and as a result of human action, you tend to be very skeptical when someone claims they have found a "natural law" of human society. > So is blaming scientists with simple theories for "atrocities". Perhaps it's ad hominem, but it really happened and more than once (except the atrocities were real and required no quotation marks). I don't know what else to tell you. > They make testable predictions which accurately predict the world Maybe, and maybe they theorize that x = t when, in fact, x" = 0. Showing that x = -t may also be a valid solution requires making huge changes, among them changing the perception that the theories are correct, which in itself perpetuates the path of the system. The only thing you can do is point to historical trends, but historians, most of them ever so careful, just sigh in exasperation, while the masses believe the "science". Just today I read something wonderful about perception of the sexes[1], yet I believe many might find simple models that perfectly predict the world they see around them, not realizing it is no more than the result of people who made it this way so-and-so years ago, and they can make it different yet again. It's like having a reverse Cassandra complex only in reverse, and it can be just as frustrating. Do you have any doubts that Netflix's prediction engine shapes people preferences more than it passively predicts them? I am not saying that it doesn't suggest movies people enjoy, only that it could have made completely different suggestions that people may have enjoyed even more. How do you measure the effectiveness of this algorithm against all others (and there are way more possible algorithms that can be tested with the sample they have)? Or do you just declare it successful and accurate? [1]: https://www.reddit.com/r/AskHistorians/comments/4c5l63/in_the_middle_ages_women_were_considered/ https://www.reddit.com/r/AskHistorians/comments/4c5l63/in_th...