4 ms·

>> Advances in computing, data collection, and statistical methods have enabled levels of quantitative analysis around natural experiments that nobody thought p

• naravara 6y ago
  > What's the best study to change my mind? Freakonomics is a pop-Econ book that’s all about behavioral economics and really mainstreamed much of the field when it came out. There’s lots of methodological issues with individual studies but it’s a good primer on how to think like an economist. > The relationship between ideology and social science theory is pretty much intractable, IMO. It’s really not. But I’m not really sure that’s a productive avenue to go down so I’m going to leave it. > The point about social scientists being aware of the limitations around generalizability.... Again, I totally disagree. A non generalizable theory is a just so story, or punditry. Social scientists claim (or at least imply) generalizability, usually. Non generalizability is a retreat in the face of criticism. No. Caveating the limitations of any particular study or finding is simply an exercise in humility about what is known and what is uncertain. “Limits of generalizability” doesn’t mean it’s non-generalizable. It just means it’s not infinitely generalizable time all circumstances. Punditry is the opposite of that. I’m starting to have trouble following what you believe “theory” means if you’re interpreting it in these ways. > IRL, an education in social science, at graduate level, at highly regarded schools, across social science disciplines... is training on how to string together an argument/analysis which generalizes theory, typologies, and whatnot. That is what social science is, in practice. Ah, training on how make an argument and do analysis is the core of what academics is. I have no idea what “generalizes theory and whatnot” is supposed to mean. Most of the work people actually do is looking into case studies and comparing them against each other, doing metaanalysis on a variety of studies, or doing empirical research or hypothesis testing on a particular situation or an interesting data set. There are some groundbreaking works that manage to piece these together into grand theories, but that’s a small minority to what social scientists actually do. The lions share is just looking into “what happens to Y when X changes?” I’m actually going to turn this back around on you. Human beings are complex and society is even moreso. I would argue that an education in hard STEM fields leaves their practitioners incapable of coping with uncertainty, evaluating the quality of evidence, or understanding how to make reasonable inferences in the face of ambiguity. The implication that theories, to be useful, must be infinitely generalizable regardless of extenuating circumstances or that outcomes much be 100% certain to be valid shows this tendency. But these assumptions aren’t actually congruent with how the real world works. The real world is messy, and collected data in it even moreso. Being able to speak about it requires a certain level of judgement and capacity for discrimination that lets you cut through the fog of ambiguity. Rather than weaknesses, these are the skills that social science equips people with.
  • netcan 6y ago
    Like you say, I think we've reached an end point on most of this. FWIW, I think we're using the term "theory" in exactly the same way, given that we've discussed half a dozen examples. I'll just respond to the last point. Human minds and societies are complex. The job is hard. So, what to do? The modern conception of the scientific method (eg a scientific theory must be falsifiable) was laid out most famously by Karl Popper. His work was adopted as a framework by hard sciences. But, it was intended as a criticism of social sciences of his day... notably marxist & freudian fields. The methods of reasoning in these fields. Social science was, I believe, deeply affected by Popper's work. But, I think the direction was entirely wrong. The conclusion should been to abandon scientific pretext when it is not possible. If what you are doing is philosophy, punditry or what Popper called "justificationism," treat it as a nonscientific avenue. Non scientific fields are valid. Non scientific theories presented as scientific are not. They're pseudoscience. The replication crisis in psychology should have been a wake up call. I fear that the loss of position social sciences are experiencing (as you say, they've lost a lot of pull in policy formation) will make things worse, not better. The way social sciences influence the world these days is mostly undergraduate education. That's a lot of influence still, but it's also a refuge from criticism. The only remaining threat to a social science theory is prevailing biases within its own field, because it's unlikely to leave the classroom. Look, leaving economics aside... what are some tangible achievements of social sciences in recent generations? What has behavioural economics or psychology taught us, that we use to some benefit?
    • naravara 6y ago
      > The conclusion should been to abandon scientific pretext when it is not possible. It’s not “not possible.” It’s only applicable under constrained context. Again I point back to certain styles of education leaving people underequipped to cope with ambiguity. That doesn’t mean the methods aren’t useful for understanding systems or processes. > Non scientific theories presented as scientific are not. This is just descending into self-referential semantic hair splitting. You’ve decided on a definition of “science” and you’ve decided that this definition is what social scientists must be trying to hew to based on no direct experience of your own on what social scientists actually do. It sounds like you’re conflating “wonkbloggers,” like 538 or Vox, with what actual social scientists do. (And these bloggers shoddy methods are something social scientists have been complaining about for years). But this is like taking Michio Kaku’s pop-sci writing as representative of how physics works. It’s also not clear why it’s a hill to die on whether something is “scientific.” It’s not social scientists who worry about this, most of them are comfortable with ambiguity and understand what contexts conclusions can be drawn and when they can’t. Nobody actually cares what’s scientific or not except people who are deeply invested in the primacy of positivist scientism as a source of truth. But that fixation is almost exclusively from non-social scientists. There is reason to doubt your sincerity when saying that non-scientific methods are valid, considering that you have, in the same breath, referred to these methods of analysis with pejorative terms like “punditry” or “just so stories.” > The replication crisis in psychology should have been a wake up call. The “replication crisis” isn’t even that germane in most social sciences. It’s more of a thing in clinical trials so I don’t know why you would expect it to be a wake up call to sociologists or political scientists. > Look, leaving economics aside... what are some tangible achievements of social sciences in recent generations? What has behavioural economics or psychology taught us, that we use to some benefit? All of public policy and policy analysis is predicated on various social science frameworks. This is a bizarre question. Where do you think explanations of things like the effects of women’s education of fertility or the most efficacious criminal justice approaches come from? The idea that the primary area of influence is in teaching undergraduates is downright bizarre. Who do you think staffs the State Department or sits on advisory committees at think tanks?