4 ms·

This particular excerpt resonated with me: > It’s hard to avoid a disheartening conclusion: Quantitative social science is barely relevant in the real world –

This particular excerpt resonated with me: > It’s hard to avoid a disheartening conclusion: Quantitative social science is barely relevant in the real world – and almost every social scientist covertly agrees. The complex math that researchers use is disposable. You deploy it to get a publication, then move on with your career. When it comes time to give policy advice, the math is AWOL. If you’re lucky, researchers default to common sense. Otherwise, they go with their ideology and status-quo bias, using the latest prestigious papers as fig leaves. Empirical social science teaches us far more about the world than pure theory. Yet in practice, even empirical researchers barely care what empirical social science really has to teach. It rings true with me even in casual discussions with peers when talking about social issues.

a123b456c 5y ago
Two possible counter arguments: 1) The people who do the math are too busy read these articles and books. 2) Amazon has 15 phd economists running a single structural model of consumer behavior. They use it to inform feature changes.
- deleted 5y ago
  [deleted]
- jmeister 5y ago
  2 only amplifies Caplan’s broader point about the importance of corrective feedback mechanisms(from real or prediction markets or anywhere else). I remember my manager in finance telling me that he would readily use a “mickey-mouse-in-a-box” as a model if it made him money.
  - HPsquared 5y ago
    Applying results from social sciences has the problem of weighing costs and benefits (policy X helps with A but hurts B). But how much 'A' do you need to get to outweigh the loss in 'B'? Business decisions are far simpler: both sides of a decision can be analytically boiled down to the same units (dollars/cents) and compared (more money = better than).
urthor 5y ago
I disagree, insofar in that qualitative social science is ineffective. It's effective. Nobody listens. The fallacy of "general management" means that general managers, who have a background in finance and cost/benefit, are not receptive to the idea that deep knowledge is extremely involved in leadership and decision making. Because then they'd be out of a job.
- xyzzyz 5y ago
  If evidence we obtain from social sciences is so effective, why are they suffering from deep and wide reaching replication crisis?
  - urthor 5y ago
    It's more of a case that bad science was made, rather than "this type of science is bad." Toyota and GM have had widespread vehicle recalls in recent types. I wouldn't put that down to "4 wheeled transportation devices being intrinsically bad."
    - fartboi 5y ago
    - HPsquared 5y ago
      It depends how deep and far-reaching it is. Cars in the 60s caused smog and used leaded fuel, so they were all pretty bad at the same time. Perhaps the field needs a similar size of transition as the change to unleaded (too many "leading questions" perhaps).
  - rocqua 5y ago
    If people start publishing faked physics papers, does that mean physics is no longer an effective source for engineering?
- barry-cotter 5y ago
  > I disagree, insofar in that qualitative social science is ineffective. > It's effective. Nobody listens. This is exaggerated but in the right direction. Consider A/B testing. We know that works. We know how. It’s been a big deal for well over a decade. And it’s still not standard. There are large, profitable companies that sell direct to consumers that don’t do it who would reliably, predictably get a 10% bump in profits off of doing it. But it does grow. Selling things on the internet isn’t as competitive as highly liquid, transparent financial markets but people do pick up $100 bills lying on the street eventually.
tgv 5y ago
Discussion might have been a bit easier if he'd just listed the assumptions and the model underlying the calculations. But the Amazon blurb of the book shows that the author isn't really interested in that > Learn why students hunt for easy As and casually forget most of what they learn after the final exam We know why, ok? And focusing on "the primary function of education" is like focusing on the primary function of cars, and concluding that traffic accidents contribute to the economy. It reads like a libertarian disguised as a utilitarian who dreams of the 19th century.
- edanm 5y ago
  > Discussion might have been a bit easier if he'd just listed the assumptions and the model underlying the calculations. But the Amazon blurb of the book shows that the author isn't really interested in that He wrote a whole book about this. A very detailed book.
  - tgv 5y ago
    If it's all in there, what's the point of discussing the spreadsheets? You won't care about that unless you disagree with the author yet make the exact same assumptions. So it's almost 100% certain you'd argue assumptions and modeling before the actual calculations. Doesn't seem very rocket-science-y to me.

a123b456c 5y ago

Two possible counter arguments: 1) The people who do the math are too busy read these articles and books. 2) Amazon has 15 phd economists running a single structural model of consumer behavior. They use it to inform feature changes.

deleted 5y ago
[deleted]
jmeister 5y ago
2 only amplifies Caplan’s broader point about the importance of corrective feedback mechanisms(from real or prediction markets or anywhere else). I remember my manager in finance telling me that he would readily use a “mickey-mouse-in-a-box” as a model if it made him money.
- HPsquared 5y ago
  Applying results from social sciences has the problem of weighing costs and benefits (policy X helps with A but hurts B). But how much 'A' do you need to get to outweigh the loss in 'B'? Business decisions are far simpler: both sides of a decision can be analytically boiled down to the same units (dollars/cents) and compared (more money = better than).

urthor 5y ago

I disagree, insofar in that qualitative social science is ineffective. It's effective. Nobody listens. The fallacy of "general management" means that general managers, who have a background in finance and cost/benefit, are not receptive to the idea that deep knowledge is extremely involved in leadership and decision making. Because then they'd be out of a job.

xyzzyz 5y ago
If evidence we obtain from social sciences is so effective, why are they suffering from deep and wide reaching replication crisis?
- urthor 5y ago
  It's more of a case that bad science was made, rather than "this type of science is bad." Toyota and GM have had widespread vehicle recalls in recent types. I wouldn't put that down to "4 wheeled transportation devices being intrinsically bad."
  - fartboi 5y ago
  - HPsquared 5y ago
    It depends how deep and far-reaching it is. Cars in the 60s caused smog and used leaded fuel, so they were all pretty bad at the same time. Perhaps the field needs a similar size of transition as the change to unleaded (too many "leading questions" perhaps).
- rocqua 5y ago
  If people start publishing faked physics papers, does that mean physics is no longer an effective source for engineering?
barry-cotter 5y ago
> I disagree, insofar in that qualitative social science is ineffective. > It's effective. Nobody listens. This is exaggerated but in the right direction. Consider A/B testing. We know that works. We know how. It’s been a big deal for well over a decade. And it’s still not standard. There are large, profitable companies that sell direct to consumers that don’t do it who would reliably, predictably get a 10% bump in profits off of doing it. But it does grow. Selling things on the internet isn’t as competitive as highly liquid, transparent financial markets but people do pick up $100 bills lying on the street eventually.

tgv 5y ago

Discussion might have been a bit easier if he'd just listed the assumptions and the model underlying the calculations. But the Amazon blurb of the book shows that the author isn't really interested in that > Learn why students hunt for easy As and casually forget most of what they learn after the final exam We know why, ok? And focusing on "the primary function of education" is like focusing on the primary function of cars, and concluding that traffic accidents contribute to the economy. It reads like a libertarian disguised as a utilitarian who dreams of the 19th century.

edanm 5y ago
> Discussion might have been a bit easier if he'd just listed the assumptions and the model underlying the calculations. But the Amazon blurb of the book shows that the author isn't really interested in that He wrote a whole book about this. A very detailed book.
- tgv 5y ago
  If it's all in there, what's the point of discussing the spreadsheets? You won't care about that unless you disagree with the author yet make the exact same assumptions. So it's almost 100% certain you'd argue assumptions and modeling before the actual calculations. Doesn't seem very rocket-science-y to me.