3 ms·

In their defense, I tend to think social science is shoehorned into an academic formula that it can never really fit into. You kind of have to have a hypothesi

In their defense, I tend to think social science is shoehorned into an academic formula that it can never really fit into. You kind of have to have a hypothesis, then "test" it with data. A more reasonable approach (IMO) would be "We found this interesting phenomenon" followed by inevitably speculative interpretation. That's realistically how it works in practice, but to be published you need to fit a popperian formula. A more ponderous version of this probably wouldn't have been accepted. I mean, using a method of diagnosing depressed individuals on a collective psychology of a society is already veering into wtf territory, in terms of interpretation. OTOH, it is interesting and probably significant somehow. IDK how or what they could have done better, and this does seem like a result worth publishing, assuming the methodology is good. Ultimately, I think people reading the paper mostly read it that way anyhow. Authors found X. Seems interesting. Here's how they're interpreting it. Here's my speculation. Journalists OTOH, they'll cite the hypothesis verbatim if it fits the article they're writing. I think that's where the shoehorning is a problem. As long as you're reading the paper directly yourself, who cares what order the paragraphs are in.

yosito 5y ago
> In their defense, I tend to think social science is shoehorned into an academic formula that it can never really fit into. Pesky reproducibility!
kortilla 5y ago
Then maybe we should come up with a new term for it and stop calling it science. If it’s not reproducible or falsifiable, it’s not producing scientific conclusions that expand our understanding of the world. It’s certainly useful data, but otherwise it lacks any meaningful analysis.
- greenbamboo 5y ago
  You probably can’t stop other people from using words you believe or even know to be inappropriate. Thankfully it’s sometimes easy to spot science that’s not really science. It’s the stuff that seems legit but isn’t that’s a problem. I’d say the problem is actually bigger in harder sciences. Anybody can look at most social science research and see it’s dodgy. Collusion rings in comp sci research are harder to detect, just to pick an example.
- tomp 5y ago
  I’ve long stopped using the phrase “social sciences” and use “social studies” instead. (I include medicine here, or at least studies with 5% p-value… not science if compared to e.g. 5-sigma physics.)
- PeterisP 5y ago
  The process of science needs this useful data - a problem is that we currently have coupled the publication of these useful observations with scientific conclusions, which results in publishing lousy conclusions and not publishing valuable observations (e.g. negative results); perhaps it would be better to decouple these things so that people can (and would be motivated) to publish their primary studies/observations, where there would not necessarily be anything falsifiable, and the conclusions can be done afterwards by others, preferably combining multiple observational studies (thus being not only reproducible but reproduced), like e.g. meta-studies often used in medicine.
- jean_tta 5y ago
  How is it useful if it does not expand our understanding of the world? What use does it have?
- dalbasal 5y ago
  Reproducibility is not a problem here. Falsifiability is. That said, this can't be "useful data" unless you are willing to accept non falsifiable interpretations. There's no way of formulating such an interpretation. Whether or not we call it science is semantics, highly loaded semantics. We could stop calling it science, but then we'd have to also stop being derisive of unscientific methodology. More realistically, maybe we should rename "the scientific method" to "the Popperian standard." After all, both the term science and its associated culture predate the definition/standard you are alluding to. The researchers themselves haven't done anything wrong. They're researching interesting things in theory field, and publishing in the format journals demand... ostensibly to satisfy objections such as yours. You can't have it both ways.
amatic 5y ago
> Authors found X. Seems interesting. Here's how they're interpreting it. Here's my speculation. Agreed, that is mostly how it goes. On a similar note, null-hypothesis testing is not a true and tried scientific method. It started sometime in the early 20th century. Aside from p-hacking and mistakes of that kind, there are many philosophical problems with the use of statistical inference itself.

yosito 5y ago

> In their defense, I tend to think social science is shoehorned into an academic formula that it can never really fit into. Pesky reproducibility!

kortilla 5y ago

Then maybe we should come up with a new term for it and stop calling it science. If it’s not reproducible or falsifiable, it’s not producing scientific conclusions that expand our understanding of the world. It’s certainly useful data, but otherwise it lacks any meaningful analysis.

greenbamboo 5y ago
You probably can’t stop other people from using words you believe or even know to be inappropriate. Thankfully it’s sometimes easy to spot science that’s not really science. It’s the stuff that seems legit but isn’t that’s a problem. I’d say the problem is actually bigger in harder sciences. Anybody can look at most social science research and see it’s dodgy. Collusion rings in comp sci research are harder to detect, just to pick an example.
tomp 5y ago
I’ve long stopped using the phrase “social sciences” and use “social studies” instead. (I include medicine here, or at least studies with 5% p-value… not science if compared to e.g. 5-sigma physics.)
PeterisP 5y ago
The process of science needs this useful data - a problem is that we currently have coupled the publication of these useful observations with scientific conclusions, which results in publishing lousy conclusions and not publishing valuable observations (e.g. negative results); perhaps it would be better to decouple these things so that people can (and would be motivated) to publish their primary studies/observations, where there would not necessarily be anything falsifiable, and the conclusions can be done afterwards by others, preferably combining multiple observational studies (thus being not only reproducible but reproduced), like e.g. meta-studies often used in medicine.
jean_tta 5y ago
How is it useful if it does not expand our understanding of the world? What use does it have?
dalbasal 5y ago
Reproducibility is not a problem here. Falsifiability is. That said, this can't be "useful data" unless you are willing to accept non falsifiable interpretations. There's no way of formulating such an interpretation. Whether or not we call it science is semantics, highly loaded semantics. We could stop calling it science, but then we'd have to also stop being derisive of unscientific methodology. More realistically, maybe we should rename "the scientific method" to "the Popperian standard." After all, both the term science and its associated culture predate the definition/standard you are alluding to. The researchers themselves haven't done anything wrong. They're researching interesting things in theory field, and publishing in the format journals demand... ostensibly to satisfy objections such as yours. You can't have it both ways.

amatic 5y ago

> Authors found X. Seems interesting. Here's how they're interpreting it. Here's my speculation. Agreed, that is mostly how it goes. On a similar note, null-hypothesis testing is not a true and tried scientific method. It started sometime in the early 20th century. Aside from p-hacking and mistakes of that kind, there are many philosophical problems with the use of statistical inference itself.