3 ms·

I’m a big fan of open data but struggle to articulate its value in a quantified way other than just my gut feeling of more transparency leads to more collaborat

I’m a big fan of open data but struggle to articulate its value in a quantified way other than just my gut feeling of more transparency leads to more collaboration leads to more research leads to more knowledge. But I think examples like this mean that we’re getting closer to if a study doesn’t release all its data- raw, cleaned, and analytical tables- then it will be discounted or ignored. And there’s fame and money to be made in finding data irregularities. We still need some way to audit and confirm data regularities, but that is boring and I think no money there. I’m not sure if there will be some chore assignment for students to pick a dataset, analyze it for rigor, and post results somewhere so every paper has a link to people who checked data and found it appropriate. I hope this means 99.9% of studies have proper data and not just when we hear about the (hopefully) rare studies that falsify or just don’t work with properly and make inaccurate conclusions.

arsome 3y ago
Honestly, most of these studies are getting caught simply by not having the expected distribution on their data. I suspect improving this analysis will more often just lead to better fakes. Unfortunately I think this means we need to disregard these popsci papers, especially in the social sciences unless there's a clear, uninterested party providing the data as well.
- somedude895 3y ago
  Sociological studies should be disregarded and remain uncited until they're reproduced by an unaffiliated team that isn't ideologically aligned with the original researchers. Which isn't gonna happen for most studies since it's prohibitively expensive and most sociologists share the same biases.
hilbert42 3y ago
"We still need some way to audit and confirm data regularities," See my post that mentions Benford's Law, it's damn good a sussing out fraudulent data. Moreover, it's quite old and has stood the test of time.

arsome 3y ago

Honestly, most of these studies are getting caught simply by not having the expected distribution on their data. I suspect improving this analysis will more often just lead to better fakes. Unfortunately I think this means we need to disregard these popsci papers, especially in the social sciences unless there's a clear, uninterested party providing the data as well.

somedude895 3y ago
Sociological studies should be disregarded and remain uncited until they're reproduced by an unaffiliated team that isn't ideologically aligned with the original researchers. Which isn't gonna happen for most studies since it's prohibitively expensive and most sociologists share the same biases.

hilbert42 3y ago

"We still need some way to audit and confirm data regularities," See my post that mentions Benford's Law, it's damn good a sussing out fraudulent data. Moreover, it's quite old and has stood the test of time.