11 ms·
I think I say this every time I see a headline like this on HN, but I'll say it again: The past decade of social science research publication has proven that pr
by peterbonney 5y ago
I think I say this every time I see a headline like this on HN, but I'll say it again: The past decade of social science research publication has proven that provocative-sounding results should be considered fraudulent until the underlying data has been published, and then (assuming the data passes muster) should be regarded with skepticism until independently replicated with a fresh data set.
- uniqueuid 5y agoSad but true, and extends to economics, psychology, lots of neurological and medical research as well. Everyone needs pre-registration, and everyone needs open data.
- q1w2 5y agoOne of the most eye-opening studies I saw was that social science papers with the most citations, had the highest likelihood of not being reproducible. Meaning that Social Sciences are actually pushing out misinformation more than not.
- hannob 5y agoThis isn't an issue of social science. This is an issue that has been found in a diverse number of fields, basically everywhere where someone has looked: https://www.frontiersin.org/articles/10.3389/fnhum.2018.00037/full https://www.frontiersin.org/articles/10.3389/fnhum.2018.0003...
- gmadsen 5y agoits a little disingenuous to make it seem like all fields are equal. when that is clearly not the case
- hulahoof 5y agoIs this somehow a flow on effect of more commonly cited papers having more reproduction attempts?
- midjji 5y agoNo, a moderately successful paper has 1-10 k citations, and usually 0-1 reproductions. Extremely successful papers have 10-40 k citations and 1-3 reproductions. They largely dont matter at all. Its mostly because extraordinary claims receive more attention, and thus is found by the group for whom the result supported their prior beliefs. They will use it whenever they need to support such. This is also why such studies tend to keep getting cited as evidence long after a long line of failures to replicate have been published. As the result is non-replicable, there also wont be a newer, improved version of the publication, meaning all cites go to the original.
- q1w2 5y agoThe theory is that there is a pressure in certain academic fields to reinforce existing popular narratives with new publications, motivating researchers to confirm them, and thus for other researchers to cite them. ...but since such efforts attract low-quality researchers and are based on an implicit initial bias, due to its self-referential nature, that the results are not reproducible by others in the future since teams that try to reproduce results are usually more neutral on the topics.
- Nasrudith 5y agoThat doesn't technically mean misinformation although but it does imply major room for improvement in methodology. If I roll a d20 and get a 7 and you roll it and get 13 that doesn't make the prior study inaccurate. The sizes are too low or not properly defined. But the combination of publish-or-perish and grant seeking pushing sensationalism isn't good for honesty let alone accuracy or precision.
- nathan_compton 5y ago"A good scientist, in other words, does not merely ignore conventional wisdom, but makes a special effort to break it. Scientists go looking for trouble. This should be the m.o. of any scholar, but scientists seem much more willing to look under rocks." -- Paul Graham.
- genousti 5y ago
- diffeomorphism 5y ago> Meaning that... No, that does not follow at all. But it seems very appropriate that you quote one statistic and then make a claim about a completely different one. Good exercise/example for the reader.
- tboyd47 5y agoJust like epidemiology, right?
- peterbonney 5y agoDoes epidemiology have a well-publicized replication crisis with numerous examples of outright fraud from high-profile researchers? If so then yes. If not then why don’t you just say what you’re trying to say rather than imply it obliquely in the form of a question?
- rackjack 5y agoThey need to imply it obliquely because their arguments tend to fall apart under scrutiny.
- tboyd47 5y agoMy “argument,” which is really more of an observation since we’re not debating anything, is clear. Epidemiology, as a social science, should not escape the same skepticism we approach the other social sciences with. Perhaps my comment is slightly off-topic, but I just like to leave a friendly reminder since I don’t find epidemiological conclusions under enough scrutiny in public fora these days.
- ibeckermayer 5y agoToo easy https://www.science.org/content/article/two-elite-medical-journals-retract-coronavirus-papers-over-data-integrity-questions https://www.science.org/content/article/two-elite-medical-jo... https://covid19up.org/neil-ferguson-fear-driven-predictions/ https://covid19up.org/neil-ferguson-fear-driven-predictions/
- peterbonney 5y agoThe best you can come up with for controversies in epidemiology are from non-epidemiologists?
- syshum 5y ago
- teloli 5y agoIt’s hard to find a discipline with “science” in its name that is actually a science.
- cco 5y agoPerhaps a new corollary to the axiom that no country with "democratic" in its name is a democracy?
- cabalamat 5y agoIn general, the more that someone is trying to push the point of view that something is true, the more likely it is to be false.
- howdydoo 5y agoAnd no country with "United" in its name is actually united.
- cabalamat 5y agoFor example, all 3 of the non-English parts of the UK have separatist movements that are big enough they have MPs elected.
- Nasrudith 5y agoWell the fact they need to advocate for separation is proof of unity by default at least. It is united as government organizations, not united as a hivemind. Unity failing under the previous is say, Birmingham besieging London. The latter never happened.
- lurquer 5y ago
- brainzap 5y agothey have to sell
- maxwellb 5y agoIf you're interested, the paper says the underlying data is freely available on request.
- boxed 5y agoThey all do though. Including the outright frauds.
- rboes 5y agoThanks I always forget