9 ms·

“Is there a heuristic we might use to identify and flag questionable papers?”

99 points by pyinstallwoes 4y ago

im3w1l 4y ago
It seems like a repeat of seo.
- highwaylights 4y ago
  Standardised Testing Optimisation Yes I realise this is an assignment and not an exam but Scheduled Assignment Optimisation doesn't really fit. Grumble.
amelius 4y ago
“When a measure becomes a target, it ceases to be a good measure” Hence, I'm afraid peer-review is still the only option left.
- DoreenMichele 4y ago
  Peer review has issues too at the moment. From what I've read, it worked when the world was a smaller place, basically. Now, peer review is another broken cog in a machine that grew organically out of past practices and which no one really intended and no one knows how to fix.
  - probably_wrong 4y ago
    As someone who does peer review for 2-3 conferences a year, I will say that peer review is still better than any alternative proposed so far. In my circles (Computational Linguistics), getting your paper approved means that you convinced at least three PhD students with a published paper (or higher, all the way up to Professor) that your paper is good PLUS the area chair(s) considered it worth publishing. All of this without knowing who you are nor what your institution is. And once your paper is accepted it is freely available on the internet forever. No human activity is free of issues, but to me double-blind peer review is a fairly effective system. And every year the system is tweaked to try and adapt it to newer developments such as the rise of ArXiv. Most criticism I read uses the term "peer review" to refer to whatever Nature, Science, and/or Elsevier are doing at the moment, but just because they suck it doesn't mean that peer-review as a whole does.
    - DoreenMichele 4y ago
      No, no human activity is free of issues. But historically when the world was smaller, people writing papers were more likely to "speak the same language," have experiences in common that fostered clear communication, etc. It's a big, wide world. The sheer variety of experiences these days introduces complications never before seen.
    - Schroedingersat 4y ago
      Reproduction by n sources with n-k unaffiliated sources would be much better, but that requires a lot more resources for research that could otherwise go to yachts or the latest ponzi scheme.
    - im3w1l 4y ago
      > getting your paper approved means that you convinced at least three PhD students with a published paper This is a recursive definition of scientist "A scientist(t=n) is someone approved by scientists(t=n-1)". If you have a good initial condition maybe it can work for a while, but it seems that time may be coming to an end. > Pournelle's Iron Law of Bureaucracy states that in any bureaucratic organization there will be two kinds of people: First, there will be those who are devoted to the goals of the organization. Examples are dedicated classroom teachers in an educational bureaucracy, many of the engineers and launch technicians and scientists at NASA, even some agricultural scientists and advisors in the former Soviet Union collective farming administration. Secondly, there will be those dedicated to the organization itself. Examples are many of the administrators in the education system, many professors of education, many teachers union officials, much of the NASA headquarters staff, etc. The Iron Law states that in every case the second group will gain and keep control of the organization. It will write the rules, and control promotions within the organization.
      - AshamedCaptain 4y ago
        > This is a recursive definition of scientist "A scientist(t=n) is someone approved by scientists(t=n-1)". If you have a good initial condition maybe it can work for a while, but it seems that time may be coming to an end. There is no formal definition for a "scientist" and likely will never be. Definitely a scientist is not "one who has a paper accepted". Here the commenter is just claiming that in order to publish a paper in a specific journal you do need be accepted by a handful of people who have previously been accepted in the same journal. Barring the obvious chicken and egg problem which is easily solved, it does not seem to be like something which "is coming to an end".
- seydor 4y ago
  Peer review is OK but peer review is not curation. Everyone wants to get published , makes up a hypothesis that barely passes statistical significance, and gets published. Make up a hypothesis in your mind about anything , there is very high chance you will find a paper that supports it (and very low chance to find a paper that refutes it). This is like postmodernism: every possible idea is deemed to have equal weight, which leads to 1=2 . Sometimes it seems it 's more productive for a scientist to become a monk, stop communicataing and start working alone in her lab undistracted by the spam notifications of published science.
Havoc 4y ago
Sounds a lot like what FB is planning with flagging questionable references on wiki
abstractalgo 4y ago
For manual flagging, you can use Unfold Research to leave a review on the paper itself for anyone to see it, and you can add review tags and a description about what you think is wrong with it: https://twitter.com/UnfoldResearch/status/1560995366492848129 https://twitter.com/UnfoldResearch/status/156099536649284812... Tools like scite (https://scite.ai/ https://scite.ai/) do automatic processing (NLP) of other papers and determine whether a reference relationships supports or rejects the paper. Also, http://octopus.ac/ http://octopus.ac/ did NLP processing on papers to make a breakdown of them and extract pieces to populate the content on their website, but they seem to have removed the content on their Github since then.
- karencarits 4y ago
  You have also got https://pubpeer.com/ https://pubpeer.com/
- gchamonlive 4y ago
  Could manual flagging be abused by a country or agent against another country or agent in order to maliciously discredit papers from that target?
  - abstractalgo 4y ago
    Most definitely. An advantage of having open peer review system is also that this kind of attempts could be detected by tools doing automatic processing of scientific knowledge graphs generated by peer review activity. This, coupled with verification of users, should be able to minimize the risk quite significantly.
    - gchamonlive 4y ago
      Seems to me that this wouldn't be enough in case of state actors or large organizations. They would just have to employ real people to carry the fake review bombs.
  - bittercynic 4y ago
    Instead of a flagging system ranking the credibility of papers once for everyone, can we have a system where each user decides which other users they trust to evaluate papers, and ignore any signal from untrusted users? For example, I could L1 trust people to evaluate papers and the trustworthiness of other users, and L2 trust others only to evaluate the trustworthiness of papers. If I notice a spurious signal getting in, I can find out who it came from and prune my list of trusted users. I imagine I'm not the first to think of this. Anyone know of reading material on the topic?
nonrandomstring 4y ago
> statmodeling.stat.columbia.edu... Checking if the site connection is secure. Enable JavaScript and cookies to continue. statmodeling.stat.columbia.edu needs to review the security of your connection before proceeding. That's questionable. But I am not allowed to value my privacy and security and ask questions of Columbia.
323 4y ago
Instead of just accusing that journal of being a paper mill and summarily blocking it, we look for face saving way of filtering papers. Because in science it's totally forbidden to accuse someone of deliberate fraud. You always must find a face saving way for the authors when dismissing their paper, like "bad statistics are being used".
avsteele 4y ago
For non-experimental, non hard science, and based on the replicability studies of the last few years, and the things we have already known for a long time: I feel pretty comfortable with a prior probability of a paper's result being true positive at ~50%. For experimental, hard science, papers I still have a high belief, maybe 90%. After this I use the following heuristics: - => Lower belief + => higher belief effects multiply, of course, but each is few %. ++ => sufficient details present that I think I can attempt to replicate without contacting authors (this is rare) + => code used is available in public repository (subset of above) + => published in med-to-high ranking journal --- => published in obscure journal -- => socially desirable outcome +++ => socially undesirable outcome -- => non-US, non-EU university or nat. lab + => national lab I mostly read papers in my field (exp. physics), of course. But I began reading a bit more in medical and psyche a few years ago. My views are influenced by books like "Medical Nihilism", and "Science Fictions". Reading COVID papers was also caused me to revise my prior-probabilities downward.
- pyinstallwoes 4y ago
  In a world where belief can be increasingly influenced in undetected ways due to sophisticated information targeting allowing any individual or demographic to be explicitly targeted and optimized against in a relentless attempt to modify behavior, how does one form belief that is defensible?
  - Bakary 4y ago
    There are a few rules of thumb you can apply: - Only a tiny minority of your beliefs have any impact on the outcome of your life. Perhaps your concept of [insert social or political policy] or [scientific topic] or [cultural idea] might have been gamed by someone at some point, but ultimately it will rarely matter whether you even hear of these concepts to begin with or not. - Of the beliefs that remain, these often become pretty defensible through practice and results. Put another way, there are plenty of barely-literate people who still have lead fascinating and worthwhile lives even in the modern era.
- exmadscientist 4y ago
polivier 4y ago
A good candidate could be Benford's law [1], which makes predictions about the distribution of digits. If the digits of the numbers found in the results section of the paper deviate too much from this law, it could be a red flag. [1] https://en.wikipedia.org/wiki/Benford%27s_law https://en.wikipedia.org/wiki/Benford%27s_law
- anonymous_sorry 4y ago
  Benford's "law" doesn't apply to everything, so it would depend a lot on the field and phenomenon being studied. And to be honest, once it was known to be used in this way, it would be simple to make fake data comply with the law.
  - bell-cot 4y ago
    True. But the "once it was known...make fake data to comply" issue applies to ~every fake-detecting method which could be applied. Big picture, the goal is not to catch every fake. It's to (1) catch enough so that smarter fakers learn to submit their stuff elsewhere (vs. to the journal, conference, or institution in question, which really cares about fake detection). And (2) dimmer fakers are mostly caught ~promptly.
- duskwuff 4y ago
  It looks more like the papers in question are taking a large public dataset and performing some statistical analysis on it, often along the lines of showing a correlation between metrics which represents pollution (like CO2 emissions) and metrics which represent economic growth. The source data is real, and the analysis is probably real too; neither is likely to trip Benford's law. The problem is that there's nothing novel being uncovered by these articles -- it will surprise no one, for example, to hear that China's CO2 emissions have risen in conjunction with their energy usage and economic growth (https://doi.org/10.1007/s11356-020-12217-6 https://doi.org/10.1007/s11356-020-12217-6). Substituting China for Pakistan, or energy usage for foreign remittances, doesn't make for novel research; it's simply plugging in another set of numbers and turning the crank.
pyinstallwoes 4y ago
It seems that the medium of writing will inevitably reach a point of too much information to process, malicious or not; while simultaneously increasing the risk of silos. What does a post-writing civilization look like? Is it reasonable to imagine scientific progress without the burden of having to battle the computational capability of sufficiently motivated actors who can easily manipulate authored information spaces? Writing is essentially a massive attack vector to the collective human consciousness prone to Sybil attacks at an alarming saturation point in a post gpt3 society. It feels like we need something post-writing.
- closedloop129 4y ago
  You still have the social graph of the authors. Professors are the gatekeepers of the next generation. Like venture capital, you will have to hustle your way to the top for an introduction if you are an outsider.
  - pyinstallwoes 4y ago
    The social graph of the authors is precisely what computational superiority is able to manipulate due to the computational resources to create false reputation graphs that are indistinguishable from someone’s bias for base truth.
    - closedloop129 4y ago
      How is that possible? You start with a group of people whom you trust. Then you check papers where those people are co-authors. You add everybody to your group who is on an acceptable paper. On the other hand, you degrade or even remove those who are on a bad paper. Computational resources are useless because you cannot fake papers with trusted authors. Trusted authors won't publish a bad paper because they want to keep their reputation. All an adversary can have is something like a network of fake articles. Unlike web pages, scientific papers are not opinions. The scientific core group can check if papers are relevant and true. The scientists can limit those checks to authors whom they have met in person to minimize their time investment.
ilamont 4y ago
A senior editor at a top-tier medical journal told me >5 years ago that they have hired staff dedicated to scrutinizing papers from two countries that have poor reputations when it comes to submissions. It's not just prestige and the career boost; in some countries there are monetary incentives associated with publishing in the top journals.
- aaaaaaaaaaab 4y ago
  Which two countries? India and China?
  - wrp 4y ago
    Funny story...A few years ago, my online reference hunting turned up a paper from China with three Chinese authors, and also exactly the same paper (down to the formatting and footnotes) from three Indian authors in India.
    - Bakary 4y ago
      It's entirely possible that these were both copies of another paper in another country.
    - still_grokking 4y ago
      Plot twist: That were the same three people, but the paper was actually written by someone else.
  - belter 4y ago
    "China tops U.S. in quantity and quality of scientific papers" - https://archive.ph/bemMy https://archive.ph/bemMy
    - deleted 4y ago
      [deleted]
    - overboard2 4y ago
      The article measures the quality of papers produced by a country by the percentage of the world's top 1% cited papers they produced. This seems like it could be gamed.
RcouF1uZ4gsC 4y ago
> The papers come from dozens of different researchers primarily not in North America or Europe, That seems to be the actual heuristic, but it would not fly politically.
- probably_wrong 4y ago
  This would be a terrible heuristic, as it would reward western bad science (it exists, it just looks differently) and punish great scientists who just happen to be born in the "wrong" place. Paradoxically enough, it would be the opposite of science.
thatjoeoverthr 4y ago
A professor I’m familiar with mentioned the use of homoglyphs to dodge anti plagiarism detection. Maybe add that to the checklist.
wrp 4y ago
The correct answer is given by Andrew Gelman right at the top: "Unfortunately, no, I don’t know of any heuristic, beyond using tools such as GRIM to check for consistency of reported numerical results." The question is essentially whether it is possible to have a structural procedure to evaluate the semantic validity of a statement.
schiptsovgithub 4y ago
Basically, everything with is based on any statistical inference. Statistics applied to not fully observable stable systems in the modern day alchemy. Simulations have similar flaws - once the model does not match reality perfectly one would get an arbitrary wrong results. Look at all the COVID related simulations and statistical models - all of them are not even close to reality.
a_bonobo 4y ago
Why are these questionable papers? Academics are forced to publish as many papers as possible. The system pushes their own to go for the smallest publishable unit, people will find a way to optimise for that, and who are we to blame them for that? They're underpaid, overworked researchers trying to guess what the opaque and brutal funding system wants from them. Is there outright fraud, or are they just dull/pointless papers? if first, yeah that's questionable, but a lot of these example papers look more like they're dull, rather than committing fraud. We should rather tear down the awful system instead of playing whack-a-mole with academia's newest outcropping.
- jeltz 4y ago
  There is outright fraud but I think most are just dull and pointless.
- PragmaticPulp 4y ago
  > We should rather tear down the awful system instead of playing whack-a-mole with academia's newest outcropping. There isn’t one “system” that can be torn down and replaced. A lot of these papers are coming from groups and institutions in foreign countries that are basically trying to role-play as researchers so they can blend in with actual researchers and try to reap the reputation and career rewards. It’s not really centralized. This is akin to suggesting that we need to tear down the entire “e-mail system” because spammers are abusing it to send spam appearing as legitimate email. Why is it a problem? Because these contribute nothing but noise and confusion to the field and make it more difficult for actual researchers to sort through real research. There is no reason to forgive these faux-researchers for trying to extract money and prestige from a system without making actual contributions. The idea to “tear down the system” doesn’t even make sense unless you propose something different. Are you suggesting we stop rewarding anyone in any way for research? That would remove the incentives, but now we have a problem that research isn’t rewarded at all and real researchers won’t do it because they need to eat.
- hammock 4y ago
  Is there also danger that any flagging system could be abused, with the end of censorship of problematic viewpoints, rather than merely bad science?
freddealmeida 4y ago
Paid for by US government?
seydor 4y ago
indeed there is no such heuristic. The question is deeper, it's about how to trust that scientists from a culture that is perceived to be alien are living up to familiar scientific standards. It's easier to identify fraud when it comes in a systematic way from a specific region of the world. However there is a lot of fraud and scientific-freeriding coming from western scientists too, it's just sporadic and not visible among the thousands of high-quality studies. It's also wrong to call a paper 'questionable' without qualifier. Questionable can mean anything, and some of the most groundbreaking ideas were questionable when they were put in the wild. I suppose the author means technically flawed.
- peteradio 4y ago
  I wonder are there tools to visualize a particular area of interest. Could be used to identify duplicate results (not a bad thing), useless results, or contradicting results (possible fraud, error or even ground breaking). When I read too deeply in almost any area I lose the forest for the trees, authors are great at rigorously describing in english what they are doing but it overflows my stack. I wish there was a way to map it semi-permanently so I could say aha yes this result looks like this on my map of the domain. Does such a general tool exist?
enriquto 4y ago
H1. If it's written in word, not good. H2. If there's no git repo with the complete code to reproduce all experiments, then you can assume it's bullshit.
- tomrod 4y ago
  Not a good heuristic, Word processors are analogous to GIMP in the photo editing world. It's what some people just know, even if Photoshop or Markdown/Pandoc/LaTeX math are just around the corner.
  - enriquto 4y ago
    In math and computer science, papers witten in word are a definite mark of crackpotery. I agree it's arbitrary and nonsensical, but clearly a good heuristic.
- progrus 4y ago
  Yup. “We did some experiments using models, here are the outputs” doesn’t cut it. I don’t trust you.
deleted 4y ago
[deleted]
psyc 4y ago
Just ask yourself if you dislike the conclusions or implications. If so, the paper isn't credible. I was under the impression we knew this method for a pretty long damn time by now.
- kQq9oHeAz6wLLS 4y ago
  Unfortunately, the reverse will also be true: if you like the conclusions or implications, it must be credible. This is confirmation bias, and has been used for terrible reasons, as well as to great comedic effect (see Sea Matheson fat studies)
bannedbybros 4y ago
[dead]
Overtonwindow 4y ago
Really the idea of publish or perish needs to be deemphasized. Someone in the Chinese government got the idea that publishing scientific papers is paramount, so Chinese scientists pump out papers, many having little or no use. I fear western researchers are in the same boat: Publish. Doesn't matter how good it is, how useful it is, just publish. If there is a heuristic it's quantity. If a laboratory, person, or country is producing an inordinate amount of papers then perhaps it's a good idea to sample and test some of those.
aabaker99 4y ago
Mentioned in TFA, the GRIM test for checking a paper’s reported mean of integers: “”” The GRIM test is straightforward to perform. For each reported mean in a paper, the sample size (N) is found, and all fractions with denominator N are calculated. The mean is then checked against this list (being aware of the fact that values may be rounded inconsistently: depending on the context, a mean of 1.125 may be reported as 1.12 or 1.13). If the mean is not in this list, it is highlighted as mathematically impossible.[2][3] “”” Source: Wikipedia
- jstanley 4y ago
  Good idea (for integer-valued measurements), but you can do it without having to bother calculating all fractions with denominator N. Let's say the reported mean is called m and the sample size is N. Multiply m and N together. Ideally mN should be almost an exact integer. Find ceil(mN)/N and floor(mN)/N and see if they're the same as the reported m, within precision limits. Example: m=6.24, N=13 mN = 81.12 (exact) 81/13 = 6.231 82/13 = 6.308 So there is no integer numerator (for denominator 13) that gives 6.24, to 2 decimal places, so 6.24 is a mistake or a lie.
ajuc 4y ago
Scientists are evaluated by how much they publish and how many times their papers are cited. There's no incentive to call others' scientists bullshit. The ranking should assign the score of a paper that was debunked to the paper that debunked it - that would provide incentive to replicating famous sketchy-looking papers.
deleted 4y ago
[deleted]
itsthemmrvax 4y ago
non-e-moose 4y ago
Count the number of "authors". Anything above 100 is absolutely garbage More then 10 is probably irrelevant and unproveable.
- pinko 4y ago
  World-class papers with >1000 authors are routinely published by large scientific collaborations like CMS, ATLAS, and LIGO.
  - exmadscientist 4y ago
    Very true, but if you consider things like "The ATLAS Collaboration" to be one (very composite) author, I think GP's reasoning has some merit. There is a lot of homogeneity in a collaboration author like that, so it's a reasonable thing to do. (Also, speaking from experience, most people in the collaboration paid no attention to the paper, unless it was a Big One.)
powera 4y ago
The problem is an academic system that rewards "publishing papers" rather than its actual goal, "academic progress".
xbar 4y ago
Title fix: "an heuristic" or possibly match the article by saying "any heuristic"
- kbelder 4y ago
  "a heuristic" would be more proper in an American paper, not "an".
GuB-42 4y ago
The most obvious heuristic is to look at the reputation of the journal that publishes it. That's the reason we still have these overpriced journals. The "publishing" part is easy now, it is just a matter of putting a pdf on a server, journals are selling reputation.
- brutusborn 4y ago
  Do you think there is a way to get papers reviewed to the same quality without a journal? If we are paying for reputation and the journals reputation is a function of the reviewers skill, maybe there is a way to cut out the middleman and pay for skillful reviewers instead.
nottorp 4y ago
Probably yes, but it will work as well as Google's "AI" support.

99 points by pyinstallwoes 4y ago

im3w1l 4y ago

It seems like a repeat of seo.

highwaylights 4y ago
Standardised Testing Optimisation Yes I realise this is an assignment and not an exam but Scheduled Assignment Optimisation doesn't really fit. Grumble.

amelius 4y ago

“When a measure becomes a target, it ceases to be a good measure” Hence, I'm afraid peer-review is still the only option left.

DoreenMichele 4y ago
Peer review has issues too at the moment. From what I've read, it worked when the world was a smaller place, basically. Now, peer review is another broken cog in a machine that grew organically out of past practices and which no one really intended and no one knows how to fix.
- probably_wrong 4y ago
  As someone who does peer review for 2-3 conferences a year, I will say that peer review is still better than any alternative proposed so far. In my circles (Computational Linguistics), getting your paper approved means that you convinced at least three PhD students with a published paper (or higher, all the way up to Professor) that your paper is good PLUS the area chair(s) considered it worth publishing. All of this without knowing who you are nor what your institution is. And once your paper is accepted it is freely available on the internet forever. No human activity is free of issues, but to me double-blind peer review is a fairly effective system. And every year the system is tweaked to try and adapt it to newer developments such as the rise of ArXiv. Most criticism I read uses the term "peer review" to refer to whatever Nature, Science, and/or Elsevier are doing at the moment, but just because they suck it doesn't mean that peer-review as a whole does.
  - DoreenMichele 4y ago
    No, no human activity is free of issues. But historically when the world was smaller, people writing papers were more likely to "speak the same language," have experiences in common that fostered clear communication, etc. It's a big, wide world. The sheer variety of experiences these days introduces complications never before seen.
  - Schroedingersat 4y ago
    Reproduction by n sources with n-k unaffiliated sources would be much better, but that requires a lot more resources for research that could otherwise go to yachts or the latest ponzi scheme.
  - im3w1l 4y ago
    > getting your paper approved means that you convinced at least three PhD students with a published paper This is a recursive definition of scientist "A scientist(t=n) is someone approved by scientists(t=n-1)". If you have a good initial condition maybe it can work for a while, but it seems that time may be coming to an end. > Pournelle's Iron Law of Bureaucracy states that in any bureaucratic organization there will be two kinds of people: First, there will be those who are devoted to the goals of the organization. Examples are dedicated classroom teachers in an educational bureaucracy, many of the engineers and launch technicians and scientists at NASA, even some agricultural scientists and advisors in the former Soviet Union collective farming administration. Secondly, there will be those dedicated to the organization itself. Examples are many of the administrators in the education system, many professors of education, many teachers union officials, much of the NASA headquarters staff, etc. The Iron Law states that in every case the second group will gain and keep control of the organization. It will write the rules, and control promotions within the organization.
    - AshamedCaptain 4y ago
      > This is a recursive definition of scientist "A scientist(t=n) is someone approved by scientists(t=n-1)". If you have a good initial condition maybe it can work for a while, but it seems that time may be coming to an end. There is no formal definition for a "scientist" and likely will never be. Definitely a scientist is not "one who has a paper accepted". Here the commenter is just claiming that in order to publish a paper in a specific journal you do need be accepted by a handful of people who have previously been accepted in the same journal. Barring the obvious chicken and egg problem which is easily solved, it does not seem to be like something which "is coming to an end".
seydor 4y ago
Peer review is OK but peer review is not curation. Everyone wants to get published , makes up a hypothesis that barely passes statistical significance, and gets published. Make up a hypothesis in your mind about anything , there is very high chance you will find a paper that supports it (and very low chance to find a paper that refutes it). This is like postmodernism: every possible idea is deemed to have equal weight, which leads to 1=2 . Sometimes it seems it 's more productive for a scientist to become a monk, stop communicataing and start working alone in her lab undistracted by the spam notifications of published science.

Havoc 4y ago

Sounds a lot like what FB is planning with flagging questionable references on wiki

abstractalgo 4y ago

For manual flagging, you can use Unfold Research to leave a review on the paper itself for anyone to see it, and you can add review tags and a description about what you think is wrong with it: https://twitter.com/UnfoldResearch/status/1560995366492848129 https://twitter.com/UnfoldResearch/status/156099536649284812... Tools like scite (https://scite.ai/ https://scite.ai/) do automatic processing (NLP) of other papers and determine whether a reference relationships supports or rejects the paper. Also, http://octopus.ac/ http://octopus.ac/ did NLP processing on papers to make a breakdown of them and extract pieces to populate the content on their website, but they seem to have removed the content on their Github since then.

karencarits 4y ago
You have also got https://pubpeer.com/ https://pubpeer.com/
gchamonlive 4y ago
Could manual flagging be abused by a country or agent against another country or agent in order to maliciously discredit papers from that target?
- abstractalgo 4y ago
  Most definitely. An advantage of having open peer review system is also that this kind of attempts could be detected by tools doing automatic processing of scientific knowledge graphs generated by peer review activity. This, coupled with verification of users, should be able to minimize the risk quite significantly.
  - gchamonlive 4y ago
    Seems to me that this wouldn't be enough in case of state actors or large organizations. They would just have to employ real people to carry the fake review bombs.
- bittercynic 4y ago
  Instead of a flagging system ranking the credibility of papers once for everyone, can we have a system where each user decides which other users they trust to evaluate papers, and ignore any signal from untrusted users? For example, I could L1 trust people to evaluate papers and the trustworthiness of other users, and L2 trust others only to evaluate the trustworthiness of papers. If I notice a spurious signal getting in, I can find out who it came from and prune my list of trusted users. I imagine I'm not the first to think of this. Anyone know of reading material on the topic?

nonrandomstring 4y ago

> statmodeling.stat.columbia.edu... Checking if the site connection is secure. Enable JavaScript and cookies to continue. statmodeling.stat.columbia.edu needs to review the security of your connection before proceeding. That's questionable. But I am not allowed to value my privacy and security and ask questions of Columbia.

323 4y ago

Instead of just accusing that journal of being a paper mill and summarily blocking it, we look for face saving way of filtering papers. Because in science it's totally forbidden to accuse someone of deliberate fraud. You always must find a face saving way for the authors when dismissing their paper, like "bad statistics are being used".

avsteele 4y ago

For non-experimental, non hard science, and based on the replicability studies of the last few years, and the things we have already known for a long time: I feel pretty comfortable with a prior probability of a paper's result being true positive at ~50%. For experimental, hard science, papers I still have a high belief, maybe 90%. After this I use the following heuristics: - => Lower belief + => higher belief effects multiply, of course, but each is few %. ++ => sufficient details present that I think I can attempt to replicate without contacting authors (this is rare) + => code used is available in public repository (subset of above) + => published in med-to-high ranking journal --- => published in obscure journal -- => socially desirable outcome +++ => socially undesirable outcome -- => non-US, non-EU university or nat. lab + => national lab I mostly read papers in my field (exp. physics), of course. But I began reading a bit more in medical and psyche a few years ago. My views are influenced by books like "Medical Nihilism", and "Science Fictions". Reading COVID papers was also caused me to revise my prior-probabilities downward.

pyinstallwoes 4y ago
In a world where belief can be increasingly influenced in undetected ways due to sophisticated information targeting allowing any individual or demographic to be explicitly targeted and optimized against in a relentless attempt to modify behavior, how does one form belief that is defensible?
- Bakary 4y ago
  There are a few rules of thumb you can apply: - Only a tiny minority of your beliefs have any impact on the outcome of your life. Perhaps your concept of [insert social or political policy] or [scientific topic] or [cultural idea] might have been gamed by someone at some point, but ultimately it will rarely matter whether you even hear of these concepts to begin with or not. - Of the beliefs that remain, these often become pretty defensible through practice and results. Put another way, there are plenty of barely-literate people who still have lead fascinating and worthwhile lives even in the modern era.
exmadscientist 4y ago

polivier 4y ago

A good candidate could be Benford's law [1], which makes predictions about the distribution of digits. If the digits of the numbers found in the results section of the paper deviate too much from this law, it could be a red flag. [1] https://en.wikipedia.org/wiki/Benford%27s_law https://en.wikipedia.org/wiki/Benford%27s_law

anonymous_sorry 4y ago
Benford's "law" doesn't apply to everything, so it would depend a lot on the field and phenomenon being studied. And to be honest, once it was known to be used in this way, it would be simple to make fake data comply with the law.
- bell-cot 4y ago
  True. But the "once it was known...make fake data to comply" issue applies to ~every fake-detecting method which could be applied. Big picture, the goal is not to catch every fake. It's to (1) catch enough so that smarter fakers learn to submit their stuff elsewhere (vs. to the journal, conference, or institution in question, which really cares about fake detection). And (2) dimmer fakers are mostly caught ~promptly.
duskwuff 4y ago
It looks more like the papers in question are taking a large public dataset and performing some statistical analysis on it, often along the lines of showing a correlation between metrics which represents pollution (like CO2 emissions) and metrics which represent economic growth. The source data is real, and the analysis is probably real too; neither is likely to trip Benford's law. The problem is that there's nothing novel being uncovered by these articles -- it will surprise no one, for example, to hear that China's CO2 emissions have risen in conjunction with their energy usage and economic growth (https://doi.org/10.1007/s11356-020-12217-6 https://doi.org/10.1007/s11356-020-12217-6). Substituting China for Pakistan, or energy usage for foreign remittances, doesn't make for novel research; it's simply plugging in another set of numbers and turning the crank.

pyinstallwoes 4y ago

It seems that the medium of writing will inevitably reach a point of too much information to process, malicious or not; while simultaneously increasing the risk of silos. What does a post-writing civilization look like? Is it reasonable to imagine scientific progress without the burden of having to battle the computational capability of sufficiently motivated actors who can easily manipulate authored information spaces? Writing is essentially a massive attack vector to the collective human consciousness prone to Sybil attacks at an alarming saturation point in a post gpt3 society. It feels like we need something post-writing.

closedloop129 4y ago
You still have the social graph of the authors. Professors are the gatekeepers of the next generation. Like venture capital, you will have to hustle your way to the top for an introduction if you are an outsider.
- pyinstallwoes 4y ago
  The social graph of the authors is precisely what computational superiority is able to manipulate due to the computational resources to create false reputation graphs that are indistinguishable from someone’s bias for base truth.
  - closedloop129 4y ago
    How is that possible? You start with a group of people whom you trust. Then you check papers where those people are co-authors. You add everybody to your group who is on an acceptable paper. On the other hand, you degrade or even remove those who are on a bad paper. Computational resources are useless because you cannot fake papers with trusted authors. Trusted authors won't publish a bad paper because they want to keep their reputation. All an adversary can have is something like a network of fake articles. Unlike web pages, scientific papers are not opinions. The scientific core group can check if papers are relevant and true. The scientists can limit those checks to authors whom they have met in person to minimize their time investment.

ilamont 4y ago

A senior editor at a top-tier medical journal told me >5 years ago that they have hired staff dedicated to scrutinizing papers from two countries that have poor reputations when it comes to submissions. It's not just prestige and the career boost; in some countries there are monetary incentives associated with publishing in the top journals.

aaaaaaaaaaab 4y ago
Which two countries? India and China?
- wrp 4y ago
  Funny story...A few years ago, my online reference hunting turned up a paper from China with three Chinese authors, and also exactly the same paper (down to the formatting and footnotes) from three Indian authors in India.
  - Bakary 4y ago
    It's entirely possible that these were both copies of another paper in another country.
  - still_grokking 4y ago
    Plot twist: That were the same three people, but the paper was actually written by someone else.
- belter 4y ago
  "China tops U.S. in quantity and quality of scientific papers" - https://archive.ph/bemMy https://archive.ph/bemMy
  - deleted 4y ago
    [deleted]
  - overboard2 4y ago
    The article measures the quality of papers produced by a country by the percentage of the world's top 1% cited papers they produced. This seems like it could be gamed.

RcouF1uZ4gsC 4y ago

> The papers come from dozens of different researchers primarily not in North America or Europe, That seems to be the actual heuristic, but it would not fly politically.

probably_wrong 4y ago
This would be a terrible heuristic, as it would reward western bad science (it exists, it just looks differently) and punish great scientists who just happen to be born in the "wrong" place. Paradoxically enough, it would be the opposite of science.

thatjoeoverthr 4y ago

A professor I’m familiar with mentioned the use of homoglyphs to dodge anti plagiarism detection. Maybe add that to the checklist.

wrp 4y ago

The correct answer is given by Andrew Gelman right at the top: "Unfortunately, no, I don’t know of any heuristic, beyond using tools such as GRIM to check for consistency of reported numerical results." The question is essentially whether it is possible to have a structural procedure to evaluate the semantic validity of a statement.

schiptsovgithub 4y ago

Basically, everything with is based on any statistical inference. Statistics applied to not fully observable stable systems in the modern day alchemy. Simulations have similar flaws - once the model does not match reality perfectly one would get an arbitrary wrong results. Look at all the COVID related simulations and statistical models - all of them are not even close to reality.

a_bonobo 4y ago

Why are these questionable papers? Academics are forced to publish as many papers as possible. The system pushes their own to go for the smallest publishable unit, people will find a way to optimise for that, and who are we to blame them for that? They're underpaid, overworked researchers trying to guess what the opaque and brutal funding system wants from them. Is there outright fraud, or are they just dull/pointless papers? if first, yeah that's questionable, but a lot of these example papers look more like they're dull, rather than committing fraud. We should rather tear down the awful system instead of playing whack-a-mole with academia's newest outcropping.

jeltz 4y ago
There is outright fraud but I think most are just dull and pointless.
PragmaticPulp 4y ago
> We should rather tear down the awful system instead of playing whack-a-mole with academia's newest outcropping. There isn’t one “system” that can be torn down and replaced. A lot of these papers are coming from groups and institutions in foreign countries that are basically trying to role-play as researchers so they can blend in with actual researchers and try to reap the reputation and career rewards. It’s not really centralized. This is akin to suggesting that we need to tear down the entire “e-mail system” because spammers are abusing it to send spam appearing as legitimate email. Why is it a problem? Because these contribute nothing but noise and confusion to the field and make it more difficult for actual researchers to sort through real research. There is no reason to forgive these faux-researchers for trying to extract money and prestige from a system without making actual contributions. The idea to “tear down the system” doesn’t even make sense unless you propose something different. Are you suggesting we stop rewarding anyone in any way for research? That would remove the incentives, but now we have a problem that research isn’t rewarded at all and real researchers won’t do it because they need to eat.
hammock 4y ago
Is there also danger that any flagging system could be abused, with the end of censorship of problematic viewpoints, rather than merely bad science?

freddealmeida 4y ago

Paid for by US government?

seydor 4y ago

indeed there is no such heuristic. The question is deeper, it's about how to trust that scientists from a culture that is perceived to be alien are living up to familiar scientific standards. It's easier to identify fraud when it comes in a systematic way from a specific region of the world. However there is a lot of fraud and scientific-freeriding coming from western scientists too, it's just sporadic and not visible among the thousands of high-quality studies. It's also wrong to call a paper 'questionable' without qualifier. Questionable can mean anything, and some of the most groundbreaking ideas were questionable when they were put in the wild. I suppose the author means technically flawed.

peteradio 4y ago
I wonder are there tools to visualize a particular area of interest. Could be used to identify duplicate results (not a bad thing), useless results, or contradicting results (possible fraud, error or even ground breaking). When I read too deeply in almost any area I lose the forest for the trees, authors are great at rigorously describing in english what they are doing but it overflows my stack. I wish there was a way to map it semi-permanently so I could say aha yes this result looks like this on my map of the domain. Does such a general tool exist?

enriquto 4y ago

H1. If it's written in word, not good. H2. If there's no git repo with the complete code to reproduce all experiments, then you can assume it's bullshit.

tomrod 4y ago
Not a good heuristic, Word processors are analogous to GIMP in the photo editing world. It's what some people just know, even if Photoshop or Markdown/Pandoc/LaTeX math are just around the corner.
- enriquto 4y ago
  In math and computer science, papers witten in word are a definite mark of crackpotery. I agree it's arbitrary and nonsensical, but clearly a good heuristic.
progrus 4y ago
Yup. “We did some experiments using models, here are the outputs” doesn’t cut it. I don’t trust you.

deleted 4y ago

[deleted]

psyc 4y ago

Just ask yourself if you dislike the conclusions or implications. If so, the paper isn't credible. I was under the impression we knew this method for a pretty long damn time by now.

kQq9oHeAz6wLLS 4y ago
Unfortunately, the reverse will also be true: if you like the conclusions or implications, it must be credible. This is confirmation bias, and has been used for terrible reasons, as well as to great comedic effect (see Sea Matheson fat studies)

bannedbybros 4y ago

[dead]

Overtonwindow 4y ago

Really the idea of publish or perish needs to be deemphasized. Someone in the Chinese government got the idea that publishing scientific papers is paramount, so Chinese scientists pump out papers, many having little or no use. I fear western researchers are in the same boat: Publish. Doesn't matter how good it is, how useful it is, just publish. If there is a heuristic it's quantity. If a laboratory, person, or country is producing an inordinate amount of papers then perhaps it's a good idea to sample and test some of those.

aabaker99 4y ago

Mentioned in TFA, the GRIM test for checking a paper’s reported mean of integers: “”” The GRIM test is straightforward to perform. For each reported mean in a paper, the sample size (N) is found, and all fractions with denominator N are calculated. The mean is then checked against this list (being aware of the fact that values may be rounded inconsistently: depending on the context, a mean of 1.125 may be reported as 1.12 or 1.13). If the mean is not in this list, it is highlighted as mathematically impossible.[2][3] “”” Source: Wikipedia

jstanley 4y ago
Good idea (for integer-valued measurements), but you can do it without having to bother calculating all fractions with denominator N. Let's say the reported mean is called m and the sample size is N. Multiply m and N together. Ideally mN should be almost an exact integer. Find ceil(mN)/N and floor(mN)/N and see if they're the same as the reported m, within precision limits. Example: m=6.24, N=13 mN = 81.12 (exact) 81/13 = 6.231 82/13 = 6.308 So there is no integer numerator (for denominator 13) that gives 6.24, to 2 decimal places, so 6.24 is a mistake or a lie.

ajuc 4y ago

Scientists are evaluated by how much they publish and how many times their papers are cited. There's no incentive to call others' scientists bullshit. The ranking should assign the score of a paper that was debunked to the paper that debunked it - that would provide incentive to replicating famous sketchy-looking papers.

deleted 4y ago

[deleted]

itsthemmrvax 4y ago

non-e-moose 4y ago

Count the number of "authors". Anything above 100 is absolutely garbage More then 10 is probably irrelevant and unproveable.

pinko 4y ago
World-class papers with >1000 authors are routinely published by large scientific collaborations like CMS, ATLAS, and LIGO.
- exmadscientist 4y ago
  Very true, but if you consider things like "The ATLAS Collaboration" to be one (very composite) author, I think GP's reasoning has some merit. There is a lot of homogeneity in a collaboration author like that, so it's a reasonable thing to do. (Also, speaking from experience, most people in the collaboration paid no attention to the paper, unless it was a Big One.)

powera 4y ago

The problem is an academic system that rewards "publishing papers" rather than its actual goal, "academic progress".

xbar 4y ago

Title fix: "an heuristic" or possibly match the article by saying "any heuristic"

kbelder 4y ago
"a heuristic" would be more proper in an American paper, not "an".

GuB-42 4y ago

The most obvious heuristic is to look at the reputation of the journal that publishes it. That's the reason we still have these overpriced journals. The "publishing" part is easy now, it is just a matter of putting a pdf on a server, journals are selling reputation.

brutusborn 4y ago
Do you think there is a way to get papers reviewed to the same quality without a journal? If we are paying for reputation and the journals reputation is a function of the reviewers skill, maybe there is a way to cut out the middleman and pay for skillful reviewers instead.

nottorp 4y ago

Probably yes, but it will work as well as Google's "AI" support.