3 ms·
Well it's worse than that. It's surprisingly common to read a paper and discover the abstract doesn't match the claims in the body of the paper itself, or that
by mike_hearn 3y ago
Well it's worse than that. It's surprisingly common to read a paper and discover the abstract doesn't match the claims in the body of the paper itself, or that the claims don't match the data in the tables. And of course press releases about a paper sometimes don't match the paper abstract, etc.
The amount of slippage in what's being claimed as papers get summarized is far too large in some fields and a major contributor to distrust in science, as it causes authors to try and have cake+eat it. Get challenged on an untrue statement you made to the public? Refer to the body of the paper where the claim isn't made, or is made equivocally, and then blame journalists for "mischaracterizing" your work.
It probably is possible to pick up signs of pseudo-science automatically with ML, but you'd want to use really big GPT-4 style LLMs with large context windows and detailed instructions. We're not quite there yet, but maybe next year.