Fantastic article from Gwern, as usual. He lays out some approaches to distinguish fake research using AI-generated ones. However I think he may be optimistic

Fantastic article from Gwern, as usual. He lays out some approaches to distinguish fake research using AI-generated ones. However I think he may be optimistic about the effort needed to correct the course on junk science and the ability of humans to discriminate between real and fake. There was a game posted on hn where you had a real title/abstract and a fake one, and you had to guess the correct one, and I actually performed worse than a coin toss. Any heuristic -e.g., this is such an unreadable string of jargon that no human would have tried to publish something like this- was actually violated by some articles. Of course, it was mostly papers outside of my expertise, like geology or biology. One would think that after having read more than a hundred papers, they would perform better. Especially as some very advanced technical points about either biology or technology drive more policy with direct impact. One mistake has always been to present "the science" as something well established. An extraordinary number of debunked claims have percolated to the general public, bad science has driven policy, the limits of knowledge and scientific reductionism can be felt in many fundamental domains (e.g. gardening, dietetics). Science would be more actually described as a faith that the process of that multi-player publishing game eventually converges to the truth. In any case, evolution does not wait on anyone, so it will be interesting to see how people and the system evolve as the fake generation improves in quality beyond reasonable means.