Statistically, if you always dismiss novel claims you will be right almost all the time. Yet this requires no intelligence at all. It's akin to a wall followin

Statistically, if you always dismiss novel claims you will be right almost all the time. Yet this requires no intelligence at all. It's akin to a wall following msze solver. Wall following maze solvers do work, but only for the subset of mazes that are simply and directly connected end to end. You are correct that some way of sorting signal from noise is required, but I think it's got to be something up to the task. There is also clearly a breadth/depth tradeoff at work here that demands some amout of cost/benefit analysis. At one extreme you have rote dogma O(1), and at another you have full iterative search of the combinatorial space of all theory O(n^n^...). The way I think about issues like this is greatly informed by my study of learning theory, which if studied deeply infuses one with a great phobia of the dreaded local maximum and a suspicion of any and all deterministic non-adaptive learning strategies... As these tend to converge at local maxima. See: no free lunch theorem, among other things. My maze solver analogy is a pretty good simple one for how the space of learning algorithms maps to the space of fitness landscapes. The ideal strategy would probably be to run multiple heuristics concurrently or switch them up over time. For example, science might embark upon multi decade investigations of orthodoxy punctuated by jubilees where an accumulated queue of minority and fringe theories are investigated. Successes here form the next set of orthodoxies, rinse and repeat.