>"extracting the t-test from the rest of analysis like that is...idiosyncratic...at best." So much nonsense about statistics has been institutionalized at this

>"extracting the t-test from the rest of analysis like that is...idiosyncratic...at best." So much nonsense about statistics has been institutionalized at this point that this sounds like a compliment. To be sure, the issue here is minor compared to the various misconceptions like "p-value is the probability my theory is wrong", etc that are dominant. >"Are you actually refusing to call these "an increase in false positives", since the z-test is working correctly, but it's been fed a database of accurate weights instead of "item weights + gunk weight")? " Yes, the hypothesis should include extra uncertainty due to gunk if that is an issue. These are true positive rejections. It is a bad hypothesis. >"How about if the item database is wrong instead (e.g., chips now come in a slightly smaller bag)?" Once again, the hypothesis was wrong. We are right to reject it. In both these cases calling it a "false positive" is misleading because it focuses the attention on something other than the source of the problem: a bad hypothesis. These are true positives. >"A MRI researcher wants to know if the BOLD signal in a cluster, once corrected for various nuisance factors, varies across conditions or not. That ("or not") is a perfectly reasonable null hypothesis." Then they should deduce a distribution of expected results directly from that hypothesis and compare the observations to that prediction. What seems to be going on is they say that is the hypothesis but then go onto test a different one. Honestly, just the fact they are using a t-test as part of a complicated process like this makes me suspect they don't know what they are doing... Maybe it makes sense somehow, but I doubt it. Disclaimer: I haven't looked into the code or anything in detail.