4 ms·

In my experience as a researcher in physics (obviously anecdotal to my field and not generalizable across physics) all we do are variations on Stage 1 - we eith

In my experience as a researcher in physics (obviously anecdotal to my field and not generalizable across physics) all we do are variations on Stage 1 - we either have a model, a prediction and an experiment that confirms it (these go to the best journals), or we only have a model + prediction (and couldn't do the experiment) or an experiment (but not a model for it), so we report those in the hopes that someone else will fill in the missing half. Since I mostly do theoretical soft matter work, having a model with an interesting but hard to verify prediction is the common case (which is far from useless, as experimental techniques are advancing incredibly quickly). Either way, if I brought up p-values I'd be unceremoniously laughed out of the room in my department. If I wrote up a prediction that's not just verifiable in principle but verifiable with existing (albeit sometimes new, rare or hard to implement) experimental methods, I wouldn't get published, unless it was some kind of a novel theoretical technique with further applications in that regard. I'm not saying your assertion is wrong - physics is wildly different across different fields, and I don't pretend to have any meaningful overview of all of them. I can only report that I see no signs of this supposed encroachment of pseudoscientific statistics in my physics work or the literature I read professionally - I hope it stays that way.

nonbel 8y ago
I'm speaking from what I saw in the LIGO and CERN (Higg's Boson) papers. They spend way too much time on rejecting some questionable model of the background rather than testing actual predictions. From media interviews, I'll also say on the theorist side of it they seem to harbor all the usual misconceptions about p-values as seen in psych and medicine. But they do eventually fit an actual prediction to the data, like a model of black holes colliding, or whatever. However, I don't personally know how many degrees of freedom they have in these fits...
- keldaris 8y ago
  LIGO and the Higgs boson example are both extreme outliers in physics in terms of scale, how much statistics they necessarily involve, the scrutiny and publicity they get and almost every other property. I'd also note that scientists trying to oversimplify things for a media interview almost always screw up somewhere. I wouldn't draw any conclusions about physics in general from either their papers or (especially) interviews.

nonbel 8y ago

I'm speaking from what I saw in the LIGO and CERN (Higg's Boson) papers. They spend way too much time on rejecting some questionable model of the background rather than testing actual predictions. From media interviews, I'll also say on the theorist side of it they seem to harbor all the usual misconceptions about p-values as seen in psych and medicine. But they do eventually fit an actual prediction to the data, like a model of black holes colliding, or whatever. However, I don't personally know how many degrees of freedom they have in these fits...

keldaris 8y ago
LIGO and the Higgs boson example are both extreme outliers in physics in terms of scale, how much statistics they necessarily involve, the scrutiny and publicity they get and almost every other property. I'd also note that scientists trying to oversimplify things for a media interview almost always screw up somewhere. I wouldn't draw any conclusions about physics in general from either their papers or (especially) interviews.