3 ms·

Setting the bar at seven sigma just means natural sciences are more biased towards our mathematical preconceptions about the world than accepting physical evide

Setting the bar at seven sigma just means natural sciences are more biased towards our mathematical preconceptions about the world than accepting physical evidence. There's nothing inherently wrong with that - the same could be done in social science. It just means the null hypothesis (the accepted preconception) will be sustained more often. And more type-II errors will be made. A six sigma result means there is a one in a million chance of getting the result we observed from our experiment if our maths-based theory is correct. And what you are saying is that such a result is just "meh, our math still checks out".

adamsmith143 4y ago
>Setting the bar at seven sigma just means natural sciences are more biased towards our mathematical preconceptions about the world than accepting physical evidence. You used a lot of words despite not saying anything. > And more type-II errors will be made. A six sigma result means there is a one in a million chance of getting the result we observed from our experiment if our maths-based theory is correct. It means nothing of the sort. It means that if the theoretical explanation is wrong there is only a 1 in a million chance that the pattern seen in the data is due to random chance or noise.
- subroutine 4y ago
  To clarify, instead of "is correct" I should have said "if our math based theory is being treated as the null". But yes, I completely agree with your statement. As for you not understanding the first bit, that might be a personal limitation, or some sort of aphasia. Setting the bar at 7 sigma means we will only reject the null (our math based theory) if the observed experimental data would have only occurred by chance, under the null, with a probability of less than one in ten million. Hence, setting such a high bar to reject the math based theory means we are relatively biased towards preserving our mathematical preconception about the world rather than accepting (unexpected) physical evidence. If we set the bar at say, three sigma, the bias shifts towards accepting the physical evidence. The point is, the p alpha criterion is not what makes an experiment rigorous.
  - adamsmith143 4y ago
    >Hence, setting such a high bar to reject the math based theory means we are relatively biased towards preserving our mathematical preconception You mean the only causal and verifiable and by far the most useful view about the world? Do you think the microchip in your Phone that you post from was built off intuition and gut instinct? >rather than accepting (unexpected) physical evidence. If we set the bar at say, three sigma, the bias shifts towards accepting the physical evidence. So the petabytes of experimental data from the LHC don't qualify as physical evidence in your bizarro world. Got it. This reads like the take that an overly impressed with themself 1st year Psych major would have after taking an experimental methods course.
    - deleted 4y ago
      [deleted]
    - subroutine 4y ago
      I am making a single claim, that shouldn't be this controversial. Not sure why you are angrily misrepresenting what I've stated. Here it is again, more generically: as the p-alpha level for rejecting the null hypothesis decreases, the tendency for Type-2 errors increases. Do you disagree? In a world where time and resources are not a factor, an infinite amount of data could be collected to ensure both types of error are minimized. In such a world, alpha can be made arbitrarily small. In worlds that have limited resources to dedicate to experiments, a judgement needs to be made as to whether we are more comfortable with Type-1 or Type-2 errors. If for a given experiment it is decided to set an extremely low alpha (e.g. 7-sigma) it is being implied that we would rather preserve the prevailing theory (null) than reject it in favor of the experimental data. If a social science experiment was funded at the level of CERN/LHB I have no doubt they would choose a more restrictive alpha, and likewise if LHB had less resources they would likely drop their alpha accordingly based on some power analysis. I say this as someone with a phd in a hard science. Obviously it doesn't excuse the field completely for their reproducibility issues. They should make a more concerted effort with their limited funds, perhaps using the model of particle physicists who pool funds to make a fewer number of experiments more rigorous through higher amount of data collection. But this is all aside the small point I originally intended to make..

adamsmith143 4y ago

>Setting the bar at seven sigma just means natural sciences are more biased towards our mathematical preconceptions about the world than accepting physical evidence. You used a lot of words despite not saying anything. > And more type-II errors will be made. A six sigma result means there is a one in a million chance of getting the result we observed from our experiment if our maths-based theory is correct. It means nothing of the sort. It means that if the theoretical explanation is wrong there is only a 1 in a million chance that the pattern seen in the data is due to random chance or noise.

subroutine 4y ago
To clarify, instead of "is correct" I should have said "if our math based theory is being treated as the null". But yes, I completely agree with your statement. As for you not understanding the first bit, that might be a personal limitation, or some sort of aphasia. Setting the bar at 7 sigma means we will only reject the null (our math based theory) if the observed experimental data would have only occurred by chance, under the null, with a probability of less than one in ten million. Hence, setting such a high bar to reject the math based theory means we are relatively biased towards preserving our mathematical preconception about the world rather than accepting (unexpected) physical evidence. If we set the bar at say, three sigma, the bias shifts towards accepting the physical evidence. The point is, the p alpha criterion is not what makes an experiment rigorous.
- adamsmith143 4y ago
  >Hence, setting such a high bar to reject the math based theory means we are relatively biased towards preserving our mathematical preconception You mean the only causal and verifiable and by far the most useful view about the world? Do you think the microchip in your Phone that you post from was built off intuition and gut instinct? >rather than accepting (unexpected) physical evidence. If we set the bar at say, three sigma, the bias shifts towards accepting the physical evidence. So the petabytes of experimental data from the LHC don't qualify as physical evidence in your bizarro world. Got it. This reads like the take that an overly impressed with themself 1st year Psych major would have after taking an experimental methods course.
  - deleted 4y ago
    [deleted]
  - subroutine 4y ago
    I am making a single claim, that shouldn't be this controversial. Not sure why you are angrily misrepresenting what I've stated. Here it is again, more generically: as the p-alpha level for rejecting the null hypothesis decreases, the tendency for Type-2 errors increases. Do you disagree? In a world where time and resources are not a factor, an infinite amount of data could be collected to ensure both types of error are minimized. In such a world, alpha can be made arbitrarily small. In worlds that have limited resources to dedicate to experiments, a judgement needs to be made as to whether we are more comfortable with Type-1 or Type-2 errors. If for a given experiment it is decided to set an extremely low alpha (e.g. 7-sigma) it is being implied that we would rather preserve the prevailing theory (null) than reject it in favor of the experimental data. If a social science experiment was funded at the level of CERN/LHB I have no doubt they would choose a more restrictive alpha, and likewise if LHB had less resources they would likely drop their alpha accordingly based on some power analysis. I say this as someone with a phd in a hard science. Obviously it doesn't excuse the field completely for their reproducibility issues. They should make a more concerted effort with their limited funds, perhaps using the model of particle physicists who pool funds to make a fewer number of experiments more rigorous through higher amount of data collection. But this is all aside the small point I originally intended to make..