3 ms·
I wouldn't call them failed tests. In science a null result mean that you didn't learn anything but in this case who cares if p is 0.6? It is still more likely
by tomjen3 16y ago
I wouldn't call them failed tests. In science a null result mean that you didn't learn anything but in this case who cares if p is 0.6? It is still more likely to make you money.
- barrkel 16y agoI think precisely the point of not being able to reject the null hypothesis is that you can't be sure it's more likely to make you more money; otherwise you could have required a lower confidence level and still rejected the null hypothesis.
- bdonlan 16y agoThe thing about non-statistically-significant tests is you can't, in fact, say "it is still more likely to make you money." You can say that the B set did in fact make you more money this time, but you can't rule out that being luck of the draw. It might even have a (very small) bias AGAINST making you money, but that bias got lost in the noise. That's why it's important to discard non-statistically-significant tests (or expand the sample size if you _really_ want to know...)