The thing about non-statistically-significant tests is you can't, in fact, say "it is still more likely to make you money." You can say that the B set did in fa

The thing about non-statistically-significant tests is you can't, in fact, say "it is still more likely to make you money." You can say that the B set did in fact make you more money this time, but you can't rule out that being luck of the draw. It might even have a (very small) bias AGAINST making you money, but that bias got lost in the noise. That's why it's important to discard non-statistically-significant tests (or expand the sample size if you _really_ want to know...)