3 ms·

Rather than get into a catalog of tests, I take the approach that "There is only one test." I wrote more about it here: http://allendowney.blogspot.com/2011/05

Rather than get into a catalog of tests, I take the approach that "There is only one test." I wrote more about it here: http://allendowney.blogspot.com/2011/05/there-is-only-one-test.html http://allendowney.blogspot.com/2011/05/there-is-only-one-te...

Jach 15y ago
Furthermore, another reason the entire classic test-based approach is bad since it encourages having binary hypotheses when many real-world problems don't, they may be many or composite or even infinite. (Of course, many real-world problems can be reduced to binary ones, which is one reason the approach became popular.) If you know enough probability theory, statistics is just a special case. The nice thing about using probability theory is if you do decide to use a 'test', all of your assumptions are put forth first. As E.T. Jaynes says: In estimating a location parameter, for example, the sample median M is often cited as a more robust estimator than the sample mean. But here it is obvious that this ‘robustness’ is bought at the price of insensitivity to much of the relevant information in the data. Many different data sets all have the same median; the values above or below the sample median may be moved about arbitrarily without affecting the estimate. Yet those data values surely contain information highly relevant to the question being asked, and all this is lost. We would have thought that the whole purpose of data analysis is to extract all the information we can from the data. Thus, while we agree that robust/resistant properties may be desirable in some cases, we think it important to emphasize their cost in performance. In the literature, ad hoc procedures have been advocated on no more grounds than that they are ‘robust’ or ‘resistant’, with no mention of the quality of the inference they deliver, much less any comparison of performance with alternative methods; yet alternative methods such as Bayesian ones are criticized on grounds of lack of robustness, without any supporting factual evidence. A recent probability book I've started that I think is pretty good is http://uncertainty.stat.cmu.edu/ http://uncertainty.stat.cmu.edu/

Jach 15y ago

Furthermore, another reason the entire classic test-based approach is bad since it encourages having binary hypotheses when many real-world problems don't, they may be many or composite or even infinite. (Of course, many real-world problems can be reduced to binary ones, which is one reason the approach became popular.) If you know enough probability theory, statistics is just a special case. The nice thing about using probability theory is if you do decide to use a 'test', all of your assumptions are put forth first. As E.T. Jaynes says: In estimating a location parameter, for example, the sample median M is often cited as a more robust estimator than the sample mean. But here it is obvious that this ‘robustness’ is bought at the price of insensitivity to much of the relevant information in the data. Many different data sets all have the same median; the values above or below the sample median may be moved about arbitrarily without affecting the estimate. Yet those data values surely contain information highly relevant to the question being asked, and all this is lost. We would have thought that the whole purpose of data analysis is to extract all the information we can from the data. Thus, while we agree that robust/resistant properties may be desirable in some cases, we think it important to emphasize their cost in performance. In the literature, ad hoc procedures have been advocated on no more grounds than that they are ‘robust’ or ‘resistant’, with no mention of the quality of the inference they deliver, much less any comparison of performance with alternative methods; yet alternative methods such as Bayesian ones are criticized on grounds of lack of robustness, without any supporting factual evidence. A recent probability book I've started that I think is pretty good is http://uncertainty.stat.cmu.edu/ http://uncertainty.stat.cmu.edu/