3 ms·

I went to the real source [1], which provides this data: http://i.imgur.com/d46JG8Q.png http://i.imgur.com/d46JG8Q.png, and plotted it. The R(square) value for

by pcrh 13y ago

I went to the real source [1], which provides this data: http://i.imgur.com/d46JG8Q.png http://i.imgur.com/d46JG8Q.png, and plotted it. The R(square) value for the fit it 0.2. The Pearson r is 0.44 with a p-value of 0.02 i.e. potentially useful. Taking out Greece, Portugal and USA changes the R(square) to 0.002, the Pearson r to 0.05 with a p-value of 0.8, i.e without the outliers there is no correlation between the values. So, indeed, the outliers are driving the association... However, one is not permitted to cherry-pick data, so the "outliers" have to be left in. [1]Barnes, S., Bouis, R., Briard, P., Dougherty, S., and Eris, M. (2011).The GDP impact of reform: a simple simulation framework,OECD Economics Department Working Papers, no. 834, Paris, France: OECD. PDF file: http://www.oecd-ilibrary.org/docserver/download/5kgk9qjnhkmt.pdf?expires=1397838927&id=id&accname=guest&checksum=583ADE3E363128945532B8219957347D http://www.oecd-ilibrary.org/docserver/download/5kgk9qjnhkmt...

ronaldx 13y ago
>However, one is not permitted to cherry-pick data, so the "outliers" have to be left in. I respect why you think this, but I don't agree. Identifying points on the graph which don't fit with the pattern of the rest of the data is a reasonable way to identify outliers. Even if you choose not to exclude those, there's a greater problem: Correlation is not robust if it depends on 2 or 3 points being in just the right place. And, robustness is one of the things you really ought to check for if you calculate Pearson's r. If you get different results with outliers in and out, that's a problem for your results. In my opinion, any significant result here is just statistical noise. See Wikipedia for further discussion: https://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#Robustness https://en.wikipedia.org/wiki/Pearson_product-moment_correla...
- pcrh 13y ago
  I agree that the statistics on this association are pretty lousy, even with the outliers in. However, one would preferably find an specific reason to exclude outliers, rather than rely on intuition.

by pcrh 13y ago

ronaldx 13y ago

>However, one is not permitted to cherry-pick data, so the "outliers" have to be left in. I respect why you think this, but I don't agree. Identifying points on the graph which don't fit with the pattern of the rest of the data is a reasonable way to identify outliers. Even if you choose not to exclude those, there's a greater problem: Correlation is not robust if it depends on 2 or 3 points being in just the right place. And, robustness is one of the things you really ought to check for if you calculate Pearson's r. If you get different results with outliers in and out, that's a problem for your results. In my opinion, any significant result here is just statistical noise. See Wikipedia for further discussion: https://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#Robustness https://en.wikipedia.org/wiki/Pearson_product-moment_correla...

pcrh 13y ago
I agree that the statistics on this association are pretty lousy, even with the outliers in. However, one would preferably find an specific reason to exclude outliers, rather than rely on intuition.