3 ms·
So I was doing a short code review of the R scripts provided by The Economist for their article on productivity comparisons between Western Europe and America.
by weatherlight 3y ago
So I was doing a short code review of the R scripts provided by The Economist for their article on productivity comparisons between Western Europe and America. I found a few interesting quirks that could potentially skew the data or its interpretation.
The code carries forward missing values for specific years from previous years. While this is a common practice, it could introduce bias if the missing data isn't random. Then there's the selective filtering of countries based on conditions like having a GDP over hours worked greater than 5000. This could introduce a selection bias that favors certain countries over others.
something else i noticed is the use of population-weighted averages. While it's a standard practice, it gives more weight to countries with larger populations, which could skew the results. The code also adjusts GDP for various factors like working-age population and employment rate. These adjustments are useful for a nuanced understanding but introduce assumptions that could affect the results if not validated.
The code calculates an index difference to the United States for each year and type of data, essentially setting the U.S. as a baseline for comparison. This could be problematic if the U.S. itself has outliers or skewed data for those years. Also, there's no normalization or standardization oof the data before making these comparisons, which could be an issue if the distributions of the variables being compared are different.
some countries like Ireland are explicitly excluded from the analysis, and the rationale for this isn't provided, which could be a point of concern.
- weatherlight 3y agoSomething else to note. The code merges data from WDI and OECD. If these sources have different methodologies for collecting or reporting data, it could introduce inconsistencies.
- pcrh 3y agoIreland would be excluded because GDP estimations for Ireland vastly over-estimate wealth due to many corporations being resident in Ireland for tax purposes. Otherwise, I have long suspected that using the US/US dollar as a baseline results in some skewing of the data. For example, GDP per capita PPP comparisons would place Mississippi as wealthier than the UK (as mentioned in the OP). However human development indices paint a very different picture, and would place Mississippi on par with Eastern Europe [0]. [0] https://i.imgur.com/yPkHUNt.jpg https://i.imgur.com/yPkHUNt.jpg Source: https://vividmaps.com/human-development-index-us-vs-europe/ https://vividmaps.com/human-development-index-us-vs-europe/