3 ms·
I'm in the same boat - working at an economics / public policy lab. Tool of choice here is stata even though a lot of what it gets used for is basic cleaning. I
by makmanalp 12y ago
I'm in the same boat - working at an economics / public policy lab. Tool of choice here is stata even though a lot of what it gets used for is basic cleaning. It has quite a bit of trouble with larger datasets, when they're larger than memory. Common statsmodels regressions go with speeds on par to stata for data sets with observations in the 10-100 millions for me. R and Python are known to a few but most folks shy away from it because they need to learn how to set it up and how to program, even though it's quite basic in terms of programming. I'm sticking to python now because of how well it works as a scraping / cleaning tool and as glue, and the trustiness of the ipython / pandas / statsmodels / matplotlib (and ggplot and seaborn and vincent etc) stack. I found RPy to be super handy to get to weird regressions and stuff that I wish I had otherwise.