4 ms·
I've put substantial amounts of R code into production - it's a nightmare. Both for development and operationally. I think 2-3 years ago R was still a superior
by rm999 10y ago
I've put substantial amounts of R code into production - it's a nightmare. Both for development and operationally. I think 2-3 years ago R was still a superior language for ML/data science dev work. But Python's library support has really caught up is now mature. The policy I put in place on my current team is to minimize R in production, with Python and Scala preferred. R in some cases still has the best machine learning libraries, which is really the only reason I've found to use it in the production stack. Even then, I prefer to just keep it at a few lines of R code (load the data, build the model, handle errors, export the model).
For analyzing data I love R and almost always prefer it to Python.
- eric_bullington 10y ago> R in some cases still has the best machine learning librari Would you mind naming some specific machine learning techniques that are still better in R? I've been studying machine learning and linear algebra the past few months, and I'd love to have a try at implementing one myself in Python, as a learning exercise.
- apathy 10y agonot the OP, but glmnet is still the standard for lasso'ing, and I'm not aware of a Python implementation for post-selection inference and statistical testing yet. That said, the secret sauce in all of those is FORTRAN.
- eric_bullington 10y agoThanks for the suggestions, I appreciate it. Alas, at the least, there are Python implementations of all those between scikit-learn and this project: https://github.com/selective-inference/Python-software https://github.com/selective-inference/Python-software. statsmodels has GLM capabilities, and there are even Python bindings for R's glmnet FORTRAN library. I'm not sure how well those compare to the R implementations, but they look well-built at first glance. Any other ideas out there?
- blahi 10y agoPython does not lack in models. It has broad enough coverage and good enough infrastructure to construct the rest if you need it. It lacks in everything else that you need in your day-to-day modeling. DoE, graphics, utility functions, inference, etc.
- apathy 10y agoInteresting! Great find on the selective inference, I should have known to look. Tucking this away for when it is needed.
- hcarvalhoalves 10y agoGlmnet and Cox proportional hazards regression (survival analysis) are two recent examples I came across missing Python implementations.
- deleted 10y ago[deleted]
- eric_bullington 10y agoDid you look in statsmodels? I appreciate the suggestions, and for a moment I was hopeful about the need for survival analysis models, but it looks like both that and GLM are well-covered in the latest version of statsmodels (don't be misled by the old sourceforge site, there's been a huge flurry of recent activity in statsmodels, hundreds of new PRs merged, look at Github and the docs site linked from that repo: http://www.statsmodels.org/stable/ http://www.statsmodels.org/stable/). There's even a Jupyter notebook comparing the R, Stata (that takes me back, used Stata in survival analysis class 10 years ago), and Python versions of proportional hazards regression: http://nbviewer.jupyter.org/urls/umich.box.com/shared/static/epie6pcdk1rgb10zcd5v.ipynb http://nbviewer.jupyter.org/urls/umich.box.com/shared/static...
- hcarvalhoalves 10y agoLast time we tried Cox PH regression from statsmodel it gave a (very) different result than R's, and we weren't comfortable with the lack of tests. Will give another try.
- rm999 10y agoGlmnet has quite a bit of functionality that is lacking in the Python elastic net implementations. The most notable is the regularization parameter sequence grid search (alpha in statsmodel, lambda in glmnet) which works remarkably well and can be orders of magnitude faster than a traditional grid search.