5 ms·

I work in pandas 95% of my day doing data automation tasks, manipulating sql queries, and data-munging for machine learning. It is literally life changing for s

I work in pandas 95% of my day doing data automation tasks, manipulating sql queries, and data-munging for machine learning. It is literally life changing for someone like me. I used to program solely in R, but after discovering pandas I really have no need to go back to R. My project workflow consists of several IPython notebooks+pandas+sklearn. Works extremely well in production, as in, on a flask web server, as well.

minimaxir 12y ago
For the particular tasks of "data automation tasks, manipulating sql queries, and data-munging for machine learning," Python is indeed better than R, especially with sklearn For other applications (especially charting and data manipulation with ggplot2 and dplyr respectively), R has an edge.
- tlmr 12y ago
  I don't think so. Have you checked out seaborn, blaze and bokeh?
  - tdaltonc 12y ago
    Not OP, but I've also been look for a way to leave ggplot2 behind and make the jump to 100% python for data analysis. These look neat. Seaborn - http://web.stanford.edu/~mwaskom/software/seaborn/ http://web.stanford.edu/~mwaskom/software/seaborn/ Blaze - http://blaze.pydata.org/docs/v_0_6_5/index.html http://blaze.pydata.org/docs/v_0_6_5/index.html Bokeh - http://bokeh.pydata.org/ http://bokeh.pydata.org/
    - hadley 12y ago
      There's also the confusingly named ggplot (for python): https://github.com/yhat/ggplot https://github.com/yhat/ggplot
      - tdaltonc 12y ago
        I've used that library, and I really don't like it at all. They tried to bring the R syntax to python, which ends up looking awful and is missing the point of the Graphical Grammar. In the same way that every language has it's own way of expressing control flow, every language should have it's own way to express the Graphical Grammar. We don't need R's GGplot2 in python, we need a pythonic way to express the Graphical Grammar. If I had stronger python-fu I would love to build "GGPy".
        tlmr 12y ago
        Isn't GGPy seaborn?
        tdaltonc 12y ago
        Maybe. The plots look a lot like GGplot2 plots, any the syntax looks like python, but I haven't dug in to it to see if it builds plots using the Graphical Grammar under the hood.
    - elliott34 12y ago
      we've shifted to plot.ly for visualization and moved away from expensive BI tools. We set up python/pandas scripts on CRON to output realtime data to our local plot.ly web server, which makes a local copy of the data and updates the chart. You simply embed the chart as an iframe where ever want internally, and BOOM, you've got a real time chart (beautiful I may add). www.plot.ly
      - jackgolding 12y ago
        $60 per user per month is a bit steep...
        elliott34 12y ago
        Considering enterprise BI software/platforms these days can costs upwards of 4k per month, this is roundoff error.
        jackgolding 12y ago
        I agree, just doesn't fit the projects I wish to use it for.
- datahipster 12y ago
  For those cases where I have to dip into R for specific functionality, I'll use Rpy2. Pandas has great support for translating Pandas data frames to R data frames!
datahipster 12y ago
You ought to take a look at https://pypi.python.org/pypi/ipython-sql https://pypi.python.org/pypi/ipython-sql. This, in conjunction with IPython magic helpers specified in your IPython profile, can turn IPython into a SQL client. Here's a good example: https://gist.github.com/slnovak/583e35083ebd42892fab https://gist.github.com/slnovak/583e35083ebd42892fab
- b_friedland 12y ago
  Thanks for this, looks like it's going to be pretty handy.
- elliott34 12y ago
  OOOOOO awesome. Thank you!

minimaxir 12y ago

For the particular tasks of "data automation tasks, manipulating sql queries, and data-munging for machine learning," Python is indeed better than R, especially with sklearn For other applications (especially charting and data manipulation with ggplot2 and dplyr respectively), R has an edge.

tlmr 12y ago
I don't think so. Have you checked out seaborn, blaze and bokeh?
- tdaltonc 12y ago
  Not OP, but I've also been look for a way to leave ggplot2 behind and make the jump to 100% python for data analysis. These look neat. Seaborn - http://web.stanford.edu/~mwaskom/software/seaborn/ http://web.stanford.edu/~mwaskom/software/seaborn/ Blaze - http://blaze.pydata.org/docs/v_0_6_5/index.html http://blaze.pydata.org/docs/v_0_6_5/index.html Bokeh - http://bokeh.pydata.org/ http://bokeh.pydata.org/
  - hadley 12y ago
    There's also the confusingly named ggplot (for python): https://github.com/yhat/ggplot https://github.com/yhat/ggplot
    - tdaltonc 12y ago
      I've used that library, and I really don't like it at all. They tried to bring the R syntax to python, which ends up looking awful and is missing the point of the Graphical Grammar. In the same way that every language has it's own way of expressing control flow, every language should have it's own way to express the Graphical Grammar. We don't need R's GGplot2 in python, we need a pythonic way to express the Graphical Grammar. If I had stronger python-fu I would love to build "GGPy".
      - tlmr 12y ago
        Isn't GGPy seaborn?
        tdaltonc 12y ago
        Maybe. The plots look a lot like GGplot2 plots, any the syntax looks like python, but I haven't dug in to it to see if it builds plots using the Graphical Grammar under the hood.
  - elliott34 12y ago
    we've shifted to plot.ly for visualization and moved away from expensive BI tools. We set up python/pandas scripts on CRON to output realtime data to our local plot.ly web server, which makes a local copy of the data and updates the chart. You simply embed the chart as an iframe where ever want internally, and BOOM, you've got a real time chart (beautiful I may add). www.plot.ly
    - jackgolding 12y ago
      $60 per user per month is a bit steep...
      - elliott34 12y ago
        Considering enterprise BI software/platforms these days can costs upwards of 4k per month, this is roundoff error.
        jackgolding 12y ago
        I agree, just doesn't fit the projects I wish to use it for.
datahipster 12y ago
For those cases where I have to dip into R for specific functionality, I'll use Rpy2. Pandas has great support for translating Pandas data frames to R data frames!

datahipster 12y ago

You ought to take a look at https://pypi.python.org/pypi/ipython-sql https://pypi.python.org/pypi/ipython-sql. This, in conjunction with IPython magic helpers specified in your IPython profile, can turn IPython into a SQL client. Here's a good example: https://gist.github.com/slnovak/583e35083ebd42892fab https://gist.github.com/slnovak/583e35083ebd42892fab

b_friedland 12y ago
Thanks for this, looks like it's going to be pretty handy.
elliott34 12y ago
OOOOOO awesome. Thank you!