7 ms·

> instead, we now use ggproto, a new OO system designed specifically for ggplot2 And this is why I gave up on R for other things than ad hoc data exploration (

by zmmmmm 11y ago

> instead, we now use ggproto, a new OO system designed specifically for ggplot2 And this is why I gave up on R for other things than ad hoc data exploration (don't get me wrong; it's killer for that). When I finally learned there were no less than four system of OO buried in base R, and still people were unhappy and inventing more, I realised that as far as well structured programming goes, R is kind of beyond help at this point. From what I can tell, my biggest disappointment with ggplot2 is not addressed - the fact that ggplot evaluates all references not found in the data frame to be plotted in the global namespace. The result is that you can't actually make reusable, composable plots in a reasonable sane way. It sends the whole illusion of composability that ggplot()'s '+' syntax suggests might be there right down the tubes. My plots were all more reusuable before I ported them to ggplot (ie, in base graphics). Apologies for the whining, and congrats to Hadley!

TheLogothete 11y ago
And what do you use that even remotely approaches the breadth and depth of tools available in R, but solves this problem?
- zmmmmm 11y ago
  I wish I had a good answer to that, but I don't. Python is taking a good shot at it. But it doesn't match R by any means.
  - kldavenport 11y ago
    seaborn, bokeh, pandas' built in methods, and plotly fill in the ggplot gap for me.
    - hunterratliff1 11y ago
      So perhaps we ought be working on building solutions that bridge the Python-R gap? Because as someone who has effectively left Python, I'd love to learn from solutions Python users find effective. After all, it's not like one language is the sole proprietor of useful approaches (;
- dagw 11y ago
  88% python + 10% calling R from python + 2% porting R code to python. edit: that being said, ggplot is still better than anything on the python side of things.
- gh02t 11y ago
  I'm not saying you're wrong, but this sounds exactly like the arguments made supporting Perl, which is an interesting comparison.
  - TheLogothete 11y ago
    It is an interesting comparison for sure, but things are different for two reasons: 1) Software and related services will grow immensely over the next decade because of the "cloud". Some of the growth will be same old, same old application logic software, but most of it will be data analysis software. 2) The language of choice for data analysis software is and will be R due to the biggest software companies (both #1 and #2, among many others) standing firmly behind it along with universities incorporating it in its curriculum, ditching SPSS and SAS.
  - xixi77 11y ago
    Well, both are pretty great languages (not without issues, but what is?), and there are a lot of similarities -- "more than one way to do it" of course, and CRAN and CPAN have much in common (I do wish CRAN took a few lessons from CPAN though).
    - hadley 11y ago
      If you have the time, could you please shoot me an email with the lessons you think CRAN could learn from CPAN? Thanks!
      - earino 11y ago
        have we talked about CPAN testers?
tenfingers 11y ago
My biggest problem with ggplot is that it's slow, unbearably slow. It also enforces all data to be realized into a single dataframe, which is true only for small (as is: fits in memory) datasets. Very often, to produce specialized plots, I have to send data to the canvas in chunks by performing pre-processing myself. ggplot really doesn't work in this scenario. Combined with the general slowness, it forces me to use alternatives quite frequently. It's a bummer, really, because I'd like my plots to have a consistent visual style, and doing that across different plotting packages is an issue. I very often resort to gnuplot when it comes to huge datasets and/or incremental plotting. The same is true also in python (matplotlib is also very slow, independently of the backend). But at least, if you use seaborn (https://github.com/mwaskom/seaborn https://github.com/mwaskom/seaborn), you can easily intermix the easiness of plotting through a DataFrame or just supply data arrays. ggplot is really awesome for what it does, but 1) the syntax doesn't really please me (feels just plainly forced onto the wrong context 2) doesn't scale, which forces me to use alternatives too frequently 3) trying to customize the plot style beyond a few minor tweaks is pure hell.
- angdis 11y ago
  At the end of the day, the whole point of ggplot is to produce a graphical representation of some aspect of the data. How much information can you possibly cram into ONE graphic and have it be readable by a human? Your problem is really a data reduction problem and not a plotting/graphics problem.
  - hunterratliff1 11y ago
    This is a very valid point that I feel we often overlook. Most folks don't think like a statistician, and over-complicating figures is the best way to render them useless. All is lost if your audience can't understand what you're trying to convey (:
  - tenfingers 11y ago
    The data that goes into the plot is unrelated to it's visual complexity. The "problem" is that ggplot also takes care of the transformation/reduction step for you. For example, a KDE plot can source potentially a limitless amount of data while still generating a very simple plot. Likewise for most smoothers. However, if I have to produce the kde/smoothed line myself, I lose almost all advantages of using ggplot (I have to manually calculate the visual density, scaling and attaching labels is another PITA). On top of that, as other have said, ggplot really struggles already with thousands of entries. A simple 5x5 faceted scatterplot with ~10k points might take seconds to render on recent hardware. When I plot data interactively for exploration, I might do this hundreds of times a day. I lose all the convenience just in the time wasted for rendering.
baldfat 11y ago
And yet R is really a Functional Programming Language. This is why people are making OO code that really is garbage. BUT to say I gave up on a language because it has so many options is strange to me.
- rubidium 11y ago
  "BUT to say I gave up on a language because it has so many options is strange to me." This is a regular concern in designing a programming language. Give coders too much freedom and they invent 20 ways to do the same thing, none really clearly better than the other. Restrict the freedom and there's more like 3-4 ways to do something, and usually one is better then the rest. As a new user to R, I must say that the single hardest thing has been learning all the different ways to do something. Each blog/stackoverflow/etc... article I read seems to propose 5 different ways to do each thing. At the end of it, I'm left not really knowing how things work best.
  - angdis 11y ago
    There is a weirdness to R. I found that taking a lot of time to focus on the basics helps. If you're committed to using R, read John Chamber's book and also look for answers on the old "R mailing list" (a place so mean, it makes stackoverflow look like mister roger's neighborhood).
    - baldfat 11y ago
      Working in the Hadley Wickham's Universe with dplyr ggplot2/ggvis and stringr really helps a ton.
      - angdis 11y ago
        Absolutely, and don't forget tidyr! Its hard to overstate Mr Wickham's contribution to making R a very handy and useful tool. I am sure that a lot of folks would have tried R and then moved on to something else were it not for the "Hadley-verse".
hadley 11y ago
I'm pretty sure the evaluation bugs have been fixed for a while (subject to some caveats). Unfortunately it's impossible to fix fully in ggplot2 without a massive rewrite, but ggvis doesn't suffer from the same problems. Now I actually know how to do non-standard evaluation correctly :)
hadley 11y ago
I get the complaint about one more OO system, but I did consider all the options and I'm pretty sure that this was the best solution, given how few people will be exposed to it. There's also a legitimate learning challenge: if I want to figure out which OO system is best for most projects, I have to try them all out. And then the payoff for translating to a common system is small. That said, for new projects I pretty much use S3 and R6 exclusively. (And it is legitimate to have two systems because they are so different and R6 is mostly needed for internal mutability)

by zmmmmm 11y ago

TheLogothete 11y ago

And what do you use that even remotely approaches the breadth and depth of tools available in R, but solves this problem?

zmmmmm 11y ago
I wish I had a good answer to that, but I don't. Python is taking a good shot at it. But it doesn't match R by any means.
- kldavenport 11y ago
  seaborn, bokeh, pandas' built in methods, and plotly fill in the ggplot gap for me.
  - hunterratliff1 11y ago
    So perhaps we ought be working on building solutions that bridge the Python-R gap? Because as someone who has effectively left Python, I'd love to learn from solutions Python users find effective. After all, it's not like one language is the sole proprietor of useful approaches (;
dagw 11y ago
88% python + 10% calling R from python + 2% porting R code to python. edit: that being said, ggplot is still better than anything on the python side of things.
gh02t 11y ago
I'm not saying you're wrong, but this sounds exactly like the arguments made supporting Perl, which is an interesting comparison.
- TheLogothete 11y ago
  It is an interesting comparison for sure, but things are different for two reasons: 1) Software and related services will grow immensely over the next decade because of the "cloud". Some of the growth will be same old, same old application logic software, but most of it will be data analysis software. 2) The language of choice for data analysis software is and will be R due to the biggest software companies (both #1 and #2, among many others) standing firmly behind it along with universities incorporating it in its curriculum, ditching SPSS and SAS.
- xixi77 11y ago
  Well, both are pretty great languages (not without issues, but what is?), and there are a lot of similarities -- "more than one way to do it" of course, and CRAN and CPAN have much in common (I do wish CRAN took a few lessons from CPAN though).
  - hadley 11y ago
    If you have the time, could you please shoot me an email with the lessons you think CRAN could learn from CPAN? Thanks!
    - earino 11y ago
      have we talked about CPAN testers?

tenfingers 11y ago

My biggest problem with ggplot is that it's slow, unbearably slow. It also enforces all data to be realized into a single dataframe, which is true only for small (as is: fits in memory) datasets. Very often, to produce specialized plots, I have to send data to the canvas in chunks by performing pre-processing myself. ggplot really doesn't work in this scenario. Combined with the general slowness, it forces me to use alternatives quite frequently. It's a bummer, really, because I'd like my plots to have a consistent visual style, and doing that across different plotting packages is an issue. I very often resort to gnuplot when it comes to huge datasets and/or incremental plotting. The same is true also in python (matplotlib is also very slow, independently of the backend). But at least, if you use seaborn (https://github.com/mwaskom/seaborn https://github.com/mwaskom/seaborn), you can easily intermix the easiness of plotting through a DataFrame or just supply data arrays. ggplot is really awesome for what it does, but 1) the syntax doesn't really please me (feels just plainly forced onto the wrong context 2) doesn't scale, which forces me to use alternatives too frequently 3) trying to customize the plot style beyond a few minor tweaks is pure hell.

angdis 11y ago
At the end of the day, the whole point of ggplot is to produce a graphical representation of some aspect of the data. How much information can you possibly cram into ONE graphic and have it be readable by a human? Your problem is really a data reduction problem and not a plotting/graphics problem.
- hunterratliff1 11y ago
  This is a very valid point that I feel we often overlook. Most folks don't think like a statistician, and over-complicating figures is the best way to render them useless. All is lost if your audience can't understand what you're trying to convey (:
- tenfingers 11y ago
  The data that goes into the plot is unrelated to it's visual complexity. The "problem" is that ggplot also takes care of the transformation/reduction step for you. For example, a KDE plot can source potentially a limitless amount of data while still generating a very simple plot. Likewise for most smoothers. However, if I have to produce the kde/smoothed line myself, I lose almost all advantages of using ggplot (I have to manually calculate the visual density, scaling and attaching labels is another PITA). On top of that, as other have said, ggplot really struggles already with thousands of entries. A simple 5x5 faceted scatterplot with ~10k points might take seconds to render on recent hardware. When I plot data interactively for exploration, I might do this hundreds of times a day. I lose all the convenience just in the time wasted for rendering.

baldfat 11y ago

And yet R is really a Functional Programming Language. This is why people are making OO code that really is garbage. BUT to say I gave up on a language because it has so many options is strange to me.

rubidium 11y ago
"BUT to say I gave up on a language because it has so many options is strange to me." This is a regular concern in designing a programming language. Give coders too much freedom and they invent 20 ways to do the same thing, none really clearly better than the other. Restrict the freedom and there's more like 3-4 ways to do something, and usually one is better then the rest. As a new user to R, I must say that the single hardest thing has been learning all the different ways to do something. Each blog/stackoverflow/etc... article I read seems to propose 5 different ways to do each thing. At the end of it, I'm left not really knowing how things work best.
- angdis 11y ago
  There is a weirdness to R. I found that taking a lot of time to focus on the basics helps. If you're committed to using R, read John Chamber's book and also look for answers on the old "R mailing list" (a place so mean, it makes stackoverflow look like mister roger's neighborhood).
  - baldfat 11y ago
    Working in the Hadley Wickham's Universe with dplyr ggplot2/ggvis and stringr really helps a ton.
    - angdis 11y ago
      Absolutely, and don't forget tidyr! Its hard to overstate Mr Wickham's contribution to making R a very handy and useful tool. I am sure that a lot of folks would have tried R and then moved on to something else were it not for the "Hadley-verse".

hadley 11y ago

I'm pretty sure the evaluation bugs have been fixed for a while (subject to some caveats). Unfortunately it's impossible to fix fully in ggplot2 without a massive rewrite, but ggvis doesn't suffer from the same problems. Now I actually know how to do non-standard evaluation correctly :)

hadley 11y ago

I get the complaint about one more OO system, but I did consider all the options and I'm pretty sure that this was the best solution, given how few people will be exposed to it. There's also a legitimate learning challenge: if I want to figure out which OO system is best for most projects, I have to try them all out. And then the payoff for translating to a common system is small. That said, for new projects I pretty much use S3 and R6 exclusively. (And it is legitimate to have two systems because they are so different and R6 is mostly needed for internal mutability)