4 ms·

There are people who think that there's elegance in R's design. Remember that is as old as C (if not more!), but it still feels like a modern language (warts an

by urlwolf 13y ago

There are people who think that there's elegance in R's design. Remember that is as old as C (if not more!), but it still feels like a modern language (warts and all). You don't compare C to say clojure for language features, they show the advances in language design over the years. R says: 'everything is a vector, and vectors can have missing values'. This is profound. It was only recently that other matrix-oriented language extensions (say panda) got missing values, even though they are meat-and-potatoes for data analysis.

jzwinck 13y ago
Most folks use IEEE floats as their value type, so in a way a whole lot of libraries have supported missing values for a long time. Many languages can use 32- and 64-bit floats; R can only use one or the other (you have to rebuild it to change this). This can make your big data twice as big as it should be. In that, R is in the company of such serious number-crunching languages as Lua, JavaScript, and Splunk.
- dbaupp 13y ago
  How many of the libraries do something sensible when given NaNs? I'd assume a nontrivial number give junk output (i.e. not even reliably NaN).
- jmount 13y ago
  R has good representation of missing values over integer, logical, string and categorical data types also.
bokchoi 13y ago
SAS has missing values as well and goes further -- there are multiple types of missing values (ie, "not available", "doesn't pass quality control review", etc.) We added missing values to our product to interop with both R and SAS: https://www.labkey.org/wiki/home/LabKey%20Server%20Documentation/page.view?name=manageMissing https://www.labkey.org/wiki/home/LabKey%20Server%20Documenta... (Pardon the extremely old screenshots.)
- hadley 13y ago
  In SAS, beware that missing values are treated as the smallest possible value (e.g. -Inf). This means that statements like x < 10 return true if x is missing.
  - lsiebert 13y ago
    Oh god yes, and it will bite you if you don't know it. That takes me back to cleaning SAS data.
- pitnips 13y ago
  But R is free!
mwexler 13y ago
Well, it's not older than C, but around the same time. Wikipedia shows S (R's commercial inspiration, for want of a better term) was first written in 1976, but C was around back in 1972. And C was adopted by lots of people a lot faster than S, though it kept chugging along. Agree with the missing values in native datatypes point, although many libraries for other languages had workarounds for missing values that allowed you to get similar results even back in the day, way before python and panda. I think the general consensus is that for every thing S and R got right, they got a bunch of other stuff not-so-right, but it was worth the pain to work around them because the good stuff was so very good.
- withme 13y ago
  Regarding the adoption rate of S, I think you hit upon a key source of the complaints about R. R existed in one form or another for a (relatively) small, specialized audience for a long, long time. This included the period where transitioning from S to R included placing a very high priority on maintaining code from S, even being able to replicate the bugs from S. This made a lot of sense from the perspective of a (relatively) small group of statisticians making a data analysis language for their own uses. Over the last 10 years, R has become massively more popular, and many, many more people are using it outside of its "base" audience. Not surprisingly, many of these new users are discovering that R was not designed with their personal use cases in mind. Given these constraints, R has done a remarkably good job of adjusting (where it can) to accommodate the diversification in its user base. But you can only go so far without just re-writing it from scratch, and the interests of R Core will always bend towards themselves and their, well, "core" audience.

by urlwolf 13y ago

jzwinck 13y ago

Most folks use IEEE floats as their value type, so in a way a whole lot of libraries have supported missing values for a long time. Many languages can use 32- and 64-bit floats; R can only use one or the other (you have to rebuild it to change this). This can make your big data twice as big as it should be. In that, R is in the company of such serious number-crunching languages as Lua, JavaScript, and Splunk.

dbaupp 13y ago
How many of the libraries do something sensible when given NaNs? I'd assume a nontrivial number give junk output (i.e. not even reliably NaN).
jmount 13y ago
R has good representation of missing values over integer, logical, string and categorical data types also.

bokchoi 13y ago

SAS has missing values as well and goes further -- there are multiple types of missing values (ie, "not available", "doesn't pass quality control review", etc.) We added missing values to our product to interop with both R and SAS: https://www.labkey.org/wiki/home/LabKey%20Server%20Documentation/page.view?name=manageMissing https://www.labkey.org/wiki/home/LabKey%20Server%20Documenta... (Pardon the extremely old screenshots.)

hadley 13y ago
In SAS, beware that missing values are treated as the smallest possible value (e.g. -Inf). This means that statements like x < 10 return true if x is missing.
- lsiebert 13y ago
  Oh god yes, and it will bite you if you don't know it. That takes me back to cleaning SAS data.
pitnips 13y ago
But R is free!

mwexler 13y ago

Well, it's not older than C, but around the same time. Wikipedia shows S (R's commercial inspiration, for want of a better term) was first written in 1976, but C was around back in 1972. And C was adopted by lots of people a lot faster than S, though it kept chugging along. Agree with the missing values in native datatypes point, although many libraries for other languages had workarounds for missing values that allowed you to get similar results even back in the day, way before python and panda. I think the general consensus is that for every thing S and R got right, they got a bunch of other stuff not-so-right, but it was worth the pain to work around them because the good stuff was so very good.

withme 13y ago
Regarding the adoption rate of S, I think you hit upon a key source of the complaints about R. R existed in one form or another for a (relatively) small, specialized audience for a long, long time. This included the period where transitioning from S to R included placing a very high priority on maintaining code from S, even being able to replicate the bugs from S. This made a lot of sense from the perspective of a (relatively) small group of statisticians making a data analysis language for their own uses. Over the last 10 years, R has become massively more popular, and many, many more people are using it outside of its "base" audience. Not surprisingly, many of these new users are discovering that R was not designed with their personal use cases in mind. Given these constraints, R has done a remarkably good job of adjusting (where it can) to accommodate the diversification in its user base. But you can only go so far without just re-writing it from scratch, and the interests of R Core will always bend towards themselves and their, well, "core" audience.