19 ms·

One Year with R

• VariableStar 5y ago
  Excellent read. I agree with a lot (after only a cursory read). One thing the author seemingly forgives R by not mentioning is how harsh and discouraging of beginners the community was at some early stage. That was my experience around 2002-2006.
  • jboynyc 5y ago
    I'm not sure that's true anymore. The one thing that I find interesting about the R community (being at home in Pythonland myself) is the availability of great teaching and learning resources. At least that's the case in the social sciences, where R has pretty strong adoption.
  • s_Hogg 5y ago
    Yeah I tried to publish a package back in like 2015 and was dealt with very harshly, got banned for a week from submitting for asking questions about the process after the first attempt didn't work. It really turned me off R and frankly I haven't looked back.
• kome 5y ago
  For my use case, R is absolutely terrible compared to some for profit statistical package / language. Using R feel like using an outdated, complicated and messy tool. But guess what: it's free.
  • socksset 5y ago
    What makes R interesting is the amazing libraries for statistical ideas that are not completely run of the mill. You almost literally can't come up empty on CRAN.
  • tweakimp 5y ago
    Maybe you can switch to Python? Its also free and has a lot of statistical packages.
  • zhdc1 5y ago
    > For my use case, R is absolutely terrible compared to some for profit statistical package / language. Which one? I've switched most of my work over to Julia, but I'd much rather use R or Python than Stata or SPSS.
    • musingsole 5y ago
      I don't understand the use case for SPSS. My local university is training their neuroscience researchers on it, which seems so odd in 2022 with Julia or python sitting right there.
      • dagw 5y ago
        Teaching someone who knows a bit of Excel and very little programming how to do statistical analysis in SPSS is easy and lets you focus on the statistics. Teaching them to do statistical analysis in Julia will involve you spending 80% of your time teaching them Julia and maybe 20% of your time teaching them statistical analysis.
        • zhdc1 5y ago
          > Teaching them to do statistical analysis in Julia will involve you spending 80% of your time teaching them Julia and maybe 20% of your time teaching them statistical analysis. This works until they run into a use case that doesn't involve running various forms of regression analysis on panel data. In the parent comment's case, I could imagine that there's an expectation that someone doing neuroscience research will eventually have to expand beyond what's possible in SPSS. In this case, it may make sense to go through the effort of teaching them how to program in Python or Julia.
        • ivansavz 5y ago
          I agree with you, though I wonder what's included in your definition of "do statistical analysis" ? Is it just using the stats functions as blackboxes without understanding what is going on under the hood? I find using Python and/or R to be very helpful for teaching, since you can implement the stats procedures using primitives (prob. calculations), so you get some experience with how things work. Sure it requires some "coding" but nothing harder than using a calculator, so I think it's worth learning. Julia is a bit more involved (need to learn something about data types), but still would be manageable.
  • Bostonian 5y ago
    The language itself can be criticized, but when academic statisticians publish a new method, they often post an R package, too, so it has recent functionality unavailable elsewhere.
  • goosedragons 5y ago
    What for profit statistical package feels better? SAS literally has 8 character name limits in places. The data input command is literally called CARDS. It feels ancient. Minitab and SPSS aren't much better in syntax with regards to scripting which is important for reporoducability.
  • _jordan 5y ago
    JMP and stata are really solid tools, but I feel like the advantage stems from the fact that they help me discover new statistical methods. In the GUI I can see an option for something I never seen before, read the documentation, and therefore expand my stats skillset. This type of discoverability is a bit harder with R/Python just due to the nature of it being purely script driven.
• evacchi 5y ago
  this comes immediately to mind http://arrgh.tim-smith.us/ http://arrgh.tim-smith.us/
  • tgvaughan 5y ago
    As somebody who also used to enjoy bashing R for similar reasons, I think the new preamble to that document is an important and nice addition. (Wasn't there last time I read it.)
• uniqueuid 5y ago
  What always surprises me is how many people make beautiful, lovingly-crafted band-aids to the language's warts and problems. Not just code + packages but social band-aids too. In a way, you could argue that the entire tidyverse is a huge effort of a band-aid. So for all the irritating design choices and idiosyncracies, R is still a network of islands that work incredibly well for people, as long as they don't ever go to sea.
• indeed30 5y ago
  I love R more than any other language I have ever used. Perhaps more than any piece of software I've ever used. All of these points are valid, and yes, it's messy, and if you try to write the same type of code that you would in Python, it will frustrate you. And yet.. it somehow works. It makes data analysis and statistical modelling a pleasure. It somehow gives off a sense of lightness, and makes it easy to investigate and explore. I would guess I am genuinely 2x as productive in R as I would be in Python on similar tasks. I know it's not a "proper" language, but I think that, maybe, not everything has to be exactly like "proper" software engineering?
  • azalemeth 5y ago
    I very much agree with this. I use python for (different types of) data analysis too, and in python in particular it feels like the "boilerplate" to "science" ratio is rather high in the direction of "boilerplate". R manages to abstract this away very effectively, as the article highlights. The beauty of R is that you can write one line of code and use some hot-off-the-PhD-thesis cutting-edge-just-published-in-J.-Stat.-Soft-chunk of statistical analysis in your totally different, completely whacky problem, and it's fast, and (by and large) works. Of course, that's its biggest problem as well. Scientifically, it will quite happily give you a 150 mm howitzer to aim at your foot, assuming you know best.
    • hervature 5y ago
      > hot-off-the-PhD-thesis cutting-edge-just-published-in-J.-Stat.-Soft-chunk of statistical analysis I think you mean "poorly-documented-cobbled-together-under-deadlines-never-to-be-maintained by someone who has no idea of software principles". Very few labs have a dedicated software engineer to actually turn this software into a usable/hackable tool let alone maintain it.
      • ekianjo 5y ago
        thats an unnecessary negative stance. not every algorithm needs to be scalable and over optimized to be useful in most cases. and if something becomes really useful in R it ends up being reimplemented in more effe five ways down the road.
• AuthorizedCust 5y ago
  I don’t like this. Much of this is: 1. pointing out that, like every other language, base R has idiosyncrasies 2. how use of R is more complex when you’re largely ignorant of the tidyverse, which is crucial for the vast majority of tissue today’s use of R 3. frustration because you’re using a language/ecosystem, that’s targeted for a few specific uses, as a general purpose programming language
  • fn-mote 5y ago
    > how use of R is more complex when you’re largely ignorant of the tidyverse This. I'm interested in non-flamewar non-religious reasons that the tidyverse is bad. He does give some. I think his complaints about inconsistency and a moving target have some validity. However, the price of not using tidyverse is (roughly) paid in the rest of the article. I would definitely not use R without it. Read his Section 5 on the tidyverse... and see how absolutely minimal his complaints in that section are. E.g. to "purrr" his objections are "largely philosophical"... but he's complaining in the previous section about the annoyance of writing lambdas (which purrr makes even easier). Yes, R has a big community and there's a lot of quirks in individual packages, especially less-used ones. Yes, there are packages presenting unified interfaces to other quirky outputs (e.g., broom). The necessity of this is not good. The existence of it is good. HN readers - do you have an "up and coming" language that you think has better structured the fundamentals from R, that you hope will someday have enough capabilities you can use it instead of R? I've tried Julia, which is beautiful but the startup/compilation times were difficult to get over. Is it reasonable to hope Julia will be good for interactive usage someday? Is it already? Are there other candidates in this area?
    • epistasis 5y ago
      Worst part of the tidyverse is learning it, and then looking up how to use specific functions. The bad documentation is mostly in the ggplot lib though. It's a pleasure to use, though!
      • AuthorizedCust 5y ago
        The “bad” ggplot documentation is mostly a function of one’s own understanding of the grammar of graphics. That’s what the “gg” in ggplot stands for. If you don’t understand the GG, then ggplot will seem opaque, and no goodness of documentation will suffice. I don’t mean to blame the user. Perhaps the ggplot documentation could improve by reinforcing the need to understand that or referencing it more frequently?
• toto444 5y ago
  The points the author mentions are fair but something feels amiss. I have used R heavily and still use it from time to time and I never use most of the functions mentioned in the post. For instance I have never used switch(). R is for data manipulation. 90% of what I do in R is manipulate dataframes or matrices and then run machinelearningmodel(mydataframe) or ggplot(mydataframe). And for this it is incredibly efficient. You can rightly argue that some elements of the language are quirky but that's missing the point. > Asked over 100 Stack Overflow R questions. As a tangent I find a hundred questions asked the first year for a very mature language is a lot.
  • dm319 5y ago
    Yes, I think if you are using switch() for an analysis in R, you're either using the wrong language, or you're doing R wrong.
• roel_v 5y ago
  I think (and I usually anger at least some people when I say this) that it's wrong to see R as a 'programming language'. I mean, it looks like one, and it's Turing complete so if you use that as a criterion it is a programming language, but I think it's more useful to see it as a stats software package with a text-only user interface. Approaching it this way instead of as just another language to 'pick up' makes using it much less frustrating.
  • gadflyinyoureye 5y ago
    I just got done making a DockerLambda in R. The lambda itself is simple. It takes a few inputs, gets data from S3/Files into dataframes, and passes it off to the real calculation. The actual math is done by another team. I approached it like you described. I wouldn't want to do a really complex REST API in it, but as a wrapper for calculations, we've got a repeatable pattern to run them in a cost effective manner.
  • mellavora 5y ago
    I supposed it depends on how you define a 'programming language'. I'm not angered, I'm more wondering about the usefulness of the arbitrary line you've drawn in the sand, and even the shape of the line. RShiney lets you build interactive webpages with advanced GUIs; if <webdev stack> counts as a programming language, why not R? Many people like python because it lets you script things, and you can even make your script executable with a shebang at the start (!# /bin/python) -- and while true, that isn't built into R, you can run R scripts programatically (> Rscript myfile.R), or make this executable by putting it in a standard shell script.
• mtizim 5y ago
  Fun fact: there exists a "Why R? Foundation" which holds yearly conferences, because after all these years the R community just cannot find a sensible reason to use the language.
  • BrokrnAlgorithm 5y ago
    I used to do R intensly. What I found, after moving to more "SE"-centric languages such as Python or C++, was that R becomes quite frustrating when you need to build something maintainable. Which is basically a definition that should apply to every package. As soon as S3, R6 and whatnot comes into play at multiple levels, I'm better off moving to C in order to minimize interacting with R's class system and having that done only at the topmost layer.
• lvl102 5y ago
  Never liked R. Always felt like a cheap alternative to either proprietary solutions (Stata and SAS) or simply a better experience in Python.
• beernet 5y ago
  Long-time R user here. Yes, many of these points are valid but I still think R is unbeaten when it comes to speed in (tabular) data exploration. In the article you mention that you missed using data.table - a significant portion of the problems you named would be solved or at least weakened by using data.table. Started working with it many years ago and never looked back. It's easy, powerful and efficient to use. I also find its Python pendant, pandas, much less intuitive to use (.loc() anyone?) although they compare performance wise.
  • tomrod 5y ago
    .loc works smashingly. %>%?
    • curiousgal 5y ago
      Well thats the thing with pandas, which one is it? [], loc, iloc, . ? Why do I have to reset_index so often? I agree with OP R has Pandas beaten when it comes to accessing data.
      • tomrod 5y ago
        iloc - numerical location, loc - location in table I rarely reset index -- perhaps its a difference in familiarity? (I use R but it isn't my background, perhaps there is a forced R pattern that is a general antipattern for indexes?)
      • _dain_ 5y ago
        I will agree about [] being overloaded, but .loc and .iloc are distinct for a good reason. .loc is for operations according to the index, .iloc is for operations according to position. You have to reset_index() so often probably because you are not using the index properly. Effective use of Pandas means effective use and consideration of the indexes on your dataframes and series. Recommended watching (32:00 onwards): https://www.youtube.com/watch?v=mWtfZaT7iSc https://www.youtube.com/watch?v=mWtfZaT7iSc
    • nojito 5y ago
• ggrothendieck 5y ago
  What needs to be added is that before R the reproducibility problem in science was compounded by the fact that analyses were done with proprietary software limiting communication and replication of those analyses. This was and continues to be a major problem, particular in some fields, but at least now there is a common widely used language that can be used to overcome this. I wouldn't focus on idiosyncrasies but rather on the major problem it addresses. Any large system will grow over time and have some inconsistencies but after a while you learn the workarounds so they are less important than the big picture.
  • _Wintermute 5y ago
    I don't see how R specifically addresses the reproducibility problem, It's been around for almost 30 years and before its recent rise in popularity, lots of science was done in C, perl, fortran etc. Not to mention that actual dependency versioning is pretty poor. I struggle to run other people's R code after about 6 months (especially if they used the tidyverse as it pulls in hundreds of unstable dependencies) and nobody records what package versions are used and functions are seemingly deprecated every week.
    • ggrothendieck 5y ago
      1. Before R commercial statistical packages were mainly used. You can, in principle, just use assembler too and develop everything yourself but it isn't practical. Regarding C/C++ and Fortran, many R packages are, in fact, wrappers around code in those or other languages making it easier to access them. From that point of view R can be regarded as a glue language. 2. Regarding keeping versions straight, all past versions of packages in the CRAN repository are kept on CRAN. Microsoft MRAN repository also maintains histories of packages that can be accessed via the checkpoint package which will install packages as they existed on a given date. Furthermore, install_version in the remotes and devtools packages can install specific versions. 3. Regarding tidyverse dependencies you can reduce the number of packages you load by not using library(tidyverse) and instead load the specific packages you need. This will result in fewer packages being loaded.
• em500 5y ago
  The common trope with R is that statisticians and love it and developers hate it. The the main reason that statisticians love it is that the libraries useful to them are much better in R than elsewhere (though Python keeps encroaching in that turf, and "real developers" dislike Python a lot less than they do R). The main reasons that developers hate it is that it is very unlike almost all other languages that they're used to. This is very valid since outside the narrow domain of statistics, there's probably nothing that R does better than other languages. So for a dev who occasionally dabbles with R by necessity, the otherness serves nothing but frustration. Still, I wonder how much criticism there is against R as a programming language, that is not some variation on this works very differently from other languages. IMHO the sub-setting syntax, and countless x-apply variations are big warts. I'm not a big fan of Tidyverse, and even less of the schism between base- and Tidy-R. I read some seemingly fundamental criticism about R's deficient scoping rules, but I'm not nearly knowledgeable enough to judge their merits. I guess it doesn't help that almost nobody learned R as their first computer (as opposed to statistics) language. Personally, I learned C, Matlab, Python/numpy, SQL, R in that order. R does seem to be quirkier than all the others, except maybe SQL. But I don't dislike working in R any more than working in any other language.
  • pantulis 5y ago
    I had some Matlab experience about 3 decades ago. What's you take in Matlab vs R as programming languages?
    • em500 5y ago
      If signal processing and matrix algoritms are your thing, you should (and probably would) be using Matlab. Most statisticians don't really do much of that, what they mostly do is data management and try to mold their tables in some form accepted by an existing R package (or even Stata). As far as I remember, Matlab was pretty horrible for data munging, even worse than R for anything non-numerical. But my Matlab experience is also almost 2 decades old, so I don't know if it's any better today.
    • tomrod 5y ago
      Not the grandparent. Matlab is much simpler for matrices than R, approaching Python's numpy is ease of use (and very reminiscent of Fortran and Julia).
• stewbrew 5y ago
  > R has two types of empty string: character(0) and "". I understand it's frustrating trying to use a language you don't understand. And instead of reading the language manual you go on rambling. "" is an empty string (almost) as you know it from other languages. character(0) is an empty vector of type character (i.e. a vector with no elements). This vector doesn't even contain an empty string. R is a vectorized language. You almost always deal with vectors. "" actually is a character(1), a character vector of length 1. Once you understand this, there is a chance for you to enjoy R.
  • usgroup 5y ago
    > I understand it's frustrating trying to use a language you don't understand. And instead of reading the language manual you go on rambling. I'd agree with this assessment. If you start doing R and it feels weird to you then -- in my opinion -- you're probably in the wrong place. Meanwhile, for the cognoscenti -- the researcher, the statistician -- R behaves just as you'd expect. That is the draw -- a language developed around statistics. R is not a great computing environment for computer science. E.g. writing iterative algorithms. Almost everything worth a damn in R is written in C++ and then FFId in. Those who do not want to use C++ can write their algorithms in Python or Julia -- and they often do. Arguably the defacto for computing oriented machine learning is Python, not R.
  • probably_wrong 5y ago
    I'm going to side with the author here: if he read "Advanced R", "R for data Science", "The R Inferno", "Rtips. Revival 2014!", the official "An Introduction to R", "R Language Definition", and "R FAQ", and yet he still has problems with the language, then maybe the language is to blame. And even if the author is the problem, I wouldn't accuse them of not reading enough.
    • stewbrew 5y ago
      Or maybe the language just isn't for every one and for every usecase. I would be hesitant to write something customer-facing in R. But it's great for doing statistics. The main problem with R is that people underestimate how different it is and thus don't care to learn practices for writig robust R code.
• crispyambulance 5y ago
  I was expecting a rant, but the OP's article is actually very thoughtful. He definitely knows what he's talking about. The thing about R, for me and many others, is that it's very much an everyday grind language. Especially with Rstudio, its natural domain is as one of "notebook" languages like python, julia, matlab, and mathematica but with a more clear focus towards the tasks of data-analysis. I just tell the BI-tool people that R is excel on 'roids. R frustrates me a lot, however. But I think the frustration comes out of the fact that when I am using R and get stuck, I am always in the middle of doing something that I need to get done and I don't feel like diving into a long "vignette". Moreover, the documentation is usually too terse and generalized for me to just understand it immediately. Even though I've been using R for years (albeit in fits and starts rather than continuously), there are things about it that I've just never picked up-- I just DON'T KNOW (or care) what F S3 and S4 mean. Unlike the OP, who clearly knows more R than myself, I grit my teeth when I am looking at docs and see the "..." in the arg list. I suspect that this is part of the heritage from R's beginnings. I once tried to read John Chamber's book but found the presentation complete ass-backwards and impractical for my immediate needs. The Tidyverse has been great, it's far more consistent and ggplot is a kick-ass tool to have in your box. The drawback is that it makes Base-R seem really alien and if you want to be good at R, you have to know more than just the Tidyverse, IMHO.
  • clove 5y ago
    I've used R for 19 years and do not have any other programming language ability. I am curious: What is frustrating to you about R relative to other languages?
    • vharuck 5y ago
      In my experience: - It has a bunch of different types of classes, and they all behave differently. Debugging isn't awful, but it's harder than it should be. Also, the documentation isn't clear about which classes to use. - A lot of the workhorse functions suffer from parameter glut. Despite having different kinds of classes, almost all functions expect plain vectors. Packages like survival show how objects make it easier to read code, reuse data, and validate data. Without the base packages doing it more, everyone's chosen their own systems. The community's been gravitating to organizing "objects" as rows in tables (i.e. tidy). - The way a function uses an argument might surprisingly change based on other arguments given (e.g., `binom.test`). And then the documentation won't have examples for the different use cases. - Most users don't have the time or desire to become better R programmers. They have other work to do. For my own work, I write packages with custom classes, functions, and template documents. For collaboration, I keep things very plain and rarely go beyond dplyr; very often, the script goes between two steps executed in a GUI software.
• kgwgk 5y ago
  > [discussing how c(list(1, 2, 3), LETTERS[1:5]) is not what the author would expect] To get list(1, 2, 3, LETTERS[1:5]), you must do something like x <- list(1, 2, 3); x[[4]] <- LETTERS[1:5]. The following works and it looks quite natural: c(list(1, 2, 3), list(LETTERS[1:5]))
• kgwgk 5y ago
  > Here’s a challenge: Find the function that checks if "es" is in "test". You’ll be on for a while. grepl(“es”, “test”)
  • merlincorey 5y ago
    I haven't used R at all in years and only used it a couple times in passing many years ago to try it out. I searched "R string functions", saw "grep" and wondered if there was something I was missing in the author's challenge. Is it because it's using regular expressions they don't consider it the correct answer or is it because they aren't as familiar with regular expressions as some other people are, I wonder?
• CapmCrackaWaka 5y ago
  I would say about 90% of the posts / articles / comments I see on the internet which discuss R are usually of the "meta" format. They talk about R's strengths or weaknesses, about the difference between R and Python, about how much they love or hate R, or any other high level subject. I can't remember the last time I saw a project someone did in R, or a tutorial on how to do something in R, get very much traction anywhere. It seems the only time people talk about R on the internet is to discuss the language itself (which is definitely frustrating), and it's getting old. Even this awesome, comprehensive document, which I would usually be foaming at the mouth to read, has me going "meh". I'm tired of the subject.
  • disgruntledphd2 5y ago
    This is definitely true on HN, at least. I think the vast majority of R-users are just plugging away on their domain specific problems daily, and tend not to participate in these conversations. Dark-matter statisticians, I guess?
  • hadley 5y ago
    They don't often come with code, but one of my recent sources of R programming joy is the folks posting their generative art to twitter: https://twitter.com/search?q=%23rstats%20%23generativeart&src=typed_query&f=top https://twitter.com/search?q=%23rstats%20%23generativeart&sr...
  • mellavora 5y ago
    > I can't remember the last time I saw a project someone did in R, or a tutorial on how to do something in R, get very much traction anywhere. well, you know, I'm not very active in C++ any more, and I haven't seen an article in over a decade on C++ which received any traction at all. So I guess C++ isn't getting any traction any more either.
• tel 5y ago
  R, and by R I mean R+tidyverse, is the world's best graphing calculator attached to an OK scheme. To which I mean R is a highly optimized, well-oiled machine if you're using it for its highly-optimized, well-oiled purposes. I tend to have notebooks full of tiny fragments like this dat_min %>% group_by(ymd = make_date(year(date), month(date), day(date))) %>% summarize(vol_btc=sum(vol_btc), vol_usdt=sum(vol_usdt), tradecount=sum(tradecount)) %>% ungroup() %>% pivot_longer(cols=c(-ymd)) %>% ggplot(aes(ymd, value)) + geom_line() + facet_grid(name ~ ., scales="free_y") It's madness if you're not familiar with the tidyverse, but 3 dozen fragments like this is enough to eviscerate a fresh data set. Almost any question you can dream of is a 3-20 line set of transforms away from a beautiful plot or analysis answering your question. Very notably, this includes some of the finest modeling tools available today. Terseness here is a huge advantage as well because in many data analysis workflows you are rerunning that same 10 line snippet over and over, making small changes, adjusting to eventually visualize the thing you're looking for perfectly. Having all of that in the same small block is ideal. Finally, for the non-trivial number of folks in this specific scenario, the integration between Stan and R/RStudio is top-notch and makes using both tools very pleasant. You can replicate all of this in Python, but optimal Python/Jupyter is still a far cry away from R/RStudio for these specific sorts of tasks.
  • CapmCrackaWaka 5y ago
    I also use R for any heavy data manipulation, but I primarily use the data.table package. The efficiency that both of these packages unlock is absolutely unparalleled in any other tabular data manipulation library, in any other language that I have used. And R has the top 2!! My skin writhes every time I need to type: table.loc[(table.column > 2) | (table.column2 < 3)].reset_index(drop=True) when I want to subset a table.
    • alexilliamson 5y ago
      Not to mention the auto complete that comes with RStudio. Is there any way to get equivalent functionality in Jupyter?
• prionassembly 5y ago
  I only briefly knew R in grad school (circa 2007) but I lived inside Stata for about 7 years, and yeah, while specific frustrations vary, the general tenor... Then -- I once made a meme to that Oliver Stone Vietnam movie that said "This is my copy of Stata. Without me it is useless. Without it, I am useless. I must cherish it as I cherish my life..." (In the original said of a rifle.) I was good with Stata, fast and precise and never "ugh, okay, let's open a quickie notebook... there goes my morning..."
• dash2 5y ago
  I think this is really interesting. The author certainly isn't an expert, for example `result[which(result < 0.5)] <- 0` is a mistake for `result[result < 0.5] <- 0`. But that's just why it's useful - R is great when you are an expert, but becoming an expert takes years. The perspective of new users is really important. (I've been using R almost 20 years, have written several packages, and still feel like an amateur. Indeed, I'd never heard of `**` as an alias for `^` until today; nor `sequence`, which apparently has always been in base; and I still can't remember what `sweep` does.) I thought some of these arguments were better than others. True that base R regex is confusing and messy (and that stringi/stringr are improvements). False that allowing string concatenation with `+` would be a good idea. That's just a footgun waiting to go off, given that R also is weakly typed. Expecting `nchar(1000)` to magically work seems naïve. `<<-` (roughly, global assignment) is an ugly necessity and a code smell, not a cool language feature. An awful lot of these problems are fixed, or try to be fixed, in the tidyverse. Not using tidyverse is a bit unusual because most beginners nowadays, I think, start with the tidyverse more than with base R. For me the worst part of R is simply it fails silently. This is really deadly, especially when you are producing scientific results. There are so many places where R will plug gamely on after you have done something deeply inappropriate. Given how badly scientists code, one has to worry. I don't agree that "R won’t change" is the base problem. It's not so simple. R is used for science. I like very much that my code from 2008 will probably still work if someone wants to replicate my results. I appreciate the R-core team's work in making this true. There are genuine trade-offs here. If you want emotional relief, it's worth following https://twitter.com/whydoesR https://twitter.com/whydoesR. Maybe Julia is the way forward? Or is R "worse is better"?
  • bluenose69 5y ago
    > Maybe Julia is the way forward? Julia is well worth learning, if you do computationally-expensive work. It is kind of a pain to use interactively, though. I use both R and Julia in my research. Think of Julia as the new Fortran, though, not the new R.
    • edgyquant 5y ago
      Isn’t Julia a compiled language “pretending” to be interpreted? I thought it was interpreted or JITd up until it’s last release or so when they mentioned it has to actually compile. It’s fast, sure, but is it faster than other compiled languages?
• vijucat 5y ago
  A minor tip : I use the hashtag #learning to annotate pieces of code which I can then revisit using a search (could be just grep '#learning' *.R | grep data.table, say) in case I'm stuck. This could work with any language, of course, but in the initial days, I found it very useful with R in particular given it's idiosyncrasies. (For the oldies on here, I grew up in the era of del.icio.us, so hashtag-ging code felt like a natural-but-novel idea :-) ) Examples: #learning : a data.frame is a list. x = df with 10 rows, 21 columns, say. as.list(x) gives you a list with 21 elements, one per column #learning : Above = getting a row and it's previous row using .I() in data.table
• clusterhacks 5y ago
  I support bioinformatics researchers and my R problem isn't the language itself but the increasing fragile tower of packages that users cobble together. At this point, I see R users (typically PhD students and post-docs) doing "science" in R by playing with parameters to functions in poorly-understood packages and publishing papers on which parameters are "best" for data generated from some specialty source. A very common situation for me is to be pulled in only after a package has been created with some vague hope of fixing performance problems (which R, Rcpp, and RcppParallel make fun to do for me, but I have some C++ background for scientific computing, ymmv). It is extremely common to find that these packages contain fundamental logic errors that probably should invalidate the (already published) results but never got caught because the code ran without actually failing. I guess I'm complaining that people are using buggy packages to write more buggy packages and it just bothers me. Library-driven development is just how the world works these days. And it should! But I'm not confident that the R bioinformatics world has the kind of guardrails I would prefer to see. I mean, I am reasonably confident tensorflow is functionally correct. Any R package that pulls in too many other R packages to begin with is probably not. As for the language itself - I guess it is ok. I have some lisp in my background and a fair amount of love for non-traditional array languages. But I don't see much R code that seems to stick to the R "standard library" rather than pulling in a million packages to do anything . . .
  • jghn 5y ago
    > I see R users (typically PhD students and post-docs) doing "science" in R by playing with parameters to functions in poorly-understood packages and publishing papers on which parameters are "best" for data generated from some specialty source. That's a lot of bioinformatics, and not specific to R. It is a huge issue with anything vaguely pushbutton in the bioinformatics domain.
  • newbamboo 5y ago
    “I don't see much R code that seems to stick to the R "standard library" rather than pulling in a million packages to do anything.” People teach the tidyverse to new r users. It makes them think that it’s standard practice to pull in lots of unnecessary but possibly convenient packages. Simple string manipulation should not require an extra package like stringr, but for many users it does. Often, they were taught this way.
• bluenose69 5y ago
  R is designed for data analysis, not for general computing. Its syntax differs from that of other systems. Python's syntax also differs from other systems. Same for Matlab. And so on. Non-uniformity imposes a burden that will be too much to bear, unless the system offers particular advantages. The fact that several systems co-exist is proof that the advantage-burden balance is favourable in each case. There is no need to converge on a single tool. Carpenters need both saws and hammers. In practical applications, language syntax is just part of the story. One must also consider the issue of available libraries. One thing that really stands out with R is its immense collection of well-vetted and well-documented packages. Python and Matlab -- the two main alternatives in my discipline -- fall far behind R in this respect. If there's a journal article on a new statistical technique, then there's a pretty good chance of a package written by same author. And, if that package is on CRAN (the repository for such things) then it has undergone quite rigorous testing on several types of computer, with several versions of R.
  • mellavora 5y ago
    AND! packages don't update every 3 weeks breaking things! My diety! someone was complaining about inconsistent syntax but doesn't recognize inconsistent dependencies?
• CornCobs 5y ago
  I've written an interpreter for R (a subset; it was for school and I left out some features like S4 and the condition system) so I have done some pretty deep dive into the language reference and GNU R source. I agree with the author's sentiment - I love a lot of what R has, but there is a lot of small madnesses. There are so many unique PL ideas in R (may not actually be unique but certainly unique among common languages today) - first class environments - named, default parameters and even the ... parameter which encourages the pattern of hierarchical library functions - there's one large customizable main workhorse function, and many wrapper functions that specify some defaults or add some behavior, but all the underlying customizations are exposed through ... - copy on write as a default - ability to choose evaluation strategy But I also wonder how many of these cool ideas would actually work well in a saner language
  • ggrothendieck 5y ago
    Is the subset R interpreter you wrote available on the net?
    • CornCobs 5y ago
      Sorry for getting back to you so late; I'd rather not link my github here as I'd rather remain as anonymous as possible on public forums
• jasonpbecker 5y ago
  I am looking forward to reading this but I need to point out that section 1.3 Ignorance is both interesting and necessary. What 1.3 basically outlines is, "I used R for a year, but the way I used R is very different than what most R people use R for."
• temp8964 5y ago
  There are a few things the author did not mention, such as RStudio Server, Shiny. If you get to know them, you will find they do certain tasks extremely well, and there is simply no equivalent of these in other data programing ecosystems. Comparing R with others as merely a language is close to meaningless. You have to take the whole ecosystems into account.
  • nojito 5y ago
    There's no greater joy than whipping up a POC in a few hours using R Shiny and showing it to senior leaders when they were told that it will take months to get a POC ready.
• deck16er 5y ago
  I am not developer by profession but have been programming since early 90s starting with basic, fortran, C++ and Matlab. Learnt JavaScript, Python, Lua as well along the way for various reasons. Found R when I was looking for something free alternative to Matlab and chanced upon R in 2010/11. Now a days R is my goto scripting language anytime when I just want to get to the results and don't care about reproducibility. I also use Shiny as alternative to multiuser scenarios involving spreadsheets since I when work in a financial firm where excel and VBAs still dominate most of the front office functions. Sure Python could be good tool but once you become fluent with R ecosystem, moving to python just feels too much that can be done with few lines of R code. For me the conciseness of data.table and ability to cook up shiny web apps with very few lines of code is biggest pull.
• j_m_b 5y ago
  My favorite operator is the pipe operator. When I first found out you could do a simple `ls | more` to read long outputs, it was an eye opening experience. In Clojure, we have the threading macros, `->` and `->>` that do a very similar thing. In R, we have `%>%` and now the native `|>`. Whenever a language has this operator and it is widely used, I know I am going to love it.
  • substation13 5y ago
    I credit F# with much of the popularity of the forward pipe operator. Unlike Haskell etc. which emphasize function binding (>>), idiomatic F# has pipes all over. [1..10] |> Seq.filter (fun x -> x % 2 = 0) |> Seq.map (fun x -> x * x * x)
    • j_m_b 5y ago
      Looks like I need to check out F#! Thanks!
• legoxx 5y ago
  I should write Frustration: one year with Swift & SwiftUI or my favorite Frustration: years with PHP ;-)
• bachmeier 5y ago
  I agree with the concerns about lists. They're a poor substitute for structs in a language with static typing. By now I've accumulated lots of knowledge and helper functions. You can dramatically simplify your life by using lists with lapply and related functions. I teach students with no previous programming experience to do some things that would otherwise be far too complex, but I also have to write a helper function to convert the output into a usable form for further analysis like plotting.
• chubot 5y ago
  The popularity of the Tidyverse is a major blow to your motivation to learn R. Why would anyone want to learn a language that is treated as secondary to some packages? Worse still, if that turns out to be the best way to use R, then you’re forced to admit that R is a polished turd with a fragmented community. As others have mentioned, just use tidyverse. I picked it up 4 years ago, and last week I went back to the code I wrote then. I was productive in minutes. I could read the code, modify it, and easily test it in the REPL. The docs for dplyr are good. ggplot2 is still awesome and the docs are good there too. ggplot2 is the fastest way to figure out what you want and make a pretty plot. (However one thing that still annoys me is that R moves faster than Debian. So it's possible to do install.packages() in R, and it will break telling you your Debian R interpreter is too old. There is no easy solution for this, just a bunch of workarounds) ----- OK, sure you can call it a polished turd, and to some degree that's true. But a polished turd is better than just using ... a turd! The error messages in R are not quite as good as Python, but I wouldn't call it a problem. I'm able to localize the source of an error, even when using tidyverse. My article comparing tidyverse to some other solutions: What Is a Data Frame? (In Python, R, and SQL) http://www.oilshell.org/blog/2018/11/30.html http://www.oilshell.org/blog/2018/11/30.html ---- But would I recommend learning it to anyone else? Absolutely not. We can do so much better. I would recommend with the caveat that it's one of the hardest languages I've had to learn. However that is partly because it changes how you think. But if you have a certain type of problem then you have to change how you think, or you'll never get it done. Data analysis is surprisingly laborious even for people who have say written compilers and such.
• newbamboo 5y ago
  “To put it plainly, R won’t change. If something about R frustrates you today, it always wilL.” LMFAO. I only read the above line and the bit about how tidyverse nearly fixes all the badness that is r. Max lol.
• katekoch 5y ago
  [flagged]
• deehouie 5y ago
  To rephrase the famous quote attributed to G. Box, `All languages are wrong, some are useful.` And R is one of them.
• jeffjobs4000 5y ago
  I first learned how to code in R before moving on to Python, then some C and Go. I think a big cause for the SWE hate of R is that it's not OO programing. R is a functional language for data analysis. If you don't grok that, then I can understand why looking at it would make you barf. Going the other way, from functional to OO, caused me physical pain as well. R is amazing for data analysis. Also, RStudio is a much more efficient solution for iteratively exploring data than Jupyter. Don't make fun of a screwdriver for not being a hammer.
  • klohto 5y ago
    Considering how loved Elixir is, functional definitely isn't a problem.
  • rafaelero 5y ago
    I made the same transition from R to Python and I still resist using OO. I never understood why people would use it instead of functions.
• tharne 5y ago
  R is a truly terrible language with a handful of bright spots, such as it's visualization libraries. The boost you get from the slightly better expressiveness of R over something like Julia or Python is not worth the headaches you'll run into down the road in trying to maintain whatever you wrote 6 months later, or God forbid, trying to integrate your code into someone else's work. R was my first language and in hindsight that was a HUGE mistake. So much of the R code out there is horribly written, and even when it isn't you still have to deal with all of the issues the author here points out. If you pick up R as your first language, you will end up picking up all sorts of bad habits; R is fine if you're working solo and you don't plan on maintaining or reusing or reusing your code. For everything else, R is garbage. It took me a year or more to undo all of the bad habits I picked up learning R. I don't agree with the "worse is better" comparison in the comments here. "Worse is Better" was meant to refer to the idea of "Don't make the perfect the enemy of the good", among other things. It was not meant to be used as a justification for poor design. If anything, python for data analysis fits the "worse is better" philosophy much better than R. It's not as well optimized for data work compared to R, but it's much simpler, more consistent, less error prone, and it plays well with others.
  • bscphil 5y ago
    Counterpoint: a lot of work in R is not "development" in the ordinary sense. The outcome is not a piece of maintainable code that needs to be built on later or be generally useful in any way other than copying an occasional snippet. In some research fields (e.g. scientific fields that use R) the ground rules are that the code needs to be understandable and it needs to be clear that the libraries involved were used correctly. That's basically it. Even hardcoded directories are common. Good development practices are not widely understood to be important and in general many people are just starting to get the hang of version control and might not use it at all. If R enables you to solve a statistical problem you have right now and it does this in a way that is better or more comprehensible for the people who use it, that means it has a niche. As someone with software development experience in a bunch of other languages, I agree with you that R is full of weird warts, but let's not forget that there are areas where its value is still obvious. Citation: my partner works in a scientific field where R is predominant.
• dxbydt 5y ago
  Lot of useful insights in the comments here. I wanted to address one specific comment - >can't remember the last time I saw a project someone did in R get very much traction anywhere...the only time people talk about R on the internet is to discuss the language itself which is definitely frustrating There is a lot of R deployed in industry, even in silicon valley, but you have to be in-the-know. R gets plenty of use in statarb & model checking in finance - speaking from personal experience at GS & BofA/ML. My one non-trivial project at Twitter involved working with this team building a model & I remarked - hey this can be done rather easily if you use this library in R - and the teamlead says, yeah that's how we're doing it! But I thought we are a Scala shop, I said. So he says, yeah but imagine building that entire library in Scala from scratch, it'll take forever! So I enquired how he gets it done - you basically spin up a socket server & the jvm sends R commands plus data as payload over the socket, the server runs R and returns the result of the model back as a string, boom done! I said it was kinda janky & he says - I won't tell if you don't ! So that's R for you - it gets the job done & its fast & somewhat messy, but it is used everywhere, yet people won't openly admit to it because its a 30 year old language & we all want to be using the latest & greatest tool. I now work at a news startup with a few million users, & all of the news personalization is done in R. So when these millions of viewers watch TV, the piece of code that decides which news clip should be shown ahead of which other news clip & which clip comes after - all of that is decided by a block of R code that I wrote. ~ 300 lines of R, uses quanteda, tidytext & parallel under the hood. Pretty much everything I do involves mcmapply, which parallelizes your compute & uses as many cores as you specify. But that's sort of the thing with R - you have to know which functions/libs to use & which ones to avoid. Just switching from tm to quanteda got us a 200% bump in perf. Switching sapply's to mcmapply was another winner. These things aren't documented cleanly - you have to keep up with cran, experiment & see what works best for you.
• JZL003 5y ago
  I've grown to really rely on R pipes, it's just how I like thinking about problems with small short lambdas. I wish python had better support (and better lambdas, without typing out 'lambda' and with tuple arguments) but for when I need to use it, toolz's pipe is pretty ok as long as you use lazy map/filter/etc It's just so close to being as good, but not quite
• saeranv 5y ago
  Personally, I feel like the biggest problems with Python for math is the lack of a native vector datatype and our subsequent reliance on Numpy which really disrupts the elegance/terseness of working with vectors/matrices. First, there's the constant inelegance/clutter/inefficiency of having to cast into and out of arrays and lists, even when doing basic list comprehension. R, Julia, and Matlab are all vector-based languages (I think), so you avoid having this casting as much. Secondly, having a native vector type means you don't have to worry about the performance penalty of operating directly on arrays if an existing prebuilt method exists. Since the efficiency of Numpy comes from calling it's underlying C library, you're forced to memorize and use prebuilt Numpy functions rather then just use the more obvious and elegant array manipulations. For example rather than calculating the the cumulative sum like this: cumsum = reduce(lambda a, b: a+b, [1,2,3,4]) We have do this: cumsum = np.cumsum([1,2,3,4]) (There are better examples, but this is all I can think of right now). And once you add something like Pytorch tensors on top of this, we now have an additionally layer of casting/redunduncy/memorization of prebuilt functions!
• sam-s 5y ago
  I am surprised no one has mentioned the awful [garbage collector](https://stackoverflow.com/q/14580233/850781 https://stackoverflow.com/q/14580233/850781). The R garbage collector is imperfect in the following (not so) subtle way: it does not move objects (i.e., it does not compact memory) because of the way it interacts with C libraries. (Some other languages/implementations suffer from this too, but others, despite also having to interact with C, manage to have a compacting generational GC which does not suffer from this problem).
  • mst 5y ago
    Honestly, probably because so many languages have non-compacting collectors that people just accept it as a trade-off - in the sense that compacting collectors are non-trivial and without lots of work can produce significantly higher GC pauses, and so doing it really well requires a lot of engineering effort that you might prefer to be spent elsewhere adding features you want more. golang's collector isn't compacting either - though it uses per-size-class arenas for allocation so you don't end up with fragmentation bloat to nearly the same extent. Part of me wonders if simply building R against jemalloc would get a decent chunk of the same advantages.
• exdsq 5y ago
  I've recently started getting into computational archaeology and found the entire ecosystem is built around R, meaning I am now starting to learn about it. Anyone have a suggestion of the standard books/courses one should start with? I found it pretty interesting that the alternative to R is Haskell for general CLI tools! Seeing some open issues in a popular tool for dealing with ancient DNA (aDNA) about making invalid states impossible within the type system made me genuinely laugh out loud in amazement. I didn't expect that level of technical knowledge within the world of archaeology.
  • mst 5y ago
    Any chance of sharing which tool(s)? I know some people who're involved in making the haskell ecosystem a better place for less mainstream users and I'm pretty sure they'd want to know more about this. (and if it turns out they already know, -I- don't, and that sounds pretty cool and fun to read up on :)
    • exdsq 5y ago
      Sure! It’s the department of archaeogenetics at the Max Planck Institute for Evolutionary Anthropology https://poseidon-framework.github.io/#/ https://poseidon-framework.github.io/#/
      • mst 5y ago
        Much appreciated, passed on in the hopes it's useful, and when it's not 0030 and I'm awake enough to actually understand any of it I'll be having a read through myself. Cheers!
        • exdsq 5y ago
          I’m a total noob but offered my development skills in trade of archaeological domain knowledge, so just getting familiar with the code base myself atm :) if you/your friends wanted any sort of intro to the team I could arrange that as I’m in a slack channel with some of them! Fwiw, it’s mostly a tool for parsing and processing this ‘Jan no’ data schema https://poseidon-framework.github.io/#/janno_details https://poseidon-framework.github.io/#/janno_details
• dash2 5y ago
  OT: does anyone know what happened to rweekly.org? It hasn't been updated in a long time and I don't know a good substitute.
• kgwgk 5y ago
  It's a good reading but some of the complaints are hard to understand. For example, in 4.5.1: Selecting and deleting at the same time doesn’t work either. For example, data[c(-1, 5)] is an error. What would it mean for that to work? He seems to acknowledge that "selecting and deleting at the same time" doesn't make sense in 4.11.1 Can you guess what data[-1:5] returns? I can’t either, so don’t ever try it. If you must know, it’s actually an error. Also in 4.11.1: The : operator is absolutely lovely… until it screws you. The solution is to prefer the seq() functions to using : [....] As I’ve said, seq() and its related functions usually fix this issue. Maybe the "related functions" fix some issues but seq(a,b) is not different from a:b In 4.11: Now what do you think names(foo) <- names(bar) does? Seriously, can you guess? I can think of roughly four realistic guesses. Is it even valid syntax? How is that surprising? Can the author also think of four realistic guesses about the effect of A[1,2] <- B[3,4] for example? In 4.13: The index in a for loop uses the same environment as its caller, so loops like for(i in 1:10) will overwrite any variable called i in the parent environment and set it to 10 when the loop finishes. [...] This sounds awful, but I’ve never encountered it in practice. Is it awful? The same happens in other languages like Python or C if I'm not mistaken. The plot() function has some strange defaults. For example, you need to have a plot before you can plot points [...] I have no idea what that means. You can plot points using plot() without having a plot beforehand. Edited to add: In 4.5.3: The $ operator is another case of R quietly changing your data structures. Is it unexpected that when we extract an element from a data structure we get a different kind of data structure? Is A[1,1] another example of silently changing one data structure (matrix) to another (number)?
  • mst 5y ago
    > Is it awful? The same happens in other languages like Python or C if I'm not mistaken. It annoys the shit out of me in python. I much prefer perl's for my $x (@array) { ... } or ES6's for (let x of array) { ... } Note that given python and ruby only do function level scoping rather than block level, I can -understand- why they work the way they do even if it annoys me. R already has the necessary granular scoping to do the (IMNSHO) sensible thing so it seems like a pointless wart. My -guess- would be that if it was intentional, it came about in R because for loops are rare enough that you want to know the last index more often than you don't because if you don't care about the index presumably you'd've written something else. (also you could argue the ES6 version would be better written using 'const', but I've lisped sufficiently my fingers invariably generate 'let' when left to their own devices - caveat emptor)
• antipaul 5y ago
  It would help to know this person's background a bit more. Is this a former software engineer, or a manager trying to pick up a technical skill? That info could help contextualize the entire piece.
• racl101 5y ago
  I gave up on R immediately when installing on a Macbook was a nightmare and especially when I was already proficient in Pandas and Numpy.
• victoryhb 5y ago
  I find R's strengths lie in its unmatched collection of statistical libraries, but I dislike R's syntax so much that, if forced to use it, would call an R package from Python (using RPy2), or just use a Python alternative (e.g. Plotnine).
• rlh2 5y ago
  I have been using R for almost 20 years now. I work on a medium-sized quant team at a large asset manager and we run several $BN off R - we mostly trade equities and vanilla derivatives. Our models are primarily statistical/econometric-based. In aggregate, we probably have about a hundred scheduled jobs associated with a variety of models and on the order of 15 shiny applications to facilitate implementation. We have an internal CRAN-like repo and everything we produce is packaged/versioned with gitlab CI/CD. We have RStudio Server at my firm and half my team uses that for development, the other half, including myself, uses emacs/ess. All of us use RConnect for scheduling & application hosting - it has it's quirks, but it's excellent in a constrained IT environment. I often chuckle when people complain about R in production and how it isn't a good general purpose programming language, my experience has been the polar opposite. You can write bad code in any language, and R is no exception, but R allows you to write so much less code and R-core is truly exceptional at backwards compatibility. Our approach to R is basically: - Don't have a lot of dependencies, and when you do have dependencies, make sure they themselves don't have a lot of dependencies. While we do use shiny as mentioned above, our core models are very dependency light and shiny is just a basic front end. - data.table (which was designed by quants) is a zero-dependency package that is by far the best tabular data manipulation package that has ever been created since the dawn of time. We generally work on an EC2 instance running linux with a ton of memory. In the < .01% of cases where a dataset doesn't fit in memory (e.g. tick data), we do initial parsing with awk if file based or SQL if DB based and then work in R. - Check/coerce argument types and lengths on function input to catch and avoid all the quirky edge cases that drive people nuts - it's so easy! - I hate OOP and I love that R doesn't encourage it. Mutable state, especially for non-software engineers, is the devil. Don't get me wrong, OOP has its place, but the fact that R encourages functional programming is one of the best things about it. The slight inefficiency this produces is almost never a problem. - R is not slow at all when used correctly. Additionally, the C API is a joy to use when necessary. - Stick to the base types: vectors, matrices, lists, environments and data.tables (only exception). The fact that you can name, and then use names to index all of the above is stunningly powerful. The only "objects" we really create are lightweight extensions of lists with an S3 print method. - We have an internal version of renv/packrat that creates a plain text "dependency file" for projects and we pin package versions in docker containers. RConnect doesn't use docker right now, but they do have a versioning system that works quite well in my experience. I definitely wouldn't want to build something like a company website in R, but I also wouldn't want to build that in C either. R definitely has it's place a server-side language even outside it's assumed domain of statistics. Haters gonna hate, but joke is on them.