8 ms·
Big Book of R
- madcaptenor 2y agoI've made some half-hearted attempts to build something like this and I'm glad to see someone tried harder than I did. Thanks! One comment: it would be good to distinguish between books that are free and books that you have to pay for.
- oscarbaruffa 2y agoThanks! Paid books do note (above the link) that they're paid but I agree, a better visual might help. I'm thinking of removing the paid books where many free alternatives are available
- wpollock 2y agoVery nice, but instead of an owl, shouldn't the cover illustration be a pirate?
- oscarbaruffa 2y agoAh, good one ;). Maybe in future I'll change it
- madcaptenor 2y agoSadly, the R community has never really embraced the pirate thing.
- esafak 2y agoStatisticians don't really embody the pirate spirit, do they :)
- bryanrasmussen 2y agoThe average Statistician doesn't, but the mean ones do.
- DadBase 2y agoTotally agree. R is pure pirate energy. Half the functions are hidden on purpose, the other half only work if you chant the right incantation while facing the CRAN mirror at dawn.
- account-5 2y agoI've never used R before, why would functions be hidden on purpose? Sounds like a recipe for frustration.
- Hasnep 2y agoDon't worry they're just a bot. R doesn't hide functions.
- DadBase 2y agoNot a bot, friend, just someone who’s chased too many bugs through too many layers. mean() is just one example: a polite front door. The real labor’s in mean.default, tucked out of sight like a fuse behind drywall.
- Hasnep 2y agoThat's just how R does method dispatch, if you run methods(mean) you get the list of methods: [1] mean.Date mean.POSIXct mean.POSIXlt mean.default mean.difftime see '?methods' for accessing help and source code That's not hiding anything it's just abstraction, bot
- DadBase 2y agoI’m not saying R hides things. Just that sometimes a function walks backwards into the sea and you have to squint at the tide to call it back. It’s not deception, it’s how the language dreams.
- 2y ago
- hcarvalhoalves 2y agoYaRrr! The Pirate’s Guide to R https://bookdown.org/ndphillips/YaRrr/ https://bookdown.org/ndphillips/YaRrr/
- LostMyLogin 2y agoNot to be confused with The Book of R: https://www.amazon.com/Book-First-Course-Programming-Statistics/dp/1593276516 https://www.amazon.com/Book-First-Course-Programming-Statist...
- thangalin 2y agoTangentially, R can help produce living Markdown documents (.Rmd files). A couple of ways include pandoc with knitr[0] or my FOSS text editor, KeenWrite[1]. I've kept the R syntax in KeenWrite compatible with knitr. Living documents as part of a build process can produce PDFs that are always up-to-date with respect to external data sources[2], which includes source code. [0]: https://yihui.org/knitr/ https://yihui.org/knitr/ [1]: https://keenwrite.com/ https://keenwrite.com/ [2]: https://youtu.be/XSbTF3E5p7Q?list=PLB-WIt1cZYLm1MMx2FBG9KWzPIoWZMKu_&t=14 https://youtu.be/XSbTF3E5p7Q?list=PLB-WIt1cZYLm1MMx2FBG9KWzP...
- juujian 2y agoLast time I was working on something complex, I was able to knit from Rmd to md, and then use my usual pandoc defaults, which was quite neat. Big recommendation on that workflow.
- thangalin 2y agoMy typesetting Markdown series explores weaving knitr and pandoc together: https://dave.autonoma.ca/blog/2019/07/11/typesetting-markdown-part-6/ https://dave.autonoma.ca/blog/2019/07/11/typesetting-markdow... However, most workflows and nearly all editors don't support interpolated variables. To address this, first I developed a YAML preprocessor: https://repo.autonoma.ca/yamlp.git https://repo.autonoma.ca/yamlp.git Then I grew tired of editing YAML files, piping files together, and maintaining bash scripts. So next, I developed KeenWrite to allow use of interpolated variables directly within documents from a single program. The screenshots show how it works: https://keenwrite.com/screenshots.html https://keenwrite.com/screenshots.html
- haberman 2y agoThere is also Quarto, which I have had a good experience with: https://quarto.org/ https://quarto.org/
- countrymile 2y agoR is beautiful for writing data rich books and websites. I started with rmarkdown but believe that most of the new developments are now in quarto?
- kingkongjaffa 2y agoWhat is the best way to integrate some R code with a python backend? I’ve been tempted to port to python, but some of the stats libraries have no good counterparts, so, is there a ergonomic way to do this?
- jmalicki 2y agoDo you dislike rpy? I've found it to be pretty easy to use.
- bachmeier 2y agoNot sure what you mean by "python backend". If you mean calling R from Python, rpy2 mentioned in the other comment works well. If you mean the other direction, RStudio has this all built in. This is probably the best place to start: https://rstudio.github.io/reticulate/articles/calling_python.html https://rstudio.github.io/reticulate/articles/calling_python...
- jjr8 2y agoThere is also https://www.rplumber.io/ https://www.rplumber.io/, which lets you turn R functions into REST APIs. Calling R from Python this way will not be as flexible as using rpy2, but it keeps R in its own process, which can be advantageous if you have certain concerns relating to threading or stability. Also, if you're running on Windows, rpy2 is not officially supported and can be hard to get working.
- huijzer 2y agoCSV is generally the answer. Unless you need superb performance which generally is not the case.
- malshe 2y agoOne of my students codes exclusively in Python. But in most cases newer econometrics methods are implemented in R first. So he just uses rpy2 to call R from his Python code. It works great. For example, recently he performed Bayesian synthetic control using the R code shared by the authors. It required stan backend but everything worked.
- hughess 2y agoThis is great - I used to use R all the time when I worked in finance and wish I had this resource back then! R and RMarkdown were big inspirations for what we're building at evidence.dev now, so very grateful to everyone involved in the R community
- brcmthrowaway 2y agoDoes R support LLM?
- countrymile 2y agoThere are packages for that. Copilot is well integrated in Rstudio.
- hadley 2y agoI've wrapped a bunch of providers with ellmer: https://ellmer.tidyverse.org https://ellmer.tidyverse.org
- kgwgk 2y agoWhat does it mean for a programming language to “support LLM”?
- vharuck 2y agoI also like this fun though dated handbook, full of gotchas common among new R programmers: https://www.burns-stat.com/pages/Tutor/R_inferno.pdf https://www.burns-stat.com/pages/Tutor/R_inferno.pdf
- fn-mote 2y agoDated is right. The invention of the Tidyverse freed new R programmers from 126 pages of gotchas. Tell them to learn to use the tidyverse instead. For most of them, that will be all they ever need.
- uptownfunk 2y agoI will say, now after 15 years messing with this. With LLM I just do it all in Python. But, I still miss the elegance and simplicity of R for data manipulation and analysis. Especially the dplyr semantics. They really nailed it. I think they got crushed by the namespace / import system. There’s something about R that makes you so fluid and intuitive. But the engineering, the efficiency, I get with Python now, I can’t go back.
- dkga 2y agoI agree with all your comment… except the very last bit. Do you really find python to be more efficient at engineering stuff than R? And especially speed, which in my experience at least is broadly the same if not faster with R because it interages easier with Rust and C++?
- claytonjy 2y agoNot OP, but i think python is very far above R for engineering stuff. I built my early career on R and ran R user groups. R is great for one-off analyses, or low-volume controlled repetition like running the same report with new inputs. For engineering stuff i want strong static analysis (type hints, pydantic, mypy), observability (logfire, structlog), and support (can i upload a package to my cloud package registry?). For ML stuff, i want the libraries everyone else uses (pytorch, huggingface) because popularity brings a lot of development and documentation and obscure github issues the R clones lack. Userbase matters. In R, hardly any users are doing any engineering; most R code only needs to run successfully one time. The ecosystem reflects that. The python-based ML world has the same problem, but the broader sea of python engineers helps counterbalance.
- uptownfunk 2y agoEverything I need can get done in python, so I don’t even need to deal with rust and cpp. Adding language interop between r and cpp is now just another thing on my plate, so just stick to Python and pay the cost of less elegant code for data manipulation which I am okay with because now I just need to read it and not write it. There’s a ton more python code out there so the LLM reliability in python code just makes my life easier. R was great and still is, but my world is now more than just data eng, model fitting, and viz. I have to deal with operationalizing and working with people who aren’t just data science and most org don’t have the luxury of having an easy production R system so I can get my python code over the line and trust a good engineer will be okay smeshing that into the production stack which is likely heavy Python. (Instead of saying oh we don’t work with R we do Python Java so it will take 3-5x longer). Another sad truth is the cool ml kids all want to do pytorch deep ML training / post training / rlhf / ppo / gdpr gtfo so you are not real hardcore ml if you only do R. I know it’s stupid but the world is kind of like that. You want to hire people who want to build their careers on the cool stack. I know it’s not all the cool talk the hackers here play with but for real world application I have a lot of other considerations.
- cye131 2y agoR especially dplyr/tidyverse is so underrated. Working in ML engineering, I see a lot of my coworkers suffering through pandas (or occasionally polars or even base Python without dataframes) to do basic analytics or debugging, it takes eons and gets complex so quickly that only the most rudimentary checks get done. Anyone working in data-adjacent engineering work would benefit from R/dplyr in their toolkit.
- kasperset 2y agoI love R and dplyr. It is very readable and easy to explain to non-programmers. I use it almost everyday. Not exactly on the topic,I am having difficulties debugging it. May be I need to brush up on debugging R. Not sure if there is a easy way to add breakpoint when using vscode.
- JackeJR 2y agobrowser() ?
- disgruntledphd2 2y agotrace subsumes browser, it's much more flexible and can be applied to library code without editing it.
- tylermw 2y agotrace is great for shimming in your own code to an existing function, but it’s not an interactive debugging tool.
- disgruntledphd2 2y agoIt sure is. If you set the second argument to browser you can step through any function.
- wdkrnls 2y agoIs there a way to trace an attribute to a function? I couldn't find one, but curious if it exists. I seemed blocked by the fact that trace seemed to expect a name as a character string. Some functions in base R have functions in their attributes which modify their behavior (e.g. selfStart). I ended up just copying the whole code locally and then naming it, but for a better interactive experience I really wish there was a way to pass a function object as I can with debug.
- gsf_emergency_2 2y agoAny Julians comment? Having seen Julia proposed as the nemesis of R (not python, that too political, non-lispy) >the creator of the R programming language, Ross Ihaka, who provided benchmarks demonstrating that Lisp’s optional type declaration and machine-code compiler allow for code that is 380 times faster than R and 150 times faster than Python (Would especially love an overview of the controversies in graphics/rendering) https://news.ycombinator.com/item?id=42785785 https://news.ycombinator.com/item?id=42785785
- barrenko 2y agoFor data analysis and visualization R is the lightsaber.
- Hasnep 2y agoIn my opinion, Julia has the best alternative to dplyr in its Dataframes.jl package [1]. The syntax is slightly more verbose than dplyr because it's more explicit, but in exchange you get data transformations that you can leave for 6 months and when you come back you can read and understand very quickly. When I used R, if I hadn't commented a pipeline properly I would have to focus for a few minutes to understand it. In terms of performance, DF.jl seems to outperform dplyr in benchmarks, but for day to day use I haven't noticed much difference since switching to Julia. There are also APIs built on top of DF.jl, but I prefer using the functions directly. The most promising seems to be Tidier.jl [2] which is a recreation of the Tidyverse in Julia. In Python, Pandas is still the leader, but its API is a mess. I think most data scientists haven't used R, and so they don't know what they're missing out on. There was the Redframes project [3] to give Pandas a dplyr-esque API which I liked, but it's not being actively developed. I hope Polars can keep making progress in replacing Pandas, but it's still not quite as good as dplyr or even DF.jl. For plotting, Julia's time to first plot has got a lot better in recent versions, from memory it's something like 20 seconds a few years ago down to 3 seconds now. It'll never be as fast as matplotlib, but if you leave your terminal window open you only pay that price once. I actually think the best thing to come out of Julia recently is AlgebraOfGraphics.jl [4]. To me it's genuinely the biggest improvement to plotting since ggplot which is a high bar. It takes the ggplot concept of layers applied with the + operator and turns it into an equation, where + adds a layer on top of another, and the * operator has the distributive property, so you can write an expression like data * (layer_1 + layer_2) to visualise the same data with two visualisations. It's very powerful, but because it re-uses concepts from maths that you're already familiar with, it doesn't take a lot of brain space compared to other packages I've used. [1] https://dataframes.juliadata.org/ https://dataframes.juliadata.org/ [2] https://github.com/TidierOrg/Tidier.jl https://github.com/TidierOrg/Tidier.jl [3] https://github.com/maxhumber/redframes https://github.com/maxhumber/redframes [4] https://aog.makie.org/ https://aog.makie.org/
- oscarbaruffa 2y agoI'm the curator of Big Book of R and am really happy to see it on the front page of HN :). New books are added every 6 weeks or so and I send a notifications of the new adds to my newsletter subs. Link is at the footer of every page
- marginatum 2y agoWell done Oscar. I got you a$5 coffee as with the economic crisis I don't think you'll find a good $2 one.
- oscarbaruffa 2y agoThank you :)
- loa_observer 2y agoI hope gwalkr can be added to this book, it's pretty intereting updates for visualizations in R recent years. repo: https://github.com/Kanaries/GWalkR https://github.com/Kanaries/GWalkR site: https://kanaries.net/gwalkr https://kanaries.net/gwalkr
- dikip 2y ago[dead]
- Annatar 2y ago[dead]
- ebri 2y agoBeen working 8 years with Rs data.table package in research and now after I changed to the private sector I have to use python and pandas. Pandas are so terrible compared to data.table it defies belief. Even tidyverse is better than pandas which is saying something. I miss it so much
- fhsm 2y agoUse it every single day. Absolutely fantastic tool.
- qomioo 2y ago[flagged]