9 ms·
R Markdown: The Definitive Guide
- halfeatenpie 6y agoI've always been a big fan of R Markdown. For me and my work, there's always been a fine line between R Markdown and R Shiny, with certain limitations (or something I felt was a limitation) in R Markdown could be overcome by building it in R Shiny.
- RobinL 6y agoAs an occasional R user, I think R markdown one of the things that R does really well. For data scientists who want to output reports (mix of text and calculations), I haven't come across anything as mature or easy to use in the Python ecosystem. I'm a big user of Jupyter notebooks, but version control issues put me off, and I've never got to grips customising the formatting of jupyter nbconvert. It's worth highlighting that R Markdown actually supports a number of languages including Python - for instance see here: https://bookdown.org/yihui/rmarkdown/language-engines.html https://bookdown.org/yihui/rmarkdown/language-engines.html For example, I am currently considering using R markdown to author the documentation for one of my Python packages. Currently my documentation is written in markdown, but I keep having to copy and paste calculations and tables into the markdown. When things change, it would be nice to just be able to re-run the R markdown, converting Rmd -> md and thereby interpolating the latest version of all calculations into the final markdown doc.
- spinningslate 6y agoI'd strongly agree with this. I use R Markdown with Python in the Rstudio IDE, and prefer it to Jupyter. The publishing pipeline is really flexible, e.g. with one-click deployment to RStudio Server [0] (which is a paid product). No affiliation to RStudio, just a happy customer. [0]: https://rstudio.com/products/rstudio/download-server/ https://rstudio.com/products/rstudio/download-server/
- gpoore 6y agoFor the Python in markdown case, you might be interested in one of my projects that allows executable Python code (including optional Jupyter kernel support) in Pandoc markdown: https://github.com/gpoore/codebraid https://github.com/gpoore/codebraid. Pandoc does all the document parsing (there is no regex preprocessor for extracting code), so converting markdown to markdown often works particularly well.
- jdgoesmarching 6y agoCurrently working on a DA masters with a background in markdown and was so happy to discover this. Then I was baffled to discover that there wasn’t an equivalent for Python. Jupyter is great, but I really love the simplicity of my text files and I’m surprised that there isn’t a major implementation given how popular both markdown and Python are. I guess notebooks are just too appealing.
- Tarq0n 6y agoHonestly jank and bad ergonomics pervade the Python data science ecosystem. The pandas API is kind of bad (multi-indexes anyone?) matplotlib is way too low level, tensorflow is a shitshow. I think the R community got lucky with a greater focus on user experience, even if it comes at the cost of explicitness sometimes (lots of DSL's with a lot of 'magic').
- CornCobs 6y agoHonestly, in my experience, R's DSLs are not a bad thing. When you are doing analytics day in and out you basically are speaking the DSL anyway, so what if you have to learn this slightly different language if it expresses what you want to accomplish clearly?
- StarlaAtNight 6y agoit was pretty wild to hear Hadley Wickham invoke Cognitive Behavioral Therapy in his most recent rstudio::conf keynote - dude is super in tune with how humans use open source data analysis tools
- c6401 6y agoHydrogen for Atom actually can have executable python in markdown, and more - write python in .py files and execute strings or define blocks by "# %%". It's not the same of course but for my usecases it's more convenient (given that it works in a proper code-editor with all needed plugins).
- mespe 6y agoI've been using Rmarkdown and knitr for a long while, and have watched its evolution over the years (roughly 8 years now). As someone who does not use RStudio, it's become a bit of a pain for me to use. The authors seem to expect it is being used from RStudio, and using it in a different environment has become a bit fragile. It's also a bit telling that this "definitive guide" does not include any troubleshooting/debugging sections - the expectation seems to be that it "just works" so long as you use it in RStudio, but otherwise you are on your own. Not sure that I am aware of many other R packages with this mentality, but I am personally not a fan.
- ska 6y agoI haven't used it for a few years, but I have previously set up some purely command line tooling for this (autogenerated verification reports) which worked well. Is this getting harder to do?
- mespe 6y agoSome functionality has been tied to RStudio's concept of a "project" (https://support.rstudio.com/hc/en-us/articles/200526207-Using-Projects https://support.rstudio.com/hc/en-us/articles/200526207-Usin...). I have had several documents authored by others which would not build on my system without some intervention due to this.
- clircle 6y agoI write my blog mostly in Rmarkdown from Emacs using ESS/polymode. Mostly works well accept for some markdown-mode induced bugginess. I'm using it occasionally at work as well, but I still prefer R scripts with nice comments for work.
- minimaxir 6y agoR Markdown/Notebooks, IMO, has evolved to be the key value proposition of RStudio over the years. From a productivity standpoint when both working with R and publishing pretty reports/PDFs it has been incredible. (that said, if VS Code gets more robust R Markdown support, I may consider switching from RStudio) I wrote a blog post a few years ago comparing R Markdown/Notebooks to Jupyter Notebooks: https://minimaxir.com/2017/06/r-notebooks/ https://minimaxir.com/2017/06/r-notebooks/
- mespe 6y agoFunny thing is this has actually motivated me to move away from using rmarkdown. To the point that I went "backwards" and wrote several reports and an academic paper in Sweave after using rmarkdown for years. My primary motivation was that I needed to know the documents could be built by any R session, not just from within RStudio. I didn't want to be in a situation where my "reproducible" document relied on a specific IDE to build correctly.
- vharuck 6y agoWhich parts don't work outside of R Studio for you? Is it something recent? I usually create my single-page and bookdown documents as one step in a separate R script, and haven't found any problems beyond the rmarkdown package's poor documentation (I really wish the R Studio developers would write in-package documentation with as much care as with their online documentation).
- mespe 6y agoIf I recall correctly, I had issues primarily with more complicated documents, and mostly due to some RStudio "project" and file path configuration issues - I would have to look to see what the specific issues were. Simple, one-file .Rmd docs are typically fine either way.
- phillc73 6y agoI'm all for finding alternatives to RStudio's dominance in the R space. No doubt they do some excellent things, but I also have a vague uneasiness at the cultishness surrounding some of their products. It sometimes feels like if you're not using the 'tidyverse', you're viewed as doing it wrong. Anyway, this feeling set me off on an exploration of alternative tools and packages. Instead of rmarkdown I'm now exploring the pander[1] package, which seems to do most of what I'm looking for, perhaps only a little limited in output formats. Edit: The Pandoc.brew examples might be most interesting from a direct alternative to rmarkdown for document creation context.[2] [1] http://rapporter.github.io/pander/ http://rapporter.github.io/pander/ [2] http://rapporter.github.io/pander/#examples http://rapporter.github.io/pander/#examples
- andylynch 6y agoVery nice to see this here. R Markdown is a great tool on is own and working with it in RStudio is a superpower, plus the clean format is really nice in version control.
- bnj 6y agoTangentially: Reading [0] from Stephen Wolfram yesterday got me digging into Wolfram notebooks. Does anyone have experience or reflections on using R Notebooks as the main way to keep track of notes? I don't see the Wolfram products talked about very much here [0]: https://writings.stephenwolfram.com/2019/02/seeking-the-productive-life-some-details-of-my-personal-infrastructure/ https://writings.stephenwolfram.com/2019/02/seeking-the-prod...
- nojito 6y agoUsing RMarkdown effectively pretty much tripled my salary. It's an amazing tool that is wholly underappreciated in the "data" space.
- quantstats 6y agoCould you expand on what you mean by using it effectively? I've just started using RStudio and any tips/ways to use it more efficiently are welcome! Thanks.
- ct0 6y agoI could imagine the time invested on making the same report in another language takes 3x.
- hinkley 6y agoSomething I find that people don't appreciate about better tools is that it's not just about going faster, it's about going farther. There's a whole bunch of stuff you can do by hand. But if you do it by hand, you'll compromise your sense of satisficing. It amazes me how often the 'work harder' crowd doesn't reconcile the two. If you can quickly and reliably figure out we're spending $100k a month, you get a raise. If you can figure out why it's $100k instead of $80k, you might be up for a promotion.
- nojito 6y agoFind a business problem in your job that requires information to get into the hands of many people but require tweaks between them. i.e. person X needs to know A but person Y needs to know B. Read up the section on parameterized reports and realize you can do the work of dozens . https://bookdown.org/yihui/rmarkdown/parameterized-reports.html https://bookdown.org/yihui/rmarkdown/parameterized-reports.h... rmarkdown is insanely effective at getting information out of your hands and into stakeholders repeatedly and in an visually appealing way. When you combine that with your domain knowledge, your value to your company will increase.
- 6y ago
- deleted 6y ago[deleted]
- runningmike 6y agoJupyter Book is more flexible, simpler and tested to death since it is build upon Sphinx and docstrings. Since jupyter uses pandoc for converting notebooks, pandoc is the power tool for publication creations nowadays.
- spinningslate 6y agoJupyter has downsides too, e.g. version control. An R Markdown document "in the raw" is just a text file. For info, Rmd also uses pandoc under the covers. Agree it's the power tool, quite a wonderful thing.
- qbasic_forever 6y agoJupyter Book (and the larger jupyter ecosystem now) are starting to standardize on MyST notebooks as a version-control friendly all text notebook format. It's similar to r markdown and uses fenced code blocks to represent code cells. You can diff them, edit in any text editor, etc. See: https://myst-nb.readthedocs.io/en/latest/ https://myst-nb.readthedocs.io/en/latest/
- ideamotor 6y agoI haven't use it myself but apparently jupytext (https://github.com/mwouts/jupytext https://github.com/mwouts/jupytext) allows you to save Jupyter notebooks as R markdown files.
- closed 6y agoI'm familiar with both Rmarkdown and Jupyter Book. Rmarkdown also uses pandoc. Both are very flexible.
- cwyers 6y agoRMarkdown uses pandoc too, and it's on version 2.6, while Jupyter Book is pre-1.0.
- randomfool 6y agoPlug that if you're a Jupyter user to chime in on the survey at https://www.surveymonkey.com/r/LCB7GBF https://www.surveymonkey.com/r/LCB7GBF. Background: https://blog.jupyter.org/survey-jupyterlab-and-beyond-88c7fbd27a79 https://blog.jupyter.org/survey-jupyterlab-and-beyond-88c7fb...
- kasperset 6y agoThis has been discussed before but I am watching the development of https://github.com/fonsp/Pluto.jl https://github.com/fonsp/Pluto.jl so that it can provide some solid alternative.
- phillc73 6y agoIf you're interested in Pluto.jl, I recently saw an announcement about Neptune.jl[1], which appears to remove some of the reactive nature of Pluto. It's a fork of Pluto and executes code sequentially one cell at a time instead. An interesting read anyway and maybe worth trying. [1] https://github.com/compleathorseplayer/Neptune.jl https://github.com/compleathorseplayer/Neptune.jl
- tfehring 6y agoHuh. I wonder what the impetus for that was and what the purported advantages over Jupyter are, since reactivity seems like the main value add of Pluto.jl - and a significant one at that.
- melling 6y agoI recently discovered R Markdown. Started doing the ISLR examples with it. https://github.com/melling/ISLR/blob/main/chapter08/08_Lab02_melling.pdf https://github.com/melling/ISLR/blob/main/chapter08/08_Lab02... https://github.com/melling/ISLR/blob/main/chapter08/08_Lab02_melling.Rmd https://github.com/melling/ISLR/blob/main/chapter08/08_Lab02... I need to figure out how to better fit images so I don’t have pages with large gaps Also, you can now embed executable Python in the files.
- data_ders 6y agoThis is the first HN thread about R where I don’t see a commenter shitposting about how “terrible” R is. Maybe that’s a testament to all the great work done by the great Yihui Xie and the RStudio folks. Currently using RStudio to write my masters papers in markdown and convert them to PDF. Love it!
- yihui 6y agoThanks for the kind words (about me and RStudio)! Well, you might be too optimistic about people not commenting on R. Usually I tend to avoid reading HN, but I feel pretty much every single time when someone brings up anything related to R, some people will start mentioning how terrible R is, sooner or later :) Now people also start fighting around Tidyverse, although I have no idea why that's relevant to R Markdown...
- StarlaAtNight 6y agogithub really needs to add R Markdown Notebook rendering support (that's the ones that end in extension .nb.html)
- yihui 6y agoWe have tried for almost four years without success: https://github.com/rstudio/rmarkdown/issues/1020 https://github.com/rstudio/rmarkdown/issues/1020 We did hear back from Github for a few times, but there has been no sign of progress at all. We will truly appreciate it if anyone can connect us to a Github contact to push this forward.