8 ms·
Advantages of Using R Notebooks for Data Analysis Instead of Jupyter Notebooks
- jbmorgado 9y agoThe team working on Spyder (the Python closest alternative to RStudio) have something like R Notebooks in their roadmap for a while now, but it keeps being pushed into the future. I wish they could use RStudio for a while and understand just how important is the feature for someone using Python for research.
- rscho 9y agoClosest alternative to RStudio (on principle, if not in practice): http://rodeo.yhat.com http://rodeo.yhat.com
- neaden 9y ago"No cell block output is ever truncated. Accidentally printing an entire 100,000+ row table to a Jupyter Notebook is a mistake you only make once." Hah, sadly this is not the case for me.
- radarsat1 9y agoYeah that sounds like something that should at least be possible to fix pretty easily using css max height and an overflow=scroll setting.
- williamstein 9y agoJupyter now has output rate limitations (I pushed for them to add them), though I think they may be off by default. I also implemented something better (I think), which is an output buffer, for CoCalc's version of collaborative Jupyter notebooks. Instead of rate limiting, CoCalc saves only the last part of the output, discarding earlier output, and provides a link to get it.
- lima 9y agoWow, CoCalc is really cool. I've often found myself wishing for a collaborative environment like that... and there it is. Thank you!
- MichailP 9y agoCan someone here recommend good practical statistics book? Something with modern methods, but explained sufficiently in depth?
- JumpCrisscross 9y agoEach field has their own "good practical statistics book". I work in finance and so recommend Fabozzi. It's good, but so are many other foundational texts. Your requirement for practicality necessarily negates a one true answer.
- reacharavindh 9y agoI am currently going through this free book : https://www.openintro.org/stat/textbook.php?stat_book=os https://www.openintro.org/stat/textbook.php?stat_book=os
- neaden 9y agoWhat's your background and what exactly do you mean by modern methods? An Introduction to Statistical Learning is good and you can download the pdf: http://www-bcf.usc.edu/~gareth/ISL/ http://www-bcf.usc.edu/~gareth/ISL/ it assumes you have a pretty decent background in mathematics though.
- ryankennedyio 9y agoIt will depend a lot on your field, but a solid grasp of fundamental probability theory should be applicable everywhere. I think this is an excellent overview [1]. Learning probability from a measure theory angle is more difficult to grok compared to the frequentist approach everyone is more familiar with, but I found it much more enjoyable. (I learnt the usual way from doing computer science undergrad, but now re-doing it more rigorously for masters in financial engineering) [1]: http://www.math.uah.edu/stat/index.html http://www.math.uah.edu/stat/index.html
- tnecniv 9y agoWhat can you do with the measure theoretic foundations vs the traditional approach? I know graduate classes typically take the measure theoretic approach, but it's never been explained to me why.
- vthriller 9y agoRelevant discussion: https://news.ycombinator.com/item?id=14035092 https://news.ycombinator.com/item?id=14035092
- baldfat 9y agoI switched from Python to R about 3 years ago. I missed iPython (Now Jupyter) for a long time. Then I just got attached to RStudio. I have tried R in Jupyter a few times and it was nice but the advantages in R Notebooks is just awesome. Git playing nice is the best advantage. I still am clueless to the religious Python vs R and the smack that is read that "serious" work is done on in Python? R works best for me.
- thearn4 9y agoOut of curiosity, what was the motivation for switching from Python to R for analysis, was there a particular R package that you were looking to use?
- jjn2009 9y agoI can't speak for top comment but I started learning R and realized that a lot of the primitives which are exposed in python as libraries are just primitives in R and are thus are more natural to use in subtle ways. Once you start thinking in R you think in data and statistics rather than how you deal with data and statistics within a language. This doesn't mean one is actually better than the other unless you want to do generic programing things along your math oriented code then probably python is better but I find this change of mental state to be useful when focusing on problems R is suppose to solve.
- int_19h 9y agoR has some really crazy metaprogramming facilities. This might sound strange coming from Python, which is already very dynamic - but R adds arbitrary infix operators, code-as-data, and environments (as in, collections of bindings, as used by variables and closures) as first class objects. On top of that, in R, argument passing in function calls is call-by-name-and-lazy-value - meaning that for every argument, the function can either just treat it as a simple value (same semantics as normal pass-by-value, except evaluation is deferred until the first use), or it can obtain the entire expression used at the point of the call, and try to creatively interpret it. This all makes it possible to do really impressive things with syntax that are implemented as pure libraries, with no changes to the main language.
- cosarara97 9y agoSage deserves a mention: http://www.sagemath.org/ http://www.sagemath.org/
- cle 9y agoOnce I got serious about this stuff, I started using Emacs with Org Mode and ESS. It has most of the features listed here. It's just a text file, magit is incredible, you can export to HTML and there are lots other formats available as ox plugins (e.g. MediaWiki), excellent LaTeX support, Org Mode is incredible for organizing large analyses and managing todos, you can glue together anything from anywhere (R, Python, shell scripts, Spark clusters, SQL, remote processes, etc.), and so on. Considering I code in addition to doing data analysis, I can reuse all my coding stuff to do data analysis too. I can take notes in Org Mode during a meeting, and then afterwards do some analysis directly on those meeting notes, export to HTML/PDF, and send it to a colleague. It is missing the quick backtick interpolation that you get with R-Markdown, and some of the nice UI stuff like inline Shiny graphs and clickable tables (but it is easy to output Org-format tables that can be piped into other languages).
- lottin 9y agoYes, ESS is fantastic. The only thing I envy about Rstudio is the GUI to save plots.
- noobhacker 9y agoESS is great, BUT its auto complete is non existent for the tidy verse, which is a deal breaker for those who work within it. It also takes setup to work with shiny and knitr, whereas Rstudio just works. On the other hand, I'm not sure ESS has anything that Rstudio doesn't offer.
- lottin 9y agoThe editing capabilities of Emacs are simply unmatched, for one thing. By comparison, writing code in Rstudio with its built-in editor feels like you're using some clunky, primitive tool :)
- confounded 9y agoThis will probably change (I hope!) with the adoption of the Language Server Protocol. I personally find Rstudio's suggestions to be a little odd (though I appreciate the fact that it actually works).
- tmulc18 9y agoReally cool article. It still seems like jupyter is the better longterm option because it offers so many different kernels.
- inputcoffee 9y agoThe big question in data science is: should I spend more time learning Python or R? The answer is always: math
- tomrod 9y agoWhich I find leads you to Python, then Julia, then bracket. /s
- lanstin 9y agoOne of the best things for R is the mailing list. At leasr when I was learning stats and R rhe knowledge both of how to do stuff in R and what math to use when was phenomenal. If gentle people didnt answer in time Prof Brian Ripley from Oxford would answer early morning British time and explain why your question was wrong and what the math meant and what you really meant to do and why and then 3 lines of R to do it.
- nl 9y agoCan I say how damn good notebooks (any notebooks) are for data exposition compared to traditional coding environments? I'm more familiar with Jupyter than R Notebooks. I'd second the point about version control in Jupyter being.. hard. There isn't really a good pattern for it yet. I would note that I believe the latest version of Jupyter has prettier tables though! Edit: Also, matplotlib makes me sad. Surely there could be something better which abandons it completely?
- noobiemcfoob 9y agoRe: matplotlib You have other options like bokeh and plotly
- ctlaltdefeat 9y agoAnd for options utilizing grammar of graphics (really worth knowing, even if you're not coming from the R world) you have ggplot and plotnine, which are both sort of ports of ggplot2 from R.
- nonbel 9y ago> "grammar of graphics (really worth knowing, even if you're not coming from the R world)" Can you explain why? I've never gotten the appeal. Besides the concept, the classic implementation (ggplot) does not make nice graphs in my opinion. To me these look wrong... I guess I'm not quite sure why. There is something about it too cookie cutter, cartoonish and information-light: http://r4stats.com/examples/graphics-ggplot2/ http://r4stats.com/examples/graphics-ggplot2/
- eruditely 9y agoI don't see this replacing Jupyter notebooks any time soon, as they simply are better.
- tgb 9y agoCall someone explain in what sense these notebooks are "reproducible" to a greater extent than just a .py or R file? I'm not that familiar with them. Do they have key metadata or something?
- Rotten194 9y agoWriting a bunch of scripts can quickly become a mess. I was working on some twitter analysis for a project, and not really worrying about the code because I didn't intend for it to be used again, and it quickly became a mess of "run this script, then run that script on the generated file, then use this shell command to process the file, then run the final analysis step on that file, then clean up all the intermediates". Not to mention, say, "one-time" data cleanup through the shell / REPL that runs into problems months down the line when you want to update the data set. And, of course, invariably none of this is documented. Notebooks don't force you to organize your code and write documentation, but they strongly encourage it.