5 ms·
I'm a scientist (PhD student in microbiolgy) that works with lots of data. My data is on the order of hundreds of gigabytes (genome collections and other sequen
by elsherbini 9y ago
I'm a scientist (PhD student in microbiolgy) that works with lots of data. My data is on the order of hundreds of gigabytes (genome collections and other sequencing data) or megabytes (flat files).
I use the `tidyverse` from R[0] for everything people use `pandas` for. I think the syntax is soooo much more pleasant to use. It's declarative and because of pipes and "quosures" is highly readable. Combined with the power of `broom`,fitting simple models to the data and working with the results is really nice. Add to that that `ggplot` (+ any sane styling defaults like `cowplot`) is the fastest way to iterate on data visualizations that I've ever found. "R for Data Science" [1] is great free resource for getting started.
Snakemake [2] is a pipeline tool that submits steps of the pipeline to a cluster and handles waiting for steps to finish before submitting dependent steps. As a result, my pipelines have very little boilerplate, they are self documented, and the cluster is abstracted away so the same pipeline can work on a cluster or a laptop.
[0] https://www.tidyverse.org/ https://www.tidyverse.org/
[1] http://r4ds.had.co.nz/ http://r4ds.had.co.nz/
[2] http://snakemake.readthedocs.io/en/stable/ http://snakemake.readthedocs.io/en/stable/
- nonbel 9y agoSometimes I think I'm the only one who isn't really a fan of the tidyverse. I've found it slower, more prone to dependency issues, more prone to silent errors, and less well documented than most R packages (ie most of what you find on CRAN).
- swarchal 9y agoYou're not the only one. Though I've found there seems to be a bit of a cult surrounding the tidyverse, a mere hint of criticism usually results in outrage and attacking other tools/packages (by users, not the authors).
- anarchimedes 9y agoI like the Tidyverse. My only complaint is that it presents another headache when moving from one language (R) to another (usually Python/SQL). Using the base R functions while integrating loops and functions lessens the fatigue of moving between R and Python.
- in9 9y agoI tend to think that, if I am working with other people, python is the best choice. But if I'm working alone, R is the way to go.
- chasedehan 9y agoI really enjoy the tidyverse, especially dplyr. I do most of my work in python now and find myself moving more and more of time in python. There are definitely some issues if you have to reliably run scripts (not to mention the difficulties of putting into production) The thing I really like about R over python is for SPECIFIC tasks like inspecting data and trying to get an answer out quickly, there really isn't a quicker or better tool to use. The ONLY reason I still even use R is because of the ease to get answers with the tidyverse
- Seanny123 9y agoI would love some examples where R makes it easier to get answers than Python. Probably would make good Pandas2 issue too!
- vobios 9y agoThis is super controversial, but no need to use Jupyter. I personally find that Jupyter feels like a hack compared to something like RStudio. You have to open a terminal and launch a web server?
- in9 9y agoDependency management, in my opinion, is one of the problems in the R ecosystem. The lack of name spaces when calling functions has made the community have many little packages that only do one thing on you are not really sure where it was actually used, unless you know the code and the package. An example is the janitor::clean_names function I like to use for standardizing the column names on a data.frame. However, the tidyverse is really serious in terms of api consistency and functional style, with pipes and purrr's functionalities. The unixy style of base R is unproductive in terms of fast iterating an analysis. Also, the idea of "everything in a data frame" (or tibble, with list columns and whatnot) together with the tidy data principles really takes the cognitive load off to just get things started.
- kirillseva 9y agoYou should try https://github.com/robertzk/lockbox https://github.com/robertzk/lockbox for dependency management It's like bundler or cargo for R
- zamazingo 9y agoI agree on these reservations, especially in terms of silent errors (which get compounded through minor ways in which backwards incompatibility can sneak in to the existing scripts) and dependencies. As a half-solution, I ended up restricting myself to a very few libraries in this family (mainly dplyr, lubridate, stringr, broom) and to using packrat to consistently freeze the library versions for these.
- DataWorker 9y agoYou are not alone. I think it’s a great thing for some people, but a net negative for the R community in the long run.
- vobios 9y agoA non-insignificant fraction of the R community only exists because of Tidyverse.
- vobios 9y ago> less well documented than most R packages I on the other hand, find most R packages provide barely readable documentation. I can just hope that the vignette exists and actually explains the inputs/outputs.
- nonbel 9y agoHere it is for one of the most often used functions: https://www.rdocumentation.org/packages/ggplot2/versions/2.2.1/topics/aes https://www.rdocumentation.org/packages/ggplot2/versions/2.2... You think this is better than barely readable?