5 ms·

I've read this book and taken this course twice, and it is easily one of the best learning experiences I've ever had. Statistics is a fascinating subject and Ri

I've read this book and taken this course twice, and it is easily one of the best learning experiences I've ever had. Statistics is a fascinating subject and Richard helps bring it alive. I had studied lots of classical statistics texts, but didn't quite "get" Bayesian statistics until I took Richard's course. Even if you aren't a data scientist or a statistician (I'm an infrastructure/software engineer, but I've dabbled as the "data person" in different startups), learning basic statistics will open your eyes to how easy it is to misinterpret data. My favorite part of this course, besides helping me understand Bayesian statistics, is the few chapters on causal relationships. I use that knowledge quite often at work and in my day-to-day life when reading the news; instead of crying "correlation is not causation!", you are armed with a more nuanced understanding of confounding variables, post-treatment bias, collider bias, etc. Lastly, don't be turned off by the use of R in this book. R is the programming language of statistics, and is quite easy to learn if you are already a software engineer and know a scripting language. It really is a powerful domain specific language for statistics, if not for the language then for all of the statisticians that have contributed to it.

jonnycomputer 5y ago
I frequently prefer R to python/pandas/numpy for data analysis--even if most of my other programming is in python.
- elcapitan 5y ago
  What's the advantage, if you already know Python? (genuine interest)
  - Bootvis 5y ago
    For me, I use R data.table a lot and I see as the main advantages are performance and the terse syntax. The terse syntax does come with a steep learning curve though.
    - jarenmf 5y ago
      Indeed, data.table is just awesome for productivity. When you're manipulating data for exploration you want the least number of keystrokes to bring an idea to life and data.table gives you that.
    - VeninVidiaVicii 5y ago
      I totally agree. I often find myself wanting data.table as a standalone database platform or ORM-type interface for non-statistical programming too.
    - boppo1 5y ago
      What is terse syntax? I can parse lisp and C, how would this be different and challenging?
      - deleted 5y ago
        [deleted]
      - bckygldstn 5y ago
        The syntax isn't self-describing and uses lots of abbreviations; it relies on some R magic that I found confusing when learning (unquoted column names and special builtin variables); and data.table is just a different approach to SQL and other dataframe libraries. Here's an example from the docs flights[carrier == "AA", lapply(.SD, mean), by = .(origin, dest, month), .SDcols = c("arr_delay", "dep_delay")] that's clearly less clear than SQL SELECT origin, dest, month, MEAN(arr_delay), MEAN(dep_delay) FROM flights WHERE carrier == "AA" GROUP BY arr_delay, dep_delay or pandas flights[filghts.carrier == 'AA'].groupby(['arr_delay', 'dep_delay']).mean() But once you get used to it data.table makes a lot of sense: every operation can be broken down to filtering/selecting, aggregating/transforming, and grouping/windowing. Taking the first two rows per group is a mess in SQL or pandas, but is super simple in data.table flights[, head(.SD, 2), by = month] That data.table has significantly better performance than any other dataframe library in any language is a nice bonus!
        kgwgk 5y ago
  - vcdimension 5y ago
    R is used by many researchers and consequentially has many more statistical libraries (e.g. try doing a dynamic panel modelling in python).
  - civilized 5y ago
    Tabular data manipulation packages are better, easier to make nontrivial charts, many R stats packages have no counterparts in Python, less bureaucracy, more batteries-included. R is a language by and for statisticians. Python is a programming language that can do some statistics.
  - jonnycomputer 5y ago
    I don't want to say "advantage", so much as preference. But a few things come to mind. - Lots of high quality statistical libraries, for one thing. - RStudio's RMarkown is great; I prefer it to Jupyter Notebook. - I personally found the syntax more intuitive, easier to pick up. I don't usually find myself confused about the structure of the objects I'm looking at. For whatever reason, the "syntax" of pandas doesn't square well (in my opinion) with python generally. I certainly want to just use python. But, shrug. - The tidyverse package, especially the pipe operator %>%, which afaik doesn't have an equivalent in Python. E.g. with_six_visits <- task_df %>% group_by(turker_id, visit) %>% summarise(n_trials = n_distinct(trial_num)) %>% mutate(completed_visit = n_trials>40) %>% filter(completed_visit) %>% summarise(n_visits = n_distinct(visit)) %>% mutate(six_visits = n_visits >= 6) %>% filter(six_visits) %>% ungroup() Here I'm filtering participants in an mturk study by those who have completed more than 40 trials at least six times across multiple sessions. It's not that I couldn't do the same transformation in pandas, but it feels very intuitive to me doing it this way. - ggplot2 for plotting; its really powerful data visualization package. Truthfully, I often do my data text parsing in Python, and then switch over to R for analysis, E.g. python's JSON parsing works really well.
    - rahimnathwani 5y ago
      I can see how this is more intuitive. In pandas I'd assign the output of groupby to a variable, and then add the new column in a separate statement. (The below is off topic, but I don't use R so I'd love to know whether I'm reading the code correctly) "Here I'm filtering participants in an mturk study by those who have completed more than 40 trials at least six times across multiple sessions." A user with this pattern of trials seems like they would fit the above definition: Session 1: 82 trials Session 2: 82 trials Session 3: 82 trials But the code seems to want 6 distinct sessions with >40 trials each. Have I misunderstood? Also, is 'mutate' necessary before 'filter' or is that just to make the intent of the code clearer to your future self?
      - jonnycomputer 5y ago
        My initial wording was sloppy. There were 50 trials in each session; so I counted a session completed if they did more than 40 in that session. They needed to have completed at least six sessions. The mutate is unnecessary. I forget why I did that.
  - iandinwoodie 5y ago
    One difference I've noticed is that R libraries are usually authored and maintained by academics in the associated field; the same can't always be said about equivalent Python libraries. This means that R library authors generally use their own libraries for publication and have an academic stake in its correctness.
  - clove 5y ago
    I have the opposite question. I have been programming in R since I was 19. I know no other programming languages. Hence my question: What's the advantage of Python if you already know R? I've heard they have similarities. Is there anything Python does better than R in terms of statistical analysis, charting, etc.?
    - noiwillnot 5y ago
      > What's the advantage of Python if you already know R? AFAIK in statistical modelling Python is better only in neural networks, so if you do not need to do fancy things with images, text, etc. you do not need Python. R is still the king. In terms of charting and dashboards, I would say that if you work high level R and Python are both pleasant. R has ggplot, but Python has Plotly Express. R has Shiny, but Python has Dash and Streamlit. You can do great with both.
  - zwaps 5y ago
    On top of what has been said, if you want to do some more advanced statistical analyses (in the inference area, not ML/predictive field), then chances are that these algorithms are either published as R or STATA packages (usually R). In Python, there is statsmodels. Here, you'll find a lot of GLM stuff, which is sort of an older approach. Modern inferential statistics, if not just Bayesian, is usually in the flavor of semi-parametric models that rely on asymptotics. As R is used by professional researchers, it is simply more on the edge of things. Python has most of the "statistics course" schoolbook methods, but not much beyond that. For example, it has become very common to have dynamic panel data which require dynamic models. Now if you want to do a Blundell-Bond type model in PYthon you have to... code it yourself using GMM, if it exists even. For statistics, that's pretty much like saying you have a Deep Learning package that maybe has GRU but no transformer modules at all. So yeah, you can code it yourself. Or you use the other one.
swayson 5y ago
Julia, R (tidyverse), Python code examples available here: https://github.com/StatisticalRethinkingJulia https://github.com/StatisticalRethinkingJulia https://github.com/pymc-devs/resources/tree/master/Rethinking_2 https://github.com/pymc-devs/resources/tree/master/Rethinkin... https://bookdown.org/content/4857/ https://bookdown.org/content/4857/
agucova 5y ago
Even if you don't like R, your can do the entire course with Julia/Turing, Julia/Stan or Python, the course github's page has a list of “code translations” for all the examples.
- fault1 5y ago
  There is also other translations, for example, in pytorch/pyro: https://fehiepsi.github.io/rethinking-pyro/ https://fehiepsi.github.io/rethinking-pyro/ I would say statistical rethinking is a great way to compare and contrast different ppl impls and languages, I've been using it with Turing, which is pretty great.

jonnycomputer 5y ago

I frequently prefer R to python/pandas/numpy for data analysis--even if most of my other programming is in python.

elcapitan 5y ago
What's the advantage, if you already know Python? (genuine interest)
- Bootvis 5y ago
  For me, I use R data.table a lot and I see as the main advantages are performance and the terse syntax. The terse syntax does come with a steep learning curve though.
  - jarenmf 5y ago
    Indeed, data.table is just awesome for productivity. When you're manipulating data for exploration you want the least number of keystrokes to bring an idea to life and data.table gives you that.
  - VeninVidiaVicii 5y ago
    I totally agree. I often find myself wanting data.table as a standalone database platform or ORM-type interface for non-statistical programming too.
  - boppo1 5y ago
    What is terse syntax? I can parse lisp and C, how would this be different and challenging?
    - deleted 5y ago
      [deleted]
    - bckygldstn 5y ago
      The syntax isn't self-describing and uses lots of abbreviations; it relies on some R magic that I found confusing when learning (unquoted column names and special builtin variables); and data.table is just a different approach to SQL and other dataframe libraries. Here's an example from the docs flights[carrier == "AA", lapply(.SD, mean), by = .(origin, dest, month), .SDcols = c("arr_delay", "dep_delay")] that's clearly less clear than SQL SELECT origin, dest, month, MEAN(arr_delay), MEAN(dep_delay) FROM flights WHERE carrier == "AA" GROUP BY arr_delay, dep_delay or pandas flights[filghts.carrier == 'AA'].groupby(['arr_delay', 'dep_delay']).mean() But once you get used to it data.table makes a lot of sense: every operation can be broken down to filtering/selecting, aggregating/transforming, and grouping/windowing. Taking the first two rows per group is a mess in SQL or pandas, but is super simple in data.table flights[, head(.SD, 2), by = month] That data.table has significantly better performance than any other dataframe library in any language is a nice bonus!
      - kgwgk 5y ago
- vcdimension 5y ago
  R is used by many researchers and consequentially has many more statistical libraries (e.g. try doing a dynamic panel modelling in python).
- civilized 5y ago
  Tabular data manipulation packages are better, easier to make nontrivial charts, many R stats packages have no counterparts in Python, less bureaucracy, more batteries-included. R is a language by and for statisticians. Python is a programming language that can do some statistics.
- jonnycomputer 5y ago
  I don't want to say "advantage", so much as preference. But a few things come to mind. - Lots of high quality statistical libraries, for one thing. - RStudio's RMarkown is great; I prefer it to Jupyter Notebook. - I personally found the syntax more intuitive, easier to pick up. I don't usually find myself confused about the structure of the objects I'm looking at. For whatever reason, the "syntax" of pandas doesn't square well (in my opinion) with python generally. I certainly want to just use python. But, shrug. - The tidyverse package, especially the pipe operator %>%, which afaik doesn't have an equivalent in Python. E.g. with_six_visits <- task_df %>% group_by(turker_id, visit) %>% summarise(n_trials = n_distinct(trial_num)) %>% mutate(completed_visit = n_trials>40) %>% filter(completed_visit) %>% summarise(n_visits = n_distinct(visit)) %>% mutate(six_visits = n_visits >= 6) %>% filter(six_visits) %>% ungroup() Here I'm filtering participants in an mturk study by those who have completed more than 40 trials at least six times across multiple sessions. It's not that I couldn't do the same transformation in pandas, but it feels very intuitive to me doing it this way. - ggplot2 for plotting; its really powerful data visualization package. Truthfully, I often do my data text parsing in Python, and then switch over to R for analysis, E.g. python's JSON parsing works really well.
  - rahimnathwani 5y ago
    I can see how this is more intuitive. In pandas I'd assign the output of groupby to a variable, and then add the new column in a separate statement. (The below is off topic, but I don't use R so I'd love to know whether I'm reading the code correctly) "Here I'm filtering participants in an mturk study by those who have completed more than 40 trials at least six times across multiple sessions." A user with this pattern of trials seems like they would fit the above definition: Session 1: 82 trials Session 2: 82 trials Session 3: 82 trials But the code seems to want 6 distinct sessions with >40 trials each. Have I misunderstood? Also, is 'mutate' necessary before 'filter' or is that just to make the intent of the code clearer to your future self?
    - jonnycomputer 5y ago
      My initial wording was sloppy. There were 50 trials in each session; so I counted a session completed if they did more than 40 in that session. They needed to have completed at least six sessions. The mutate is unnecessary. I forget why I did that.
- iandinwoodie 5y ago
  One difference I've noticed is that R libraries are usually authored and maintained by academics in the associated field; the same can't always be said about equivalent Python libraries. This means that R library authors generally use their own libraries for publication and have an academic stake in its correctness.
- clove 5y ago
  I have the opposite question. I have been programming in R since I was 19. I know no other programming languages. Hence my question: What's the advantage of Python if you already know R? I've heard they have similarities. Is there anything Python does better than R in terms of statistical analysis, charting, etc.?
  - noiwillnot 5y ago
    > What's the advantage of Python if you already know R? AFAIK in statistical modelling Python is better only in neural networks, so if you do not need to do fancy things with images, text, etc. you do not need Python. R is still the king. In terms of charting and dashboards, I would say that if you work high level R and Python are both pleasant. R has ggplot, but Python has Plotly Express. R has Shiny, but Python has Dash and Streamlit. You can do great with both.
- zwaps 5y ago
  On top of what has been said, if you want to do some more advanced statistical analyses (in the inference area, not ML/predictive field), then chances are that these algorithms are either published as R or STATA packages (usually R). In Python, there is statsmodels. Here, you'll find a lot of GLM stuff, which is sort of an older approach. Modern inferential statistics, if not just Bayesian, is usually in the flavor of semi-parametric models that rely on asymptotics. As R is used by professional researchers, it is simply more on the edge of things. Python has most of the "statistics course" schoolbook methods, but not much beyond that. For example, it has become very common to have dynamic panel data which require dynamic models. Now if you want to do a Blundell-Bond type model in PYthon you have to... code it yourself using GMM, if it exists even. For statistics, that's pretty much like saying you have a Deep Learning package that maybe has GRU but no transformer modules at all. So yeah, you can code it yourself. Or you use the other one.

swayson 5y ago

Julia, R (tidyverse), Python code examples available here: https://github.com/StatisticalRethinkingJulia https://github.com/StatisticalRethinkingJulia https://github.com/pymc-devs/resources/tree/master/Rethinking_2 https://github.com/pymc-devs/resources/tree/master/Rethinkin... https://bookdown.org/content/4857/ https://bookdown.org/content/4857/

agucova 5y ago

Even if you don't like R, your can do the entire course with Julia/Turing, Julia/Stan or Python, the course github's page has a list of “code translations” for all the examples.

fault1 5y ago
There is also other translations, for example, in pytorch/pyro: https://fehiepsi.github.io/rethinking-pyro/ https://fehiepsi.github.io/rethinking-pyro/ I would say statistical rethinking is a great way to compare and contrast different ppl impls and languages, I've been using it with Turing, which is pretty great.