4 ms·

I think this is really interesting. The author certainly isn't an expert, for example `result[which(result < 0.5)] <- 0` is a mistake for `result[result < 0.5]

by dash2 5y ago

I think this is really interesting. The author certainly isn't an expert, for example `result[which(result < 0.5)] <- 0` is a mistake for `result[result < 0.5] <- 0`. But that's just why it's useful - R is great when you are an expert, but becoming an expert takes years. The perspective of new users is really important. (I've been using R almost 20 years, have written several packages, and still feel like an amateur. Indeed, I'd never heard of `**` as an alias for `^` until today; nor `sequence`, which apparently has always been in base; and I still can't remember what `sweep` does.) I thought some of these arguments were better than others. True that base R regex is confusing and messy (and that stringi/stringr are improvements). False that allowing string concatenation with `+` would be a good idea. That's just a footgun waiting to go off, given that R also is weakly typed. Expecting `nchar(1000)` to magically work seems naïve. `<<-` (roughly, global assignment) is an ugly necessity and a code smell, not a cool language feature. An awful lot of these problems are fixed, or try to be fixed, in the tidyverse. Not using tidyverse is a bit unusual because most beginners nowadays, I think, start with the tidyverse more than with base R. For me the worst part of R is simply it fails silently. This is really deadly, especially when you are producing scientific results. There are so many places where R will plug gamely on after you have done something deeply inappropriate. Given how badly scientists code, one has to worry. I don't agree that "R won’t change" is the base problem. It's not so simple. R is used for science. I like very much that my code from 2008 will probably still work if someone wants to replicate my results. I appreciate the R-core team's work in making this true. There are genuine trade-offs here. If you want emotional relief, it's worth following https://twitter.com/whydoesR https://twitter.com/whydoesR. Maybe Julia is the way forward? Or is R "worse is better"?

bluenose69 5y ago
> Maybe Julia is the way forward? Julia is well worth learning, if you do computationally-expensive work. It is kind of a pain to use interactively, though. I use both R and Julia in my research. Think of Julia as the new Fortran, though, not the new R.
- edgyquant 5y ago
  Isn’t Julia a compiled language “pretending” to be interpreted? I thought it was interpreted or JITd up until it’s last release or so when they mentioned it has to actually compile. It’s fast, sure, but is it faster than other compiled languages?
  - bluenose69 5y ago
    I do not have deep experience with this, but the word on the street is that Julia can be faster than other compiled languages. I think that's partly because it can find a good algorithm, based on your data structure; think of loop unrolling, etc. You can inspect the assembly code for anything you're working on, and that can be quite helpful at times; see e.g. https://youtu.be/wU6c8CDRXJE?t=3887 https://youtu.be/wU6c8CDRXJE?t=3887. I think the reason why quite a few high-performance people (I mean in the science community -- I don't know much about other communities) are excited about Julia is simply that well-respected experts are also excited. An example is the Julia implementation of the MIT GCM (general circulation model) for the ocean; for similar projects, see https://github.com/CliMA https://github.com/CliMA. Programmer effort is also a factor in scientific computation. If the system can do some of your work for you, so much the better; see e.g. https://www.youtube.com/watch?v=rZS2LGiurKY https://www.youtube.com/watch?v=rZS2LGiurKY for a lecture that touches upon how the framework of Julia eases the burden of machine-learning tasks. As I say, though, I do not have deep experience with Julia. I've rewritten one of my numerical models in Julia and the speed is about the same as before, but my code is much shorter and easier to understand. I would not burn up 6 months translating a complex code, but nor would I start a 6-month coding project in Fortran anymore.
  - adgjlsfhk1 5y ago
    In general, Julia has similar performance characteristics to other fast compiled languages (Fortran/C++). There are some performance differences do to different semantics (eg bounds checks by default), but Julia is good about giving you the ability to opt out of these easily. There are also definitely places where Julia makes it a lot easier to get better code by making it a lot easier to use better algorithms, so in general, simple Julia code tends to be similar/better performance than Fortran/C++, and optimized Julia code tends to be about the same speed as Fortran/C++, but with 10x less code.
dm319 5y ago
I think a lot of the problem with comparing R to other languages is that a lot people don't get the problem space that R is working in. Science deals a lot with categorical variables, missing data and high dimensional data, and the 'table' or 'dataframe' is adept at storing and working with this information. Under the hood it's just a load of optimised fortran code working matrices, but the code clearly shows what kinds of data manipulations and transformation you are doing to eek the right information and visualisations out from the dataset. I see problems when people take an imperative approach to solving numerical problems, and something like Python is better suited to that. Also, R isn't really set up to work with matrices like Matlab/Julia are.

by dash2 5y ago

bluenose69 5y ago

> Maybe Julia is the way forward? Julia is well worth learning, if you do computationally-expensive work. It is kind of a pain to use interactively, though. I use both R and Julia in my research. Think of Julia as the new Fortran, though, not the new R.

edgyquant 5y ago
Isn’t Julia a compiled language “pretending” to be interpreted? I thought it was interpreted or JITd up until it’s last release or so when they mentioned it has to actually compile. It’s fast, sure, but is it faster than other compiled languages?
- bluenose69 5y ago
  I do not have deep experience with this, but the word on the street is that Julia can be faster than other compiled languages. I think that's partly because it can find a good algorithm, based on your data structure; think of loop unrolling, etc. You can inspect the assembly code for anything you're working on, and that can be quite helpful at times; see e.g. https://youtu.be/wU6c8CDRXJE?t=3887 https://youtu.be/wU6c8CDRXJE?t=3887. I think the reason why quite a few high-performance people (I mean in the science community -- I don't know much about other communities) are excited about Julia is simply that well-respected experts are also excited. An example is the Julia implementation of the MIT GCM (general circulation model) for the ocean; for similar projects, see https://github.com/CliMA https://github.com/CliMA. Programmer effort is also a factor in scientific computation. If the system can do some of your work for you, so much the better; see e.g. https://www.youtube.com/watch?v=rZS2LGiurKY https://www.youtube.com/watch?v=rZS2LGiurKY for a lecture that touches upon how the framework of Julia eases the burden of machine-learning tasks. As I say, though, I do not have deep experience with Julia. I've rewritten one of my numerical models in Julia and the speed is about the same as before, but my code is much shorter and easier to understand. I would not burn up 6 months translating a complex code, but nor would I start a 6-month coding project in Fortran anymore.
- adgjlsfhk1 5y ago
  In general, Julia has similar performance characteristics to other fast compiled languages (Fortran/C++). There are some performance differences do to different semantics (eg bounds checks by default), but Julia is good about giving you the ability to opt out of these easily. There are also definitely places where Julia makes it a lot easier to get better code by making it a lot easier to use better algorithms, so in general, simple Julia code tends to be similar/better performance than Fortran/C++, and optimized Julia code tends to be about the same speed as Fortran/C++, but with 10x less code.

dm319 5y ago

I think a lot of the problem with comparing R to other languages is that a lot people don't get the problem space that R is working in. Science deals a lot with categorical variables, missing data and high dimensional data, and the 'table' or 'dataframe' is adept at storing and working with this information. Under the hood it's just a load of optimised fortran code working matrices, but the code clearly shows what kinds of data manipulations and transformation you are doing to eek the right information and visualisations out from the dataset. I see problems when people take an imperative approach to solving numerical problems, and something like Python is better suited to that. Also, R isn't really set up to work with matrices like Matlab/Julia are.