3 ms·

R seems to have taken Perl's "there's more than one way to do it" as a credo, all of the official documentation I've seen seems to be an exercise in trying to s

by cwyers 11y ago

R seems to have taken Perl's "there's more than one way to do it" as a credo, all of the official documentation I've seen seems to be an exercise in trying to say as little as possible with as many words as can be managed, all of the syntax seems to be deliberately designed to make no sense whatsoever and I can give it a task that JMP (no speed demon it) can finish in under an hour and not get results back after letting it churn for nearly a day. Everything about R makes me wonder what I've done with my life to end up using it.

ekianjo 11y ago
> and I can give it a task that JMP (no speed demon it) can finish in under an hour and not get results back after letting it churn for nearly a day Example, please. I'd really like to see what you are talking about here. And btw, good luck treating any kind of Big Data with JMP.
- jackmaney 11y ago
  Depending on what you mean by "Big Data", you may as well say "good luck treating any kind of Big Data without some kind of distributed system".
- cwyers 11y ago
  > And btw, good luck treating any kind of Big Data with JMP. Oh I'm definitely not intending to. (To the commenter below, it really doesn't matter how you define Big Data for that statement.) That much I had already figured out, which was why I was evaluating R with an eye towards pitching my boss on it. The test case I'm referring to here was a pretty simple neural net to my mind -- roughly 350,000 rows of data, six predictor variables, one hidden layer with 30 nodes. I can verify that the neural net code ran because if I truncated the data set down to 1,000 rows I got a result back. But the full dataset just chugged for hours and hours without stop.
nkurz 11y ago
That's unfair to to the pig. I mean, unfair to Perl. In the past I've done lots of programming in Perl, and am currently doing lots of programming in R. I would say without reservation that Perl is a much better designed language, with good documentation, and a coherent philosophy. Perl has some issues, but has a solid core. R has some nice individual packages, and some bright people involved, but the core is rotten. While it is possible to write high performance R, and possible to write easy-to-read R, I don't think it's possible to do both at the same time. It's easy to drop to C or C++ to speed up the critical sections, but often the glue between the pieces is so slow that it's not worth bothering. I'm not recommending that you use Perl for statistics (Python or Julia would be better choices), but I think a language written by a linguist for sysadmins is a much better choice than a language written by statisticians for statisticians. Although maybe that's because I'm a programmer and not a statistician?
- nkurz 11y ago
  As evidence for my position, and as evidence that I'm doing my best to fix these issues rather than idly carping, here's a bug report I submitted today: https://stat.ethz.ch/pipermail/r-devel/2015-September/071744.html https://stat.ethz.ch/pipermail/r-devel/2015-September/071744... Some of my sampling code was running much slower than I thought made sense (even for R), and the strange part was that larger samples were sometimes 1000x faster than smaller samples. The answer was that R (wisely) uses two algorithms for sampling without replacement, but the cutoff between them is based on a fixed sample size of 1e7, rather than the percentage of population sampled. Perhaps there's a good reason?

by cwyers 11y ago

ekianjo 11y ago

> and I can give it a task that JMP (no speed demon it) can finish in under an hour and not get results back after letting it churn for nearly a day Example, please. I'd really like to see what you are talking about here. And btw, good luck treating any kind of Big Data with JMP.

jackmaney 11y ago
Depending on what you mean by "Big Data", you may as well say "good luck treating any kind of Big Data without some kind of distributed system".
cwyers 11y ago
> And btw, good luck treating any kind of Big Data with JMP. Oh I'm definitely not intending to. (To the commenter below, it really doesn't matter how you define Big Data for that statement.) That much I had already figured out, which was why I was evaluating R with an eye towards pitching my boss on it. The test case I'm referring to here was a pretty simple neural net to my mind -- roughly 350,000 rows of data, six predictor variables, one hidden layer with 30 nodes. I can verify that the neural net code ran because if I truncated the data set down to 1,000 rows I got a result back. But the full dataset just chugged for hours and hours without stop.

nkurz 11y ago

That's unfair to to the pig. I mean, unfair to Perl. In the past I've done lots of programming in Perl, and am currently doing lots of programming in R. I would say without reservation that Perl is a much better designed language, with good documentation, and a coherent philosophy. Perl has some issues, but has a solid core. R has some nice individual packages, and some bright people involved, but the core is rotten. While it is possible to write high performance R, and possible to write easy-to-read R, I don't think it's possible to do both at the same time. It's easy to drop to C or C++ to speed up the critical sections, but often the glue between the pieces is so slow that it's not worth bothering. I'm not recommending that you use Perl for statistics (Python or Julia would be better choices), but I think a language written by a linguist for sysadmins is a much better choice than a language written by statisticians for statisticians. Although maybe that's because I'm a programmer and not a statistician?

nkurz 11y ago
As evidence for my position, and as evidence that I'm doing my best to fix these issues rather than idly carping, here's a bug report I submitted today: https://stat.ethz.ch/pipermail/r-devel/2015-September/071744.html https://stat.ethz.ch/pipermail/r-devel/2015-September/071744... Some of my sampling code was running much slower than I thought made sense (even for R), and the strange part was that larger samples were sometimes 1000x faster than smaller samples. The answer was that R (wisely) uses two algorithms for sampling without replacement, but the cutoff between them is based on a fixed sample size of 1e7, rather than the percentage of population sampled. Perhaps there's a good reason?