4 ms·

Yep. The idea that I should have to code defensively to make sure that the matrices I'm multiplying are of the correct size[1] is complete bullshit. cat(pa

Yep. The idea that I should have to code defensively to make sure that the matrices I'm multiplying are of the correct size[1] is complete bullshit. cat(paste("That", "and", "string", "manipulation", "in", "R", "is", "a", "pain", "in", "the", "ass", sep=" ")) [1]: If A is n by m and B is p by q, and m is a multiple of p, then R will SILENTLY concatenate copies of B to itself to form an m by q matrix, and then do the multiplication.

cwyers 11y ago
R seems to have taken Perl's "there's more than one way to do it" as a credo, all of the official documentation I've seen seems to be an exercise in trying to say as little as possible with as many words as can be managed, all of the syntax seems to be deliberately designed to make no sense whatsoever and I can give it a task that JMP (no speed demon it) can finish in under an hour and not get results back after letting it churn for nearly a day. Everything about R makes me wonder what I've done with my life to end up using it.
- ekianjo 11y ago
  > and I can give it a task that JMP (no speed demon it) can finish in under an hour and not get results back after letting it churn for nearly a day Example, please. I'd really like to see what you are talking about here. And btw, good luck treating any kind of Big Data with JMP.
  - jackmaney 11y ago
    Depending on what you mean by "Big Data", you may as well say "good luck treating any kind of Big Data without some kind of distributed system".
  - cwyers 11y ago
    > And btw, good luck treating any kind of Big Data with JMP. Oh I'm definitely not intending to. (To the commenter below, it really doesn't matter how you define Big Data for that statement.) That much I had already figured out, which was why I was evaluating R with an eye towards pitching my boss on it. The test case I'm referring to here was a pretty simple neural net to my mind -- roughly 350,000 rows of data, six predictor variables, one hidden layer with 30 nodes. I can verify that the neural net code ran because if I truncated the data set down to 1,000 rows I got a result back. But the full dataset just chugged for hours and hours without stop.
- nkurz 11y ago
  That's unfair to to the pig. I mean, unfair to Perl. In the past I've done lots of programming in Perl, and am currently doing lots of programming in R. I would say without reservation that Perl is a much better designed language, with good documentation, and a coherent philosophy. Perl has some issues, but has a solid core. R has some nice individual packages, and some bright people involved, but the core is rotten. While it is possible to write high performance R, and possible to write easy-to-read R, I don't think it's possible to do both at the same time. It's easy to drop to C or C++ to speed up the critical sections, but often the glue between the pieces is so slow that it's not worth bothering. I'm not recommending that you use Perl for statistics (Python or Julia would be better choices), but I think a language written by a linguist for sysadmins is a much better choice than a language written by statisticians for statisticians. Although maybe that's because I'm a programmer and not a statistician?
  - nkurz 11y ago
    As evidence for my position, and as evidence that I'm doing my best to fix these issues rather than idly carping, here's a bug report I submitted today: https://stat.ethz.ch/pipermail/r-devel/2015-September/071744.html https://stat.ethz.ch/pipermail/r-devel/2015-September/071744... Some of my sampling code was running much slower than I thought made sense (even for R), and the strange part was that larger samples were sometimes 1000x faster than smaller samples. The answer was that R (wisely) uses two algorithms for sampling without replacement, but the cutoff between them is based on a fixed sample size of 1e7, rather than the percentage of population sampled. Perhaps there's a good reason?
a_bonobo 11y ago
I have a feeling that everything involving strings in R is a major headache. Case in point: - stringsAsFactors is True by default, leading to all kinds of weird silent behaviors - length('string') being 1, since that's silently a vector of length 1 (~ length(c('string')) ). Have to use nchar() or str_length() But, analyzing strings is pretty cool - stringdist() or levenshteinDist() are great!
- jackmaney 11y ago
  Oh yes, stringsAsFactors...just seeing that almost conjures up flashbacks. Data frames in Pandas have no such implicit stupidity. The rough equivalent, IIRC, is setting up one or more columns as an index (or multi-index, respectively), and that's mostly done explicitly (with the exception of importing or exporting CSVs; the first column is imported as the index by default, and the index values are written out by default).

cwyers 11y ago

R seems to have taken Perl's "there's more than one way to do it" as a credo, all of the official documentation I've seen seems to be an exercise in trying to say as little as possible with as many words as can be managed, all of the syntax seems to be deliberately designed to make no sense whatsoever and I can give it a task that JMP (no speed demon it) can finish in under an hour and not get results back after letting it churn for nearly a day. Everything about R makes me wonder what I've done with my life to end up using it.

ekianjo 11y ago
> and I can give it a task that JMP (no speed demon it) can finish in under an hour and not get results back after letting it churn for nearly a day Example, please. I'd really like to see what you are talking about here. And btw, good luck treating any kind of Big Data with JMP.
- jackmaney 11y ago
  Depending on what you mean by "Big Data", you may as well say "good luck treating any kind of Big Data without some kind of distributed system".
- cwyers 11y ago
  > And btw, good luck treating any kind of Big Data with JMP. Oh I'm definitely not intending to. (To the commenter below, it really doesn't matter how you define Big Data for that statement.) That much I had already figured out, which was why I was evaluating R with an eye towards pitching my boss on it. The test case I'm referring to here was a pretty simple neural net to my mind -- roughly 350,000 rows of data, six predictor variables, one hidden layer with 30 nodes. I can verify that the neural net code ran because if I truncated the data set down to 1,000 rows I got a result back. But the full dataset just chugged for hours and hours without stop.
nkurz 11y ago
That's unfair to to the pig. I mean, unfair to Perl. In the past I've done lots of programming in Perl, and am currently doing lots of programming in R. I would say without reservation that Perl is a much better designed language, with good documentation, and a coherent philosophy. Perl has some issues, but has a solid core. R has some nice individual packages, and some bright people involved, but the core is rotten. While it is possible to write high performance R, and possible to write easy-to-read R, I don't think it's possible to do both at the same time. It's easy to drop to C or C++ to speed up the critical sections, but often the glue between the pieces is so slow that it's not worth bothering. I'm not recommending that you use Perl for statistics (Python or Julia would be better choices), but I think a language written by a linguist for sysadmins is a much better choice than a language written by statisticians for statisticians. Although maybe that's because I'm a programmer and not a statistician?
- nkurz 11y ago
  As evidence for my position, and as evidence that I'm doing my best to fix these issues rather than idly carping, here's a bug report I submitted today: https://stat.ethz.ch/pipermail/r-devel/2015-September/071744.html https://stat.ethz.ch/pipermail/r-devel/2015-September/071744... Some of my sampling code was running much slower than I thought made sense (even for R), and the strange part was that larger samples were sometimes 1000x faster than smaller samples. The answer was that R (wisely) uses two algorithms for sampling without replacement, but the cutoff between them is based on a fixed sample size of 1e7, rather than the percentage of population sampled. Perhaps there's a good reason?

a_bonobo 11y ago

I have a feeling that everything involving strings in R is a major headache. Case in point: - stringsAsFactors is True by default, leading to all kinds of weird silent behaviors - length('string') being 1, since that's silently a vector of length 1 (~ length(c('string')) ). Have to use nchar() or str_length() But, analyzing strings is pretty cool - stringdist() or levenshteinDist() are great!

jackmaney 11y ago
Oh yes, stringsAsFactors...just seeing that almost conjures up flashbacks. Data frames in Pandas have no such implicit stupidity. The rough equivalent, IIRC, is setting up one or more columns as an index (or multi-index, respectively), and that's mostly done explicitly (with the exception of importing or exporting CSVs; the first column is imported as the index by default, and the index values are written out by default).