5 ms·
R beats Python, R beats Julia, Anyone else wanna challenge R? (2014)
- leeleelee 11y agoI find python's multiprocessing module to be simple to use. I would be interested in hearing the author elaborate on why he feels it is clunky and has not had a good experience with it.
- gaze 11y ago"In their trial run, Julia was much faster than R. But I objected, because random walk is a sum. Thus one can generate the entire process in R as vector calls, one to generate the steps and then a call to cumsum(), e.g." You missed the point. Everything is fast when you export what you're doing to a library. Julia is fast when you do, and when you DON'T.
- trymas 11y ago+1, wanted to say the same. P.S. that function name though...
- epistasis 11y agoBut not using the language's idioms is not a fair comparison. In modern times, vector based thinking like that that's present in R or numpy is the only sane way to program. Because these are not "library calls," they are the primitives of modern architectures.
- idunning 11y agoThe problem is that you may have to stray outside that vectorized bubble, and at that point you enter into a two-language situation if you are using R (e.g. RCpp) or Python (Cython?) - which was one of the motivations for the whole Julia language, to avoid needing to do that.
- epistasis 11y agoBut that's no excuse for a bad benchmark. If it actually were necessary to stray out of vectorization, then the Julia benchmark would have a point. But those cases are few and far between in what I have experienced, and apparently in what the benchmark authors have experienced.
- JupiterMoon 11y agoScientists don't want to have to change the way they think to solve every single problem. We want to code something yesterday that solves today's problem and move on. A language that allows us to use simple idioms will be awesome.
- epistasis 11y agoGiven the popularity of R, I would argue that scientists prefer those idioms to the more staid serial and imperative methods of Julia. I think that scientists are much more likely to enjoy the lisp nature of R than the code-monkeys that get caught up because arrays start with 1 instead of 0.
- JupiterMoon 11y agoNo physical and life scientists do not in the general case care about how "lisp nature" or "serial and imperative methods". We use R because there is a package in CRAN for the calculations we want to do -- and because our colleague/boss does. Often we will use loops because the mental effort of coding a vector solution can be higher
- srean 11y ago> scientists are much more likely to enjoy the lisp nature of R That confused me a great deal. Julia seems more lispy than R. From the comment section it seems that you have perhaps implicitly decided apriori that R is better. So you would rationalize out of evidence and examples of situations to the contrary.
- gtrubetskoy 11y ago"First, R is written by statisticians, for statisticians." To see the fallacy of this statement substitute "statistician" with any other profession, e.g. farmer, philosopher, doctor. Good software is written by programmers.
- benhamner 11y agoHave you used R? As both a heavy R user and a software engineer, I can promise you that one of the quintessential aspects of R is it's actually "written by statisticians, for statisticians." You can't accuse R of having great code and language design. Or good code and language design. Or even mediocre code and language design. Imagine what you would get if you got a million monkeys drunk, put them on a roller coaster with laptops, and had them bang keys while they were upside down on loops. And then the result suddenly, miraculously runs and produces output. Now you understand R's software design.
- msellout 11y agoI think you read his comment too quickly. It seems you're agreeing that being "written by statisticians" is a disadvantage.
- jackmaney 11y agoYep. The idea that I should have to code defensively to make sure that the matrices I'm multiplying are of the correct size[1] is complete bullshit. cat(paste("That", "and", "string", "manipulation", "in", "R", "is", "a", "pain", "in", "the", "ass", sep=" ")) [1]: If A is n by m and B is p by q, and m is a multiple of p, then R will SILENTLY concatenate copies of B to itself to form an m by q matrix, and then do the multiplication.
- cwyers 11y agoR seems to have taken Perl's "there's more than one way to do it" as a credo, all of the official documentation I've seen seems to be an exercise in trying to say as little as possible with as many words as can be managed, all of the syntax seems to be deliberately designed to make no sense whatsoever and I can give it a task that JMP (no speed demon it) can finish in under an hour and not get results back after letting it churn for nearly a day. Everything about R makes me wonder what I've done with my life to end up using it.
- dalke 11y ago(NB: the article was written in May 2014. There are many comments at the bottom of the page.) "This has been the subject of huge controversy over the years, so Guido Van Rossum, inventor of the language, added a multiprocessing module." I think that assigns too much agency to van Rossum. PEP 371 describes the justification for bringing pyProcessing into the standard library. The primary developer was Richard Oudkerk, who along with Jesse Noller volunteered to maintain it in the standard library. While it's true that van Rossum accepted the PEP (on Thu, Jun 5, 2008 at 1:22 PM according to the mailing list), that's not quite the same thing as adding it. For that matter, the version control logs point out who originally added the code, in its most literal sense: user: Benjamin Peterson <benjamin@python.org> date: Wed Jun 11 02:40:25 2008 +0000 summary: add the multiprocessing package to fulfill PEP 371
- n0us 11y agoActually the GIL is not necessarily an issue for NumPy since C extensions can break the GIL. I don't see why this needs to be a winner take all competition between languages for statistics. It's worthwhile to have different approaches and just because in the author's opinion R is better than Python and Julia doesn't mean that it is always the better option no matter what.
- waitingkuo 11y agoI've tried to learn R several time, but its index started with 1 always stop me from learning.
- ww520 11y agoAnd Python's index starts at...what?
- invalidfunction 11y agozero
- deleted 11y ago[deleted]
- nulltype 11y agoI tried to learn Python but my computer was turned off so I gave up.
- Redoubts 11y agoI've always wondered why Julia went this route too. They more or less said "because matlab does it"
- idunning 11y agoSource? One of the best arguments, that is personally relevant to the work I do, is that indexing in math often starts at 1 (e.g. the indexing of rows and columns of a matrix). Translating this to code results in a mental overhead of mapping the math-index to the code-index, a cost I don't pay in Julia.
- Redoubts 11y agohttps://github.com/JuliaLang/julia/issues/558 https://github.com/JuliaLang/julia/issues/558 > There is a huge discussion about this on the mailing list; please see that. If 0 is mathematically "better", then why does the field of mathematics itself start indexes at 1? We have chosen 1 to be more similar to existing math software. A post above also appeals to mathematica. It's been a while since I've had to internalize the "cost" of zero indexing, so this never felt very compelling.
- msellout 11y agoThe author seems to have a very narrow view of "data science". In the comments he criticizes all of machine learning as simply unprincipled revival of nonparametric curve estimation. While there's much truth to the "unprincipled revival" accusation, some aspects of machine learning arose for good practicality-beats-purity software engineering rather than simple ignorance. My interpretation of the term "data science" is much more in line with practicality of engineering rather than the purity of traditional statistics. To that end, Python is leagues ahead.
- blumkvist 11y agoML and Statistics are so different to the point that comparison between them is worthless. ML predicts, Statistics explains (and can predict in some limited scenarios). Also ML workflow includes some statistical techniques.
- otabdeveloper 11y ago> he criticizes all of machine learning as simply unprincipled revival of nonparametric curve estimation He's right. 'Machine learning' is just statistics with better, cleaner names for things. The underlying theory is the same. (The commenter below says that 'statistics explains, ML predicts', but this isn't really true. Both statistics and ML build models, whether you use these models for prediction or explanation is up to you.) One problem is that the 'statistics' as usually taught in a college course or explained in a textbook comes from a much earlier age, before supercomputers were available to the average man; in essence, it's like a kind of machine learning as done by paper-and-pencil. In contrast, 'ML' assumes from the start that computing resources will be available.
- msellout 11y ago> In contrast, 'ML' assumes from the start that computing resources will be available. Right, but I'm not sure that "better, cleaner names for things" actually follows. Instead, I find that the ML folks just hacked their way to similar results as traditional statistics, but in many cases were comfortable with the algorithms as "black boxes" rather than having a clear understanding of why the algorithms worked. In that sense, the author's "unprincipled" criticism is valid. This is less true today, but the new research in convolutional neural nets shows how ML starts from hacking things until they produce practical results then backing into the theory of why. This habit has resulted in much duplication of effort and naming schemes. My ML prof at Georgia Tech (Isbell was awesome! http://www.cc.gatech.edu/~isbell/ http://www.cc.gatech.edu/~isbell/) constantly trashed "genetic algorithms" for being a silly form of randomized hill-climbing. The beneficial side of these less-principled techniques is that they happen to work on larger scale datasets. It turns out approximate results are more scalable than exact results.
- geomark 11y agoThese days I use both R and Python regularly. I don't have a strong opinion about one being better than the other, and I feel pretty good about that.
- 59nadir 11y ago> This vectorized R code turned out to be much faster than the Julia code–more than 1000 times faster, in fact, in the case of simulation 1000000 steps. For 100000000 steps, Julia actually is much faster than R, but the point is that the claims made about Julia’s speed advantage are really overblown. What are we supposed to glean from this paragraph?
- wmt 11y agoR beats Julia if you ignore all the cases where Julia beats R!
- kazagistar 11y agoJulia is slower while its "JIT" warms up, and then beats the snot out of R to make up for it?
- sgt101 11y ago" My impression of Julia’s parallel computation facilities so far, admittedly limited, is similar." 1. Not good to make assertions that you immediately undermine by admitting you don't know what you are talking about 2. I find @spawn and @fetch quite nice, in fact I think that Julia was built from the ground up around notions of parallelism (ie. multiple dispatch)