5 ms·

Programming R at native speed using Haskell

145 points by psibi 11y ago

jzwinck 11y ago
How does garbage collection work? Does this version improve on the R I know from a year ago, which had a reference counting system with only three values: 0, 1, and 2+?
- rfergie 11y ago
  https://tweag.github.io/HaskellR/docs/managing-memory.html https://tweag.github.io/HaskellR/docs/managing-memory.html https://tweag.github.io/HaskellR/docs/memory-allocation.html https://tweag.github.io/HaskellR/docs/memory-allocation.html Both these links have more information on this. As far as I can tell (and most of this is way over my head) the native R garbage collection is used
  - mboes 11y ago
    Yes - the R GC manages R objects allocated by the R interpreter, and conversely the Haskell GC manages Haskell objects allocated by the Haskell runtime.
- hadley 11y ago
  You're confusing reference counting (which is used primarily for R's copy-on-modify strategy) and its generational garbage collector.
  - jzwinck 11y ago
    In that case, does HaskellR interfere with copy-on-modify in R, for example does it increment the reference count of R objects, then try to decrement them (which R doesn't or didn't really support), leaving those objects appearing to be multiply-referenced, and therefore forcing unnecessary copies as we see in plain R?
ximeng 11y ago
Does anyone have good links to documentation on quasiquotation in Haskell? Last time I looked I struggled to find a good summary.
- chriswarbo 11y ago
  I've been teaching myself Template Haskell recently, and found https://ocharles.org.uk/blog/guest-posts/2014-12-22-template-haskell.html https://ocharles.org.uk/blog/guest-posts/2014-12-22-template... to be a really understandable explanation. It specifically doesn't cover quasiquotation, but it has many links to things which do.
- psibi 11y ago
  This is a fairly nice quasiquotation tutorial: https://www.fpcomplete.com/user/marcin/quasiquotation-101 https://www.fpcomplete.com/user/marcin/quasiquotation-101
baldfat 11y ago
I love R and I love programming in it, but I have always thought I need to learn Julia due to a few things including speed, but that the R community keeps coming out with great answers to those questions where Julia keeps my interest. I don't see how this would speed up my typical R programming or code to be worth learning Haskell. What am I missing and yes I read the whole article. I have found Haskell very difficult to use with Cabal and its package management. The reason I went to Haskell was to teach myself functional programming. After struggling and going through 2 books I still felt like I hated Haskell due to making the environment just work. I work in three different locations and to get all 5 computers to work in Haskell was a serious pain. This lead me to see what else is out there. It lead me to https://www.coursera.org/course/proglang https://www.coursera.org/course/proglang which introduced me to Racket and I loved it and felt that Racket was the perfect fit for me.
- mboes 11y ago
  You should give Haskell another shot. Since last you tried, this happened: https://github.com/commercialhaskell/stack https://github.com/commercialhaskell/stack. In fact using stack is the recommended installation method for HaskellR, rather than using the cabal-install command.
- chriswarbo 11y ago
  > I have found Haskell very difficult to use with Cabal and its package management. The reason I went to Haskell was to teach myself functional programming. After struggling and going through 2 books I still felt like I hated Haskell due to making the environment just work. I work in three different locations and to get all 5 computers to work in Haskell was a serious pain. Package management is certainly the worst thing about Haskell at the moment. Just yesterday I had to fight with cabal, and sit through hours of compilation, to build my application with profiling enabled :( There's recently been a surge of activity in this area though, which seems to have been driven by FPComplete and consulting with industry: - First there's "Stackage", which is basically a curated version of the package repository Hackage. Anyone can upload a new package to Hackage, and package authors can make breaking changes whenever they like. Stackage takes consistent snapshots of Hackage, where the package versions are known to work together. - Next there's "Stack", which is an alternative UI for Cabal (ie. it replaces the commandline tools, but uses the same libraries and infrastructure). Nice features of stack are that it can fetch/install GHC (so "bootstrapping" is much easier), and it doesn't use a global package database (cabal has this feature with "sandboxes", but they're opt-in). - Personally, I use Nix for managing Haskell packages. It's a bit like stack or cabal sandboxes taken to the extreme, although its Windows support is still pretty experimental. It will fetch pre-built packages from a cache, rather than building stuff locally (as long as you're using the defaults, at least). Take a look at https://www.fpcomplete.com/blog/2015/06/stack-0-1-release https://www.fpcomplete.com/blog/2015/06/stack-0-1-release for more info. > This lead me to see what else is out there. It lead me to https://www.coursera.org/course/proglang https://www.coursera.org/course/proglang which introduced me to Racket and I loved it and felt that Racket was the perfect fit for me. Racket (and Scheme in general) is also an excellent functional programming language. Having dynamic types and macros makes it a very different beast to Haskell, so it's definitely worth learning both :)
jbssm 11y ago
I don't know almost anything about Haskell (tried to learn it some years ago but never pass form the basics). Yet from what I'm seeing we are only calling R functions from Haskell. I'm probably missing something on the explanation, but how does this speed up R?
- dagw 11y ago
  Often what is slow in an R program is not the core analysis function you want to call, but everything you have to do before and after you call that function, IE turning your raw data into a data structure containing the relevant data to feed into the function and taking the output of the function and munging it into whatever output format you actually need. This should help speed up those bits quite a lot.
  - jbssm 11y ago
    I see, thanks. I my own code I suspect that it's the for loops I can't vectorise that are making R taking so long in some operations, so this is where this project would help. Must give it a try then.
    - Lofkin 11y ago
      You can use python, call out to R libraries with Rpy2 and speed up loops to faster than Julia and around native Speed with numba
    - Mikeb85 11y ago
      In R its very easy to simply create a function in C++ or Fortran, then call that. Plus there's plenty of functions already that you can call instead of using for loops in R (which really aren't idiomatic).
- thesz 11y ago
  I don't know about speed, but with this package you can have Haskell types at your disposal. If you use them properly, you would not be able to compare p values of experiments directly (a common mistake).
JanneVee 11y ago
Well haskell is not the only option. From .NET: http://rdotnet.codeplex.com/ http://rdotnet.codeplex.com/ there is also a dataprovider directly to F# : http://bluemountaincapital.github.io/FSharpRProvider/ http://bluemountaincapital.github.io/FSharpRProvider/ Also there is C++ binding available also: http://www.rcpp.org/ http://www.rcpp.org/
- pron 11y ago
  Or just run it with FastR (https://bitbucket.org/allr/fastr https://bitbucket.org/allr/fastr). Here it is in action (running with JS and C in the same REPL): https://dl.dropboxusercontent.com/u/292832/useR_multilang_demo.mp4 https://dl.dropboxusercontent.com/u/292832/useR_multilang_de...
jmount 11y ago
Looks fun. However, what is REALLY slow in R isn't the interpreter or garbage collector: but some of the object re-allocation patterns you accidentally trigger. Here is my note making some comparisons and recommendations on writing fast code in R: http://www.win-vector.com/blog/2015/07/efficient-accumulation-in-r/ http://www.win-vector.com/blog/2015/07/efficient-accumulatio...
Mikeb85 11y ago
While this really has nothing to do with programming R, this is a neat way to use Haskell for some statistics/data science tasks whilst making use of all the work that has gone into R over the years. I personally don't see this as a way of using R, rather a way of using Haskell.

145 points by psibi 11y ago

How does garbage collection work? Does this version improve on the R I know from a year ago, which had a reference counting system with only three values: 0, 1, and 2+?

rfergie 11y ago
https://tweag.github.io/HaskellR/docs/managing-memory.html https://tweag.github.io/HaskellR/docs/managing-memory.html https://tweag.github.io/HaskellR/docs/memory-allocation.html https://tweag.github.io/HaskellR/docs/memory-allocation.html Both these links have more information on this. As far as I can tell (and most of this is way over my head) the native R garbage collection is used
- mboes 11y ago
  Yes - the R GC manages R objects allocated by the R interpreter, and conversely the Haskell GC manages Haskell objects allocated by the Haskell runtime.
hadley 11y ago
You're confusing reference counting (which is used primarily for R's copy-on-modify strategy) and its generational garbage collector.
- jzwinck 11y ago
  In that case, does HaskellR interfere with copy-on-modify in R, for example does it increment the reference count of R objects, then try to decrement them (which R doesn't or didn't really support), leaving those objects appearing to be multiply-referenced, and therefore forcing unnecessary copies as we see in plain R?

ximeng 11y ago

Does anyone have good links to documentation on quasiquotation in Haskell? Last time I looked I struggled to find a good summary.

chriswarbo 11y ago
I've been teaching myself Template Haskell recently, and found https://ocharles.org.uk/blog/guest-posts/2014-12-22-template-haskell.html https://ocharles.org.uk/blog/guest-posts/2014-12-22-template... to be a really understandable explanation. It specifically doesn't cover quasiquotation, but it has many links to things which do.
psibi 11y ago
This is a fairly nice quasiquotation tutorial: https://www.fpcomplete.com/user/marcin/quasiquotation-101 https://www.fpcomplete.com/user/marcin/quasiquotation-101

baldfat 11y ago

I love R and I love programming in it, but I have always thought I need to learn Julia due to a few things including speed, but that the R community keeps coming out with great answers to those questions where Julia keeps my interest. I don't see how this would speed up my typical R programming or code to be worth learning Haskell. What am I missing and yes I read the whole article. I have found Haskell very difficult to use with Cabal and its package management. The reason I went to Haskell was to teach myself functional programming. After struggling and going through 2 books I still felt like I hated Haskell due to making the environment just work. I work in three different locations and to get all 5 computers to work in Haskell was a serious pain. This lead me to see what else is out there. It lead me to https://www.coursera.org/course/proglang https://www.coursera.org/course/proglang which introduced me to Racket and I loved it and felt that Racket was the perfect fit for me.

mboes 11y ago
You should give Haskell another shot. Since last you tried, this happened: https://github.com/commercialhaskell/stack https://github.com/commercialhaskell/stack. In fact using stack is the recommended installation method for HaskellR, rather than using the cabal-install command.
chriswarbo 11y ago
> I have found Haskell very difficult to use with Cabal and its package management. The reason I went to Haskell was to teach myself functional programming. After struggling and going through 2 books I still felt like I hated Haskell due to making the environment just work. I work in three different locations and to get all 5 computers to work in Haskell was a serious pain. Package management is certainly the worst thing about Haskell at the moment. Just yesterday I had to fight with cabal, and sit through hours of compilation, to build my application with profiling enabled :( There's recently been a surge of activity in this area though, which seems to have been driven by FPComplete and consulting with industry: - First there's "Stackage", which is basically a curated version of the package repository Hackage. Anyone can upload a new package to Hackage, and package authors can make breaking changes whenever they like. Stackage takes consistent snapshots of Hackage, where the package versions are known to work together. - Next there's "Stack", which is an alternative UI for Cabal (ie. it replaces the commandline tools, but uses the same libraries and infrastructure). Nice features of stack are that it can fetch/install GHC (so "bootstrapping" is much easier), and it doesn't use a global package database (cabal has this feature with "sandboxes", but they're opt-in). - Personally, I use Nix for managing Haskell packages. It's a bit like stack or cabal sandboxes taken to the extreme, although its Windows support is still pretty experimental. It will fetch pre-built packages from a cache, rather than building stuff locally (as long as you're using the defaults, at least). Take a look at https://www.fpcomplete.com/blog/2015/06/stack-0-1-release https://www.fpcomplete.com/blog/2015/06/stack-0-1-release for more info. > This lead me to see what else is out there. It lead me to https://www.coursera.org/course/proglang https://www.coursera.org/course/proglang which introduced me to Racket and I loved it and felt that Racket was the perfect fit for me. Racket (and Scheme in general) is also an excellent functional programming language. Having dynamic types and macros makes it a very different beast to Haskell, so it's definitely worth learning both :)

jbssm 11y ago

I don't know almost anything about Haskell (tried to learn it some years ago but never pass form the basics). Yet from what I'm seeing we are only calling R functions from Haskell. I'm probably missing something on the explanation, but how does this speed up R?

dagw 11y ago
Often what is slow in an R program is not the core analysis function you want to call, but everything you have to do before and after you call that function, IE turning your raw data into a data structure containing the relevant data to feed into the function and taking the output of the function and munging it into whatever output format you actually need. This should help speed up those bits quite a lot.
- jbssm 11y ago
  I see, thanks. I my own code I suspect that it's the for loops I can't vectorise that are making R taking so long in some operations, so this is where this project would help. Must give it a try then.
  - Lofkin 11y ago
    You can use python, call out to R libraries with Rpy2 and speed up loops to faster than Julia and around native Speed with numba
  - Mikeb85 11y ago
    In R its very easy to simply create a function in C++ or Fortran, then call that. Plus there's plenty of functions already that you can call instead of using for loops in R (which really aren't idiomatic).
thesz 11y ago
I don't know about speed, but with this package you can have Haskell types at your disposal. If you use them properly, you would not be able to compare p values of experiments directly (a common mistake).

JanneVee 11y ago

Well haskell is not the only option. From .NET: http://rdotnet.codeplex.com/ http://rdotnet.codeplex.com/ there is also a dataprovider directly to F# : http://bluemountaincapital.github.io/FSharpRProvider/ http://bluemountaincapital.github.io/FSharpRProvider/ Also there is C++ binding available also: http://www.rcpp.org/ http://www.rcpp.org/

pron 11y ago
Or just run it with FastR (https://bitbucket.org/allr/fastr https://bitbucket.org/allr/fastr). Here it is in action (running with JS and C in the same REPL): https://dl.dropboxusercontent.com/u/292832/useR_multilang_demo.mp4 https://dl.dropboxusercontent.com/u/292832/useR_multilang_de...

jmount 11y ago

Looks fun. However, what is REALLY slow in R isn't the interpreter or garbage collector: but some of the object re-allocation patterns you accidentally trigger. Here is my note making some comparisons and recommendations on writing fast code in R: http://www.win-vector.com/blog/2015/07/efficient-accumulation-in-r/ http://www.win-vector.com/blog/2015/07/efficient-accumulatio...

Mikeb85 11y ago

While this really has nothing to do with programming R, this is a neat way to use Haskell for some statistics/data science tasks whilst making use of all the work that has gone into R over the years. I personally don't see this as a way of using R, rather a way of using Haskell.