4 ms·

I've personally been interested in implementing something similar on top of BLAS / LAPACK in Scheme (either CHICKEN or Guile). Any tips from the authors? Any ti

I've personally been interested in implementing something similar on top of BLAS / LAPACK in Scheme (either CHICKEN or Guile). Any tips from the authors? Any tips at all? My main questions concern how you decided on an API, or how you decided to construct primitives (matrices, ndarrays) within the library itself? Basically, I'm just not sure where to start with such a large project. Currently I'm just trying to learn the language and ecosystem (how the FFI works, etc.) to the point where something like this may be feasible. I want to switch over to using scheme as my de-facto language, and this would be the number 1 library that would make it possible; Unfortunately, the lack of resources on the web for learning how to make good lisp-like APIs is lacking, and I really need some help figuring out how to do it. Unlike Python, we don't have PEP 8, and I think the difficulty in constructing a domain specific language for numerics is under-appreciated. Thoughts? Tips? It seems MatLisp has been around for some time (2000, I guess?), so there's got to be someone with considerable experience here, right?

enupten 12y ago
Getting API right is hard because you have no idea about the scope of your project at first. It is easy to get initial inspiration from other projects, but I still believe that having an API set in stone is a bad idea; Matlisp itself has literally been rewritten ~4 times. That said, I'd recommend checking out Numpy, and projects in lisp lisp scmutils, lisp-matrix, matlisp, femlisp...
- agibsonccc 12y ago
  I wrote what amounts to this for scala/JVM. The API is definitely hard. I found numpy to be the best to follow in terms of API. It's what most people understand.
- ThatGeoGuy 12y ago
  Thanks for the response. I definitely know about Numpy, it's currently what I use instead of lisp / scheme, mostly because it's currently the least painful way to do numerical computing (in my opinion, that is). Unfortunately, Python is not a lisp (and yes, I know of Hy) which does bring forth it's own limitations. The biggest concern I have with re-implementing Numpy in Scheme / Lisp is that it's very object oriented. Certainly, many of the ufunc methods in Numpy can be composed or chained in a very functional style, but at its core Numpy is Pythonic, and Pythonic code means using OOP for abstraction. Mainly, I'm concerned with how one actually transfers this over into Lisp like languages. Sure, CHICKEN has coops, Guile has goops, which are basically Common Lisps' CLOS, but taking the concept of Numpy and expressing it in a CLOS-like system sometimes just feels wrong. That said, I think the guys behind clojure.core.matrix have done some work facing similar challenges, but I don't really know enough to evaluate if they've done the right thing. Moreover, Clojure itself almost makes this job easier, because it has interfaces / protocols from the get-go, whereas Scheme and CL don't quite share the same properties. In any case, thanks again for your reply. If you ever write anything regarding those 4 rewrites, or if you have written anything regarding your decisions on refactoring and improving an API, I'd love to read it. I know a lot of people hold Numpy up as a gold standard, but I still find it very hard to put in explicit terms what makes programming with Numpy arrays more pleasant than using MATLAB arrays, Eigen (C++) matrices / vectors, or other similar systems.
  - enupten 12y ago
    Well, sure you can't exactly copy what Numpy does, but it is not too difficult to do what they do with CLOS. I think getting the slicing sematics right is crucial for making things easy, and without an infix reader it is hard, for a Lisp to compete on usability in Numerical computing.
    - BruceM 12y ago
      CLOS-style OO and infix syntax leads pretty naturally to Dylan (http://opendylan.org/ http://opendylan.org/) ... I'd love to see numeric stuff in Dylan, but no time for it myself. :(
      - enupten 12y ago
        I like my math infix, and my programs in prefix; Dylan - atleast the newer incarnation - does not fit the bill.

enupten 12y ago

Getting API right is hard because you have no idea about the scope of your project at first. It is easy to get initial inspiration from other projects, but I still believe that having an API set in stone is a bad idea; Matlisp itself has literally been rewritten ~4 times. That said, I'd recommend checking out Numpy, and projects in lisp lisp scmutils, lisp-matrix, matlisp, femlisp...

agibsonccc 12y ago
I wrote what amounts to this for scala/JVM. The API is definitely hard. I found numpy to be the best to follow in terms of API. It's what most people understand.
ThatGeoGuy 12y ago
Thanks for the response. I definitely know about Numpy, it's currently what I use instead of lisp / scheme, mostly because it's currently the least painful way to do numerical computing (in my opinion, that is). Unfortunately, Python is not a lisp (and yes, I know of Hy) which does bring forth it's own limitations. The biggest concern I have with re-implementing Numpy in Scheme / Lisp is that it's very object oriented. Certainly, many of the ufunc methods in Numpy can be composed or chained in a very functional style, but at its core Numpy is Pythonic, and Pythonic code means using OOP for abstraction. Mainly, I'm concerned with how one actually transfers this over into Lisp like languages. Sure, CHICKEN has coops, Guile has goops, which are basically Common Lisps' CLOS, but taking the concept of Numpy and expressing it in a CLOS-like system sometimes just feels wrong. That said, I think the guys behind clojure.core.matrix have done some work facing similar challenges, but I don't really know enough to evaluate if they've done the right thing. Moreover, Clojure itself almost makes this job easier, because it has interfaces / protocols from the get-go, whereas Scheme and CL don't quite share the same properties. In any case, thanks again for your reply. If you ever write anything regarding those 4 rewrites, or if you have written anything regarding your decisions on refactoring and improving an API, I'd love to read it. I know a lot of people hold Numpy up as a gold standard, but I still find it very hard to put in explicit terms what makes programming with Numpy arrays more pleasant than using MATLAB arrays, Eigen (C++) matrices / vectors, or other similar systems.
- enupten 12y ago
  Well, sure you can't exactly copy what Numpy does, but it is not too difficult to do what they do with CLOS. I think getting the slicing sematics right is crucial for making things easy, and without an infix reader it is hard, for a Lisp to compete on usability in Numerical computing.
  - BruceM 12y ago
    CLOS-style OO and infix syntax leads pretty naturally to Dylan (http://opendylan.org/ http://opendylan.org/) ... I'd love to see numeric stuff in Dylan, but no time for it myself. :(
    - enupten 12y ago
      I like my math infix, and my programs in prefix; Dylan - atleast the newer incarnation - does not fit the bill.