7 ms·

Owl – An OCaml Numerical Library

173 points by jimyl 9y ago

jimyl 9y ago
There will be a tutorial on using Owl in data science.[1] [1]: http://cufp.org/2017/c7-liang-wang-own-data-science-in-ocaml.html http://cufp.org/2017/c7-liang-wang-own-data-science-in-ocaml...
ziotom78 9y ago
Very nice project! Apart from the impressive set of features, what impresses me the most is the README, which is extremely clear and enjoyable.
- throwaway7645 9y ago
  Agreed...was surprised to see a lot of plots which is nice.
zerr 9y ago
Not to be confused with OWL framework from Borland.
- KineticLensman 9y ago
  Or the Web Ontology Language (OWL) [1] whose inconsistent acronym is not in fact a tribute to the spelling-challenged Owl character in Winnie-the-Pooh [2] [1] https://en.wikipedia.org/wiki/Web_Ontology_Language https://en.wikipedia.org/wiki/Web_Ontology_Language [2] https://en.wikipedia.org/wiki/List_of_Winnie-the-Pooh_characters#Owl https://en.wikipedia.org/wiki/List_of_Winnie-the-Pooh_charac...
  - pluma 9y ago
    Or the OWL region in Germany: https://en.wikipedia.org/wiki/Ostwestfalen-Lippe https://en.wikipedia.org/wiki/Ostwestfalen-Lippe TLAs tend to be horribly overloaded and confusing without a very obvious context.
    - TimonKnigge 9y ago
      Or the nocturnal bird.
      - pluma 9y ago
        Or, apparently, this: http://www.urbandictionary.com/define.php?term=the%20owl http://www.urbandictionary.com/define.php?term=the%20owl (NSFW)
        kronos29296 9y ago
        This chain is just a little bit too long. Just a little bit. Also most of the OWL in this chain I just don't know (not the bird the rest)
tbenst 9y ago
I looked at OCaml a few years back for scientific computing, but was turned off by the global lock GC for multithreading (similar to python). Anyone know if this has been changed yet? Edit: doesn't look like this has changed, but there's an effort at https://github.com/ocamllabs/ocaml-multicore https://github.com/ocamllabs/ocaml-multicore that is slowly making progress
- snowcrshd 9y ago
  IIRC, there was a PR preparing upstream ocaml for multicore. I think multicore capabilities will be here soon. Would things like LWT or async help you in any way?
  - srean 9y ago
    > I think multicore capabilities will be here soon. Just a word of caution to set expectations. The "will be here soon" status has been there for a while, to the extent that it now evokes memories of GNU Hurd, Duke Nukem Forever. It will be ready whenever it will be ready, its in good hands, but don't hold your breath, try to work around it for now.
    - avsm 9y ago
      Multicore is picking up pace, and there will be a paper on the formalised memory model at this year's OCaml Workshop in Oxford in September. There are a series of milestones to hit: the runtime GC, the memory model, the low-level programming model using one-shot continuations, and how it affects libraries running over it (e.g. algebraic effects). Each of these have associated papers and talks (see the ocamllabs.ionews section), so it's not quite fair to compare it to Duke Nukem Forever :-) Some quick links to recent papers: - memory model: http://kcsrk.info/papers/memory_model_ocaml17.pdf http://kcsrk.info/papers/memory_model_ocaml17.pdf - programming model: http://kcsrk.info/papers/awkward_effects_ml17.pdf http://kcsrk.info/papers/awkward_effects_ml17.pdf
      - srean 9y ago
        > so it's not quite fair to compare it to Duke Nukem Forever :-) Apologies if it came out harsh. Oodles of respect for all the work you guys are doing. Its a lot of work and that's why I said things are in good hands.
vog 9y ago
Wow, this even supports Algorithmic Differentiation! https://github.com/ryanrhymes/owl/wiki/Tutorial:-Algorithmic-Differentiation https://github.com/ryanrhymes/owl/wiki/Tutorial:-Algorithmic... (although "only" forward and reverse mode)
- Sean1708 9y ago
  Slightly off topic sorry, but how Algorithm Differentiation usually implemented? Do you just apply the chain rule to an AST, or is there something more complex you have to do?
  - etatoby 9y ago
    Notice that AD is a different technique than symbolic differentiation, which is what we do with pen and paper to compute a derivative. Wikipedia explains it properly, if a bit tersely: https://en.wikipedia.org/wiki/Automatic_differentiation https://en.wikipedia.org/wiki/Automatic_differentiation In the case of Owl, it would appear that you have to write your functions using a set of overloaded operators contained in module Maths here: https://github.com/ryanrhymes/owl/blob/master/lib/owl_algodiff_generic.ml#L585 https://github.com/ryanrhymes/owl/blob/master/lib/owl_algodi... Those operators, as explained in the Wikipedia page, manipulate "dual numbers," which are pairs of (number, epsilon) with their own specific algebra.
    - vog 9y ago
      > ... with their own specific algebra Nevertheless, for simple computations this specific algebra indeed boils down to "applying the chain rule to the AST" (as well as the diff rules for all primitive functions). In forward mode, the only difference to symbolic differentiation is that common terms are reused (items in the AST, treating the AST as DAG), so the expression terms do not explode in size, and the computation is very efficient. In reverse mode (and all other, more complicated modes), the chain rule still plays a central role, but is applied in a different way. Here, saying the "chain rule is applied to the AST" might still be technically correct, but would be very misleading, as the whole control flow is different, not to mention memory usage.
fithisux 9y ago
Any hope for windows users? Hopefully OPAM works under windows OOTB.
- etatoby 9y ago
  Have you tried the recent Linux subsystem for Windows, or Ubuntu for Windows? I have had a lot of success running various development tools in it. You can use pre-compiled Ubuntu binaries or compile your own stack, and it all runs at native speed, since there is no virtualization or emulation of any kind. https://news.ycombinator.com/item?id=11390545 https://news.ycombinator.com/item?id=11390545 https://news.ycombinator.com/item?id=14741714 https://news.ycombinator.com/item?id=14741714
  - fithisux 9y ago
    I prefer mingw all the way.
etatoby 9y ago
This is going to be extremely useful! I just recently settled on OCaml as my language of choice for most projects, after working with dozens of others, from C to Haskell. I was actually looking for a good numerical library, supporting complex numbers, matrix operations, and multi-dimensional arrays. I'm looking forward to using and contributing to this one!
- kornish 9y ago
  What are some of the factors that contribute to OCaml being your language of choice / do you feel limited by the poor multicore parallelism story?
  - Barrin92 9y ago
    Not OP but also OCaml user here, I consider Haskell's laziness to be the wrong default and think it makes it harder to reason about programs, and I prefer the 'mostly pure' approach of ML languages to strict purity. For me OCaml or the ML family hit the sweet spot of language design. I will agree that the parallelism issue can be a problem.
    - gbersac 9y ago
      Then you should try Idriss. This language is haskell with dependent typing and static by default.
    - m-j-fox 9y ago
      Please tell me about the parallelism problem? I love that Haskell can auto-parallelize and is very safe. But, I'm ready to kick it to the curb over stupid lazyiness-induced memory explosions. But I'd be sad to go back to having to worry about threads and safety. I'd love to give OCaml a go. Tell me why I'll end up disgruntled.
      - xfer 9y ago
        There is no parallelism, the runtime doesn't offer any. The best you can do is fork(). But there is an effort ongoing for a few years now, multicore-ocaml, which will be available mainstream when it's Ready(no ETA).
mrkgnao 9y ago
I didn't think "algorithmic differentiation" meant automatic differentiation. Instead: https://arxiv.org/abs/1610.07690 https://arxiv.org/abs/1610.07690
JacobiX 9y ago
We were just beginning to implement some linear algebra routines in ocaml for our library that parses text into structured data. This is a very good news. We will contribute back if we have any enhancements.
m-j-fox 9y ago
I find this design clean and appealing. Compared to the current fashion of all these Python libraries, it's a breath of fresh air. But I wonder if it can work. Python has going for it: kitchen sink of tools, easy to learn, tooling that took 20 years to develop, you already know it. Question: What's wrong with Matlab? Is it that it's closed and costs money? Other than that it seems clean, purpose-built, complete and not really expensive. Why all the open-source attempts to replace it? I really thought Julia was going to do that, but it's been years and they never got the tooling. They never solved the ridiculous, self-inflicted problem of having to recompile the entire system every time you run it. You're code will execute in 300ms, but only after 10 minutes of compiling. (Or did they make progress? I haven't checked in in a while.) Anyway, serious question: Why not Matlab?
- 2309kdujj 9y ago
  Julia's improved a lot on the issue you're mentioning, so you might want to give it a try again. I don't notice it anymore, although I used to. I do think Julia's performance, while pretty impressive, isn't quite what it's been billed as. How it compares to other systems in that regard, I don't know. As for Matlab, I think the big turnoff for me is its closed-source nature, which is maybe a more serious problem than you're giving it credit for. I also think that things like Python, that are more general-purpose, integrate much better than Matlab into other systems (e.g., web systems--you might not think it would matter if your estimation program doesn't integrate into a web app easily, but I didn't either and now I'm very sensitive to it). This is partially due to the open-source nature of things like Python, which lends itself to competition and experimentation, but also it being a more general-purpose language (this latter point is another sticking point with Julia, although wouldn't apply to OCaml). This OCaml library is really beautiful, actually. I looked into OCaml several years ago for this sort of thing, and if it had been around then, I might have invested more in it at the time, and kept using it. I'll have to revisit it.
  - gaius 9y ago
    I think the big turnoff for me is its closed-source nature Well, the issue of open-vs-closed is less clear with things like MATLAB. Sure you can't download it for free-as-in-beer, but if you own a copy, you are free to inspect its source code, as MATLAB is mainly written in MATLAB (same for Mathematica and other similar packages).

173 points by jimyl 9y ago

There will be a tutorial on using Owl in data science.[1] [1]: http://cufp.org/2017/c7-liang-wang-own-data-science-in-ocaml.html http://cufp.org/2017/c7-liang-wang-own-data-science-in-ocaml...

ziotom78 9y ago

Very nice project! Apart from the impressive set of features, what impresses me the most is the README, which is extremely clear and enjoyable.

throwaway7645 9y ago
Agreed...was surprised to see a lot of plots which is nice.

zerr 9y ago

Not to be confused with OWL framework from Borland.

KineticLensman 9y ago
Or the Web Ontology Language (OWL) [1] whose inconsistent acronym is not in fact a tribute to the spelling-challenged Owl character in Winnie-the-Pooh [2] [1] https://en.wikipedia.org/wiki/Web_Ontology_Language https://en.wikipedia.org/wiki/Web_Ontology_Language [2] https://en.wikipedia.org/wiki/List_of_Winnie-the-Pooh_characters#Owl https://en.wikipedia.org/wiki/List_of_Winnie-the-Pooh_charac...
- pluma 9y ago
  Or the OWL region in Germany: https://en.wikipedia.org/wiki/Ostwestfalen-Lippe https://en.wikipedia.org/wiki/Ostwestfalen-Lippe TLAs tend to be horribly overloaded and confusing without a very obvious context.
  - TimonKnigge 9y ago
    Or the nocturnal bird.
    - pluma 9y ago
      Or, apparently, this: http://www.urbandictionary.com/define.php?term=the%20owl http://www.urbandictionary.com/define.php?term=the%20owl (NSFW)
      - kronos29296 9y ago
        This chain is just a little bit too long. Just a little bit. Also most of the OWL in this chain I just don't know (not the bird the rest)

tbenst 9y ago

I looked at OCaml a few years back for scientific computing, but was turned off by the global lock GC for multithreading (similar to python). Anyone know if this has been changed yet? Edit: doesn't look like this has changed, but there's an effort at https://github.com/ocamllabs/ocaml-multicore https://github.com/ocamllabs/ocaml-multicore that is slowly making progress

snowcrshd 9y ago
IIRC, there was a PR preparing upstream ocaml for multicore. I think multicore capabilities will be here soon. Would things like LWT or async help you in any way?
- srean 9y ago
  > I think multicore capabilities will be here soon. Just a word of caution to set expectations. The "will be here soon" status has been there for a while, to the extent that it now evokes memories of GNU Hurd, Duke Nukem Forever. It will be ready whenever it will be ready, its in good hands, but don't hold your breath, try to work around it for now.
  - avsm 9y ago
    Multicore is picking up pace, and there will be a paper on the formalised memory model at this year's OCaml Workshop in Oxford in September. There are a series of milestones to hit: the runtime GC, the memory model, the low-level programming model using one-shot continuations, and how it affects libraries running over it (e.g. algebraic effects). Each of these have associated papers and talks (see the ocamllabs.ionews section), so it's not quite fair to compare it to Duke Nukem Forever :-) Some quick links to recent papers: - memory model: http://kcsrk.info/papers/memory_model_ocaml17.pdf http://kcsrk.info/papers/memory_model_ocaml17.pdf - programming model: http://kcsrk.info/papers/awkward_effects_ml17.pdf http://kcsrk.info/papers/awkward_effects_ml17.pdf
    - srean 9y ago
      > so it's not quite fair to compare it to Duke Nukem Forever :-) Apologies if it came out harsh. Oodles of respect for all the work you guys are doing. Its a lot of work and that's why I said things are in good hands.

vog 9y ago

Wow, this even supports Algorithmic Differentiation! https://github.com/ryanrhymes/owl/wiki/Tutorial:-Algorithmic-Differentiation https://github.com/ryanrhymes/owl/wiki/Tutorial:-Algorithmic... (although "only" forward and reverse mode)

Sean1708 9y ago
Slightly off topic sorry, but how Algorithm Differentiation usually implemented? Do you just apply the chain rule to an AST, or is there something more complex you have to do?
- etatoby 9y ago
  Notice that AD is a different technique than symbolic differentiation, which is what we do with pen and paper to compute a derivative. Wikipedia explains it properly, if a bit tersely: https://en.wikipedia.org/wiki/Automatic_differentiation https://en.wikipedia.org/wiki/Automatic_differentiation In the case of Owl, it would appear that you have to write your functions using a set of overloaded operators contained in module Maths here: https://github.com/ryanrhymes/owl/blob/master/lib/owl_algodiff_generic.ml#L585 https://github.com/ryanrhymes/owl/blob/master/lib/owl_algodi... Those operators, as explained in the Wikipedia page, manipulate "dual numbers," which are pairs of (number, epsilon) with their own specific algebra.
  - vog 9y ago
    > ... with their own specific algebra Nevertheless, for simple computations this specific algebra indeed boils down to "applying the chain rule to the AST" (as well as the diff rules for all primitive functions). In forward mode, the only difference to symbolic differentiation is that common terms are reused (items in the AST, treating the AST as DAG), so the expression terms do not explode in size, and the computation is very efficient. In reverse mode (and all other, more complicated modes), the chain rule still plays a central role, but is applied in a different way. Here, saying the "chain rule is applied to the AST" might still be technically correct, but would be very misleading, as the whole control flow is different, not to mention memory usage.

fithisux 9y ago

Any hope for windows users? Hopefully OPAM works under windows OOTB.

etatoby 9y ago
Have you tried the recent Linux subsystem for Windows, or Ubuntu for Windows? I have had a lot of success running various development tools in it. You can use pre-compiled Ubuntu binaries or compile your own stack, and it all runs at native speed, since there is no virtualization or emulation of any kind. https://news.ycombinator.com/item?id=11390545 https://news.ycombinator.com/item?id=11390545 https://news.ycombinator.com/item?id=14741714 https://news.ycombinator.com/item?id=14741714
- fithisux 9y ago
  I prefer mingw all the way.

etatoby 9y ago

This is going to be extremely useful! I just recently settled on OCaml as my language of choice for most projects, after working with dozens of others, from C to Haskell. I was actually looking for a good numerical library, supporting complex numbers, matrix operations, and multi-dimensional arrays. I'm looking forward to using and contributing to this one!

kornish 9y ago
What are some of the factors that contribute to OCaml being your language of choice / do you feel limited by the poor multicore parallelism story?
- Barrin92 9y ago
  Not OP but also OCaml user here, I consider Haskell's laziness to be the wrong default and think it makes it harder to reason about programs, and I prefer the 'mostly pure' approach of ML languages to strict purity. For me OCaml or the ML family hit the sweet spot of language design. I will agree that the parallelism issue can be a problem.
  - gbersac 9y ago
    Then you should try Idriss. This language is haskell with dependent typing and static by default.
  - m-j-fox 9y ago
    Please tell me about the parallelism problem? I love that Haskell can auto-parallelize and is very safe. But, I'm ready to kick it to the curb over stupid lazyiness-induced memory explosions. But I'd be sad to go back to having to worry about threads and safety. I'd love to give OCaml a go. Tell me why I'll end up disgruntled.
    - xfer 9y ago
      There is no parallelism, the runtime doesn't offer any. The best you can do is fork(). But there is an effort ongoing for a few years now, multicore-ocaml, which will be available mainstream when it's Ready(no ETA).

mrkgnao 9y ago

I didn't think "algorithmic differentiation" meant automatic differentiation. Instead: https://arxiv.org/abs/1610.07690 https://arxiv.org/abs/1610.07690

JacobiX 9y ago

We were just beginning to implement some linear algebra routines in ocaml for our library that parses text into structured data. This is a very good news. We will contribute back if we have any enhancements.

m-j-fox 9y ago

I find this design clean and appealing. Compared to the current fashion of all these Python libraries, it's a breath of fresh air. But I wonder if it can work. Python has going for it: kitchen sink of tools, easy to learn, tooling that took 20 years to develop, you already know it. Question: What's wrong with Matlab? Is it that it's closed and costs money? Other than that it seems clean, purpose-built, complete and not really expensive. Why all the open-source attempts to replace it? I really thought Julia was going to do that, but it's been years and they never got the tooling. They never solved the ridiculous, self-inflicted problem of having to recompile the entire system every time you run it. You're code will execute in 300ms, but only after 10 minutes of compiling. (Or did they make progress? I haven't checked in in a while.) Anyway, serious question: Why not Matlab?

2309kdujj 9y ago
Julia's improved a lot on the issue you're mentioning, so you might want to give it a try again. I don't notice it anymore, although I used to. I do think Julia's performance, while pretty impressive, isn't quite what it's been billed as. How it compares to other systems in that regard, I don't know. As for Matlab, I think the big turnoff for me is its closed-source nature, which is maybe a more serious problem than you're giving it credit for. I also think that things like Python, that are more general-purpose, integrate much better than Matlab into other systems (e.g., web systems--you might not think it would matter if your estimation program doesn't integrate into a web app easily, but I didn't either and now I'm very sensitive to it). This is partially due to the open-source nature of things like Python, which lends itself to competition and experimentation, but also it being a more general-purpose language (this latter point is another sticking point with Julia, although wouldn't apply to OCaml). This OCaml library is really beautiful, actually. I looked into OCaml several years ago for this sort of thing, and if it had been around then, I might have invested more in it at the time, and kept using it. I'll have to revisit it.
- gaius 9y ago
  I think the big turnoff for me is its closed-source nature Well, the issue of open-vs-closed is less clear with things like MATLAB. Sure you can't download it for free-as-in-beer, but if you own a copy, you are free to inspect its source code, as MATLAB is mainly written in MATLAB (same for Mathematica and other similar packages).