12 ms·

A better inliner for OCaml, and why it matters

135 points by ch 11y ago

vegabook 11y ago
I love Ocaml but have ditched it for Erlang/Elixir, losing a lot of elegance on the way, but massively gaining on horizontal scalability and pragmatism. I would love to stay with Ocaml but there just isn't enough of a clear roadmap to it on multicore (seemingly fraught with controversy within the community), or even multi-node distributed computing that does not require me to reverse engineer someone's 10-year-old, badly documented thesis project. I almost feel like the Ocaml project is stuck optimizing a single-core past, and that it might be better to devise a new Ocaml-based language altogether, designed from the ground up for distributed computing. I'd be there immediately.
- giancarlostoro 11y ago
  I'm not an expert in either language but have you also given F# a though? The compiler is open source and should run under Mono. I do know it's heavily influenced by OCaml.
  - vegabook 11y ago
    I really need Linux to be the first class citizen (yes I see MS has bought Xamarin but still). Also last I looked, F# wasn't looking any better than Ocaml on multinode distributed computing (please do correct me if I'm wrong).
    - giancarlostoro 11y ago
      Xamarin is a C# shop, mono as a platform for F# is independent of this, and if I'm not mistaken the F# compiler is open source[0] under the Apache 2.0 License. There are a few shops who have praised F# for it's multicore capabilities[1], but again I am not a full expert so it may require independent research. MonoDevelop supports F# as well. There are Linux build instructions in the GitHub repository. [0]: https://github.com/fsharp/fsharp https://github.com/fsharp/fsharp [1]: http://fsharp.org/testimonials/#grange-insurance-1 http://fsharp.org/testimonials/#grange-insurance-1
    - ZenoArrow 11y ago
      If you're looking for to use a distributed architecture with F# it's worth checking out Akka.NET: http://getakka.net/ http://getakka.net/
    - profquail 11y ago
      As another poster mentioned, you can use Akka.Net with F# (that project was started and is still run by F# developers). Another option with deeper language integration is mbrace: http://mbrace.io/ http://mbrace.io/ Vesa Karvonen ported CML to F# as Hopac, which has very good performance: https://github.com/Hopac/Hopac https://github.com/Hopac/Hopac Finally, there're Orleans and Naiad from MSFT: https://github.com/dotnet/orleans https://github.com/dotnet/orleans https://github.com/MicrosoftResearch/Naiad https://github.com/MicrosoftResearch/Naiad
- amirmc 11y ago
  Have you seen any of the work on OCaml multicore? KC's blog posts talk about it e.g. http://kcsrk.info/ocaml/multicore/2015/05/20/effects-multicore/ http://kcsrk.info/ocaml/multicore/2015/05/20/effects-multico...
  - FreezerburnV 11y ago
    Is there any kind of timeline for when multicore is actually going to be implemented though? Last thing I saw was some posts from last year saying something about "if everything goes well, it will be in 4.03". As of now, I haven't been able to find anything about it being in 4.03, or anything else about the timeline to implementation. It would be great if it were coming soon, but I'm pretty sure there have been rumblings about it for years now, with nothing more solid than "Coming Soon(tm)".
    - vegabook 11y ago
      ... moreover, github's charts don't show it to have much of a recent pulse. https://github.com/ocamllabs/ocaml-multicore/graphs/contributors https://github.com/ocamllabs/ocaml-multicore/graphs/contribu...
      - amirmc 11y ago
        That doesn't mean there isn't work taking place. Here's a recent video where KC talks about it. https://m.youtube.com/watch?list=PLnqUlCo055hU46uoONmhYGUbYAK27Y6rS&v=jZB8CZRRuEo https://m.youtube.com/watch?list=PLnqUlCo055hU46uoONmhYGUbYA... Edit: and there's also more than one repo out there.
      - avsm 11y ago
        The runtime aspects of the multicore runtime has been pretty stable. Most of the effort currently is going into the algebraic effects extension that is used to map direct-style concurrency into multiple (parallel) cores: https://github.com/ocamllabs/ocaml-effects https://github.com/ocamllabs/ocaml-effects
    - amirmc 11y ago
- rixed 11y ago
  This is not clear. Multicore and distributed computing are unrelated at best. To make it clearer, can you explain what you are doing in erlang that you couldn't do in ocaml?
  - vegabook 11y ago
    I am message passing thousands of financial ticks per second to hundreds of terminal-user consumers, each of which may wish to preselect filters that they wish applied to each series, or combinations of series. This is trivially easy to organize in Erlang/OTP, it is trivially easy to scale, and I have no clue how I would organize it in Ocaml without going to message passing first principles, and in which case I would have to build the entire load balancing, PID and security infrastructure from scratch. Moreover I do not see how you do not see that multicore and multi-node are related. On Erlang, having two 8-core boxes is almost the same as having one 16-core box. Again, if there is something I am missing about the Ocaml environment I would be happy to know about it.
    - wyager 11y ago
      You may be interested in Cloud Haskell. It's basically Erlang networking nicely re-implemented in Haskell with the strong static typing you expect from Haskell. Gets you some of the advantages of both erlang and ocaml.
      - lmm 11y ago
        Similarly, Scala with Akka gives you an ML-like language with first-class Linux support, excellent multicore performance, and Erlang-style distribution.
      - elihu 11y ago
        For what it's worth, "Cloud Haskell", the project, corresponds to the distributed-process library, which may be easier to search for than cloud Haskell. The API is described in chapter 14 of Parallel and Concurrent Programming in Haskell, by Simon Marlow. (I haven't been paying close attention to the distributed-process library, so I don't know whether that chapter is up-to-date with recent versions of the library.)
- Ono-Sendai 11y ago
  I'm working on such a language, called Winter: http://www.forwardscattering.org/post/22 http://www.forwardscattering.org/post/22 Hopefully it will be open sourced in the not-to-distant future.
  - dman 11y ago
    Just read through your blog - found it very informative! Hope to see Winter open sourced soon.
    - Ono-Sendai 11y ago
      Thanks :)
- xiaoma 11y ago
  What elegance was lost? Where is the biggest pain point on your new stack?
- bajsejohannes 11y ago
  A lot of the things I loved in Ocaml, I've found in Rust. Additionally, Rust has great multicore support. You won't find very mature multi-node libraries in Rust yet, though.
  - LeonidasXIV 11y ago
    It is not that surprising, since the Rust compiler used to be written in OCaml, so naturally OCaml influenced the dasign of Rust in some way. Glad to see OCaml concepts get popularized by other languages, that's a win-win situation for everyone.
- fsloth 11y ago
  I understand you are writing a server. To be fair, not all compute loads are embarrassingly parallel. For those tasks single thread performance - especially from the point of view of memory bandwidth - is critical. Single core is not 'past'. 1. CPU:s are not getting that much faster every generation (one could argue that this means parallellization is critical, but...) 2. The number of cores on the average desktop is not that much. One of the reasons I think is that in many cases the individual tasks that must be performed sequentially in a typical desktop program are so tiny or hard to split that that the platform overhead and complexity of multiple threads cause the benefits from actually utilizing multiple threads to be a bit harder to reach than just using a language with a nice parallellization story. By 'benefits' I mean making the program actually faster for the user. Of course, I'm talking about Amdahl's law https://en.m.wikipedia.org/wiki/Amdahl%27s_law https://en.m.wikipedia.org/wiki/Amdahl%27s_law
  - vegabook 11y ago
    Yep - my use case analyses the streaming incoming data using compute-intensive algorithms. I need very high performance single core, for which Erlang is unsuitable. Therefore I'm using calls to numpy. But then I need to distribute data widely in heterogenous, changeable ways, and for that, Erlang wins hands down.
    - twic 11y ago
      Are you calling from Erlang to Numpy? If so, wow, and how is that working out?
      - vegabook 11y ago
        Nothing fancy - I message pass via zeromq to a cluster running python/numpy instances. It's pretty coarse granularity - the (BFGS curve fitting) optimizations I need from numpy will take between 0.5 and 3 seconds usually so I'm not suffering too much on serialization time in comparison. The optimizations involved do not need to be run in real time against the very latest streaming data, they just need to be as recent as possible, which makes this solution quite workable.
mwcampbell 11y ago
I wonder what kind of impact this will have on the performance of Mirage. It seems to me that whole-program optimization is potentially a great benefit for a unikernel written almost entirely in a high-level language like OCaml.
- wk_end 11y ago
  To be clear, I don't believe flambda is providing anything on the scale of whole program optimization; it inlines more aggressively, but not the entire program.
  - Drup 11y ago
    That is not exactly correct, it also does cross module inlining. Now, suppose you tweak a bit your parameters to inline ... liberally. It's going to inline all the mirage functors directly into your main, which enables various things. Of course, that's going to take a bit of time to compile. Also, IIRC, all the ingredients are in place for link time optimizations, but they were not developed for ocaml 4.03.
cm3 11y ago
What about dead code elimination? A simple executable using Core is >20MB.
- lpw25 11y ago
  The size of Core executables is mostly addressed by module aliases. Unfortunately the public release of Core still uses packing instead of module aliases because oasis/ocamlbuild don't easily support them.
  - cm3 11y ago
    Will we also get dead code elimination generally speaking in the compiler? I remember a mailing post where one of the flambda devs announced he managed to generate standalone hello world of 43k but that was just a PoC.
  - cm3 11y ago
    Slightly off-topic, but will there be a consolidation of oasis, ocamlbuild, corebuild, to go with opam?
tempodox 11y ago
I'm really looking forward to modular implicits. Having something like Haskell type classes is a great tool for abstraction.

135 points by ch 11y ago

I love Ocaml but have ditched it for Erlang/Elixir, losing a lot of elegance on the way, but massively gaining on horizontal scalability and pragmatism. I would love to stay with Ocaml but there just isn't enough of a clear roadmap to it on multicore (seemingly fraught with controversy within the community), or even multi-node distributed computing that does not require me to reverse engineer someone's 10-year-old, badly documented thesis project. I almost feel like the Ocaml project is stuck optimizing a single-core past, and that it might be better to devise a new Ocaml-based language altogether, designed from the ground up for distributed computing. I'd be there immediately.

giancarlostoro 11y ago
I'm not an expert in either language but have you also given F# a though? The compiler is open source and should run under Mono. I do know it's heavily influenced by OCaml.
- vegabook 11y ago
  I really need Linux to be the first class citizen (yes I see MS has bought Xamarin but still). Also last I looked, F# wasn't looking any better than Ocaml on multinode distributed computing (please do correct me if I'm wrong).
  - giancarlostoro 11y ago
    Xamarin is a C# shop, mono as a platform for F# is independent of this, and if I'm not mistaken the F# compiler is open source[0] under the Apache 2.0 License. There are a few shops who have praised F# for it's multicore capabilities[1], but again I am not a full expert so it may require independent research. MonoDevelop supports F# as well. There are Linux build instructions in the GitHub repository. [0]: https://github.com/fsharp/fsharp https://github.com/fsharp/fsharp [1]: http://fsharp.org/testimonials/#grange-insurance-1 http://fsharp.org/testimonials/#grange-insurance-1
  - ZenoArrow 11y ago
    If you're looking for to use a distributed architecture with F# it's worth checking out Akka.NET: http://getakka.net/ http://getakka.net/
  - profquail 11y ago
    As another poster mentioned, you can use Akka.Net with F# (that project was started and is still run by F# developers). Another option with deeper language integration is mbrace: http://mbrace.io/ http://mbrace.io/ Vesa Karvonen ported CML to F# as Hopac, which has very good performance: https://github.com/Hopac/Hopac https://github.com/Hopac/Hopac Finally, there're Orleans and Naiad from MSFT: https://github.com/dotnet/orleans https://github.com/dotnet/orleans https://github.com/MicrosoftResearch/Naiad https://github.com/MicrosoftResearch/Naiad
amirmc 11y ago
Have you seen any of the work on OCaml multicore? KC's blog posts talk about it e.g. http://kcsrk.info/ocaml/multicore/2015/05/20/effects-multicore/ http://kcsrk.info/ocaml/multicore/2015/05/20/effects-multico...
- FreezerburnV 11y ago
  Is there any kind of timeline for when multicore is actually going to be implemented though? Last thing I saw was some posts from last year saying something about "if everything goes well, it will be in 4.03". As of now, I haven't been able to find anything about it being in 4.03, or anything else about the timeline to implementation. It would be great if it were coming soon, but I'm pretty sure there have been rumblings about it for years now, with nothing more solid than "Coming Soon(tm)".
  - vegabook 11y ago
    ... moreover, github's charts don't show it to have much of a recent pulse. https://github.com/ocamllabs/ocaml-multicore/graphs/contributors https://github.com/ocamllabs/ocaml-multicore/graphs/contribu...
    - amirmc 11y ago
      That doesn't mean there isn't work taking place. Here's a recent video where KC talks about it. https://m.youtube.com/watch?list=PLnqUlCo055hU46uoONmhYGUbYAK27Y6rS&v=jZB8CZRRuEo https://m.youtube.com/watch?list=PLnqUlCo055hU46uoONmhYGUbYA... Edit: and there's also more than one repo out there.
    - avsm 11y ago
      The runtime aspects of the multicore runtime has been pretty stable. Most of the effort currently is going into the algebraic effects extension that is used to map direct-style concurrency into multiple (parallel) cores: https://github.com/ocamllabs/ocaml-effects https://github.com/ocamllabs/ocaml-effects
  - amirmc 11y ago
rixed 11y ago
This is not clear. Multicore and distributed computing are unrelated at best. To make it clearer, can you explain what you are doing in erlang that you couldn't do in ocaml?
- vegabook 11y ago
  I am message passing thousands of financial ticks per second to hundreds of terminal-user consumers, each of which may wish to preselect filters that they wish applied to each series, or combinations of series. This is trivially easy to organize in Erlang/OTP, it is trivially easy to scale, and I have no clue how I would organize it in Ocaml without going to message passing first principles, and in which case I would have to build the entire load balancing, PID and security infrastructure from scratch. Moreover I do not see how you do not see that multicore and multi-node are related. On Erlang, having two 8-core boxes is almost the same as having one 16-core box. Again, if there is something I am missing about the Ocaml environment I would be happy to know about it.
  - wyager 11y ago
    You may be interested in Cloud Haskell. It's basically Erlang networking nicely re-implemented in Haskell with the strong static typing you expect from Haskell. Gets you some of the advantages of both erlang and ocaml.
    - lmm 11y ago
      Similarly, Scala with Akka gives you an ML-like language with first-class Linux support, excellent multicore performance, and Erlang-style distribution.
    - elihu 11y ago
      For what it's worth, "Cloud Haskell", the project, corresponds to the distributed-process library, which may be easier to search for than cloud Haskell. The API is described in chapter 14 of Parallel and Concurrent Programming in Haskell, by Simon Marlow. (I haven't been paying close attention to the distributed-process library, so I don't know whether that chapter is up-to-date with recent versions of the library.)
Ono-Sendai 11y ago
I'm working on such a language, called Winter: http://www.forwardscattering.org/post/22 http://www.forwardscattering.org/post/22 Hopefully it will be open sourced in the not-to-distant future.
- dman 11y ago
  Just read through your blog - found it very informative! Hope to see Winter open sourced soon.
  - Ono-Sendai 11y ago
    Thanks :)
xiaoma 11y ago
What elegance was lost? Where is the biggest pain point on your new stack?
bajsejohannes 11y ago
A lot of the things I loved in Ocaml, I've found in Rust. Additionally, Rust has great multicore support. You won't find very mature multi-node libraries in Rust yet, though.
- LeonidasXIV 11y ago
  It is not that surprising, since the Rust compiler used to be written in OCaml, so naturally OCaml influenced the dasign of Rust in some way. Glad to see OCaml concepts get popularized by other languages, that's a win-win situation for everyone.
fsloth 11y ago
I understand you are writing a server. To be fair, not all compute loads are embarrassingly parallel. For those tasks single thread performance - especially from the point of view of memory bandwidth - is critical. Single core is not 'past'. 1. CPU:s are not getting that much faster every generation (one could argue that this means parallellization is critical, but...) 2. The number of cores on the average desktop is not that much. One of the reasons I think is that in many cases the individual tasks that must be performed sequentially in a typical desktop program are so tiny or hard to split that that the platform overhead and complexity of multiple threads cause the benefits from actually utilizing multiple threads to be a bit harder to reach than just using a language with a nice parallellization story. By 'benefits' I mean making the program actually faster for the user. Of course, I'm talking about Amdahl's law https://en.m.wikipedia.org/wiki/Amdahl%27s_law https://en.m.wikipedia.org/wiki/Amdahl%27s_law
- vegabook 11y ago
  Yep - my use case analyses the streaming incoming data using compute-intensive algorithms. I need very high performance single core, for which Erlang is unsuitable. Therefore I'm using calls to numpy. But then I need to distribute data widely in heterogenous, changeable ways, and for that, Erlang wins hands down.
  - twic 11y ago
    Are you calling from Erlang to Numpy? If so, wow, and how is that working out?
    - vegabook 11y ago
      Nothing fancy - I message pass via zeromq to a cluster running python/numpy instances. It's pretty coarse granularity - the (BFGS curve fitting) optimizations I need from numpy will take between 0.5 and 3 seconds usually so I'm not suffering too much on serialization time in comparison. The optimizations involved do not need to be run in real time against the very latest streaming data, they just need to be as recent as possible, which makes this solution quite workable.

mwcampbell 11y ago

I wonder what kind of impact this will have on the performance of Mirage. It seems to me that whole-program optimization is potentially a great benefit for a unikernel written almost entirely in a high-level language like OCaml.

wk_end 11y ago
To be clear, I don't believe flambda is providing anything on the scale of whole program optimization; it inlines more aggressively, but not the entire program.
- Drup 11y ago
  That is not exactly correct, it also does cross module inlining. Now, suppose you tweak a bit your parameters to inline ... liberally. It's going to inline all the mirage functors directly into your main, which enables various things. Of course, that's going to take a bit of time to compile. Also, IIRC, all the ingredients are in place for link time optimizations, but they were not developed for ocaml 4.03.

cm3 11y ago

What about dead code elimination? A simple executable using Core is >20MB.

lpw25 11y ago
The size of Core executables is mostly addressed by module aliases. Unfortunately the public release of Core still uses packing instead of module aliases because oasis/ocamlbuild don't easily support them.
- cm3 11y ago
  Will we also get dead code elimination generally speaking in the compiler? I remember a mailing post where one of the flambda devs announced he managed to generate standalone hello world of 43k but that was just a PoC.
- cm3 11y ago
  Slightly off-topic, but will there be a consolidation of oasis, ocamlbuild, corebuild, to go with opam?

tempodox 11y ago

I'm really looking forward to modular implicits. Having something like Haskell type classes is a great tool for abstraction.