4 ms·

I looked at OCaml a few years back for scientific computing, but was turned off by the global lock GC for multithreading (similar to python). Anyone know if thi

by tbenst 9y ago

I looked at OCaml a few years back for scientific computing, but was turned off by the global lock GC for multithreading (similar to python). Anyone know if this has been changed yet? Edit: doesn't look like this has changed, but there's an effort at https://github.com/ocamllabs/ocaml-multicore https://github.com/ocamllabs/ocaml-multicore that is slowly making progress

snowcrshd 9y ago
IIRC, there was a PR preparing upstream ocaml for multicore. I think multicore capabilities will be here soon. Would things like LWT or async help you in any way?
- srean 9y ago
  > I think multicore capabilities will be here soon. Just a word of caution to set expectations. The "will be here soon" status has been there for a while, to the extent that it now evokes memories of GNU Hurd, Duke Nukem Forever. It will be ready whenever it will be ready, its in good hands, but don't hold your breath, try to work around it for now.
  - avsm 9y ago
    Multicore is picking up pace, and there will be a paper on the formalised memory model at this year's OCaml Workshop in Oxford in September. There are a series of milestones to hit: the runtime GC, the memory model, the low-level programming model using one-shot continuations, and how it affects libraries running over it (e.g. algebraic effects). Each of these have associated papers and talks (see the ocamllabs.ionews section), so it's not quite fair to compare it to Duke Nukem Forever :-) Some quick links to recent papers: - memory model: http://kcsrk.info/papers/memory_model_ocaml17.pdf http://kcsrk.info/papers/memory_model_ocaml17.pdf - programming model: http://kcsrk.info/papers/awkward_effects_ml17.pdf http://kcsrk.info/papers/awkward_effects_ml17.pdf
    - srean 9y ago
      > so it's not quite fair to compare it to Duke Nukem Forever :-) Apologies if it came out harsh. Oodles of respect for all the work you guys are doing. Its a lot of work and that's why I said things are in good hands.
    - JacobiX 9y ago
      I second on this. The multicore support is not far away. Right now we have a memory model for multicore OCaml and even a multicore OCaml ARM64 backend.
escapetech 9y ago
Also problematic for numeric code in Ocaml is the lack of native interoperability with C data types and native arrays and instead having to use time-consuming abstractions such as BigArray or marshalling via FFI. Unfortunately, solutions like as CTypes only seem to compound the problem by not addressing a particular issue with a runtime, but finding clever ways around the limitation.
JacobiX 9y ago
A lot of scientific computing is done in python (numpy) and even in Lua (using torch the great tensor library). Much of the computations are done in the GPU anyway.
m-j-fox 9y ago
Does it matter a lot for this sort of application (which I assume is interactive data analysis)? I think if you're worried about performance then you want a back-end to map your computation out to a big cluster, no? Like, write OCaml, target hadoop would be the logical thing if you need HPC. Of course someone will have to write that compiler (not it). :P

by tbenst 9y ago

snowcrshd 9y ago

IIRC, there was a PR preparing upstream ocaml for multicore. I think multicore capabilities will be here soon. Would things like LWT or async help you in any way?

srean 9y ago
> I think multicore capabilities will be here soon. Just a word of caution to set expectations. The "will be here soon" status has been there for a while, to the extent that it now evokes memories of GNU Hurd, Duke Nukem Forever. It will be ready whenever it will be ready, its in good hands, but don't hold your breath, try to work around it for now.
- avsm 9y ago
  Multicore is picking up pace, and there will be a paper on the formalised memory model at this year's OCaml Workshop in Oxford in September. There are a series of milestones to hit: the runtime GC, the memory model, the low-level programming model using one-shot continuations, and how it affects libraries running over it (e.g. algebraic effects). Each of these have associated papers and talks (see the ocamllabs.ionews section), so it's not quite fair to compare it to Duke Nukem Forever :-) Some quick links to recent papers: - memory model: http://kcsrk.info/papers/memory_model_ocaml17.pdf http://kcsrk.info/papers/memory_model_ocaml17.pdf - programming model: http://kcsrk.info/papers/awkward_effects_ml17.pdf http://kcsrk.info/papers/awkward_effects_ml17.pdf
  - srean 9y ago
    > so it's not quite fair to compare it to Duke Nukem Forever :-) Apologies if it came out harsh. Oodles of respect for all the work you guys are doing. Its a lot of work and that's why I said things are in good hands.
  - JacobiX 9y ago
    I second on this. The multicore support is not far away. Right now we have a memory model for multicore OCaml and even a multicore OCaml ARM64 backend.

escapetech 9y ago

Also problematic for numeric code in Ocaml is the lack of native interoperability with C data types and native arrays and instead having to use time-consuming abstractions such as BigArray or marshalling via FFI. Unfortunately, solutions like as CTypes only seem to compound the problem by not addressing a particular issue with a runtime, but finding clever ways around the limitation.

JacobiX 9y ago

A lot of scientific computing is done in python (numpy) and even in Lua (using torch the great tensor library). Much of the computations are done in the GPU anyway.

m-j-fox 9y ago

Does it matter a lot for this sort of application (which I assume is interactive data analysis)? I think if you're worried about performance then you want a back-end to map your computation out to a big cluster, no? Like, write OCaml, target hadoop would be the logical thing if you need HPC. Of course someone will have to write that compiler (not it). :P