10 ms·

Multicore OCaml: April 2021

145 points by sadiq 5y ago

sadiq 5y ago
Happy to answer any questions.
- NanoCoaster 5y ago
  I'm sorry if these are dumb questions, I'd be totally fine with some pointers in the right directions if you don't want to waste your time :) I'm under the impression that the implementation is based on algebraic effects. Does that include extending the language in a way that lets users define their own effects & handlers? Also, how's the performance? Last time I looked this stuff up (and played around with Eff), which was quite a while ago, I was told that regular usage of effects may impact performance quite noticeably.
  - sadiq 5y ago
    Not dumb questions at all. Multicore adds parallelism via Domains (which are essentially heavyweight threads) and concurrency via Effects and fibers. There's a multicore GC that supports both of those. We plan to upstream things in two parts. First domains-only parallelism and then effects as a follow-up. When the latter lands users will be able to define their own effects and handlers, yes. Performance is pretty good, you can see our PLDI2021 paper for a proper performance evaluation and loads more details: https://arxiv.org/abs/2104.00250 https://arxiv.org/abs/2104.00250
    - NanoCoaster 5y ago
      Thank you, interesting stuff. Very much looking forward to user-definable effects! I like Ocaml in principal, but haven't yet strayed beyond learning the very basics some time ago. Seeing a usable implementation of algebraic effects in a somewhat popular language...that I gotta see :)
      - sadiq 5y ago
        Worth pointing out you don't just need to stop at seeing it, you can get started playing around with it today: https://github.com/ocaml-multicore/multicore-opam#install-multicore-ocaml https://github.com/ocaml-multicore/multicore-opam#install-mu...
        NanoCoaster 5y ago
        Cool, thanks for the link. Will definitely play around with it as soon as I find the time. Too many programming languages, too little free time, sadly.
- mirekrusin 5y ago
  Is it currently possible to play with multicore ocaml on M1?
  - mseri 5y ago
    I think not yet: https://github.com/ocaml-multicore/ocaml-multicore/issues/86#issuecomment-822000783 https://github.com/ocaml-multicore/ocaml-multicore/issues/86... Fwiw, OCaml itself can be compiled natively on M1.
    - sadiq 5y ago
      Yes, arm64 support is non-functional but we're planning to get it back in to a working state very soon.
- wuschel 5y ago
  Hello sadiq, I have some more general questions for you: As understand that currently, the situation is analogous to Python. The GIL allows for process based concurrency, with the well known disadvantage regarding memory consumption. Also, my guess is that OCaml relies library based solutions at the moment? 1) What does the introduction of MultiCore potential mean for Ocaml? Will Ocaml be a much better fit to run a webserver backend? Could you perhaps give a comparison to other programming languages? 2) How will Ocaml stand out with Multicore in the PL world, and what solutions would be it uniquely suited for? 3) What tools are going to be present to deal with bugs introduced by your new runtime e.g. race conditions? 4) If one wants to have a go at Multicore and play around with it, where/how do I start? Many thanks!
  - sadiq 5y ago
    Good questions! 1) As I mentioned in https://news.ycombinator.com/item?id=27142502 https://news.ycombinator.com/item?id=27142502 there is support for parallelism and concurrency. Giving an example of where these might be useful in a webservice. The addition of shared-memory parallelism is beneficial where you might have a great deal of shared state that needs to be used to service requests. An in-memory cache is a good example - with a processed-based approach managing read/writes and avoiding significant overhead from marshalling the data is difficult. Concurrency via effects at a minimum can make writing network-based services much more pleasant (and debuggable!). See the examples in https://arxiv.org/abs/2104.00250 https://arxiv.org/abs/2104.00250 where programs can be written in a direct-style similar to blocking IO but using effects are transformed to use asynchronous interfaces. There's work going on in the project at the moment to build fast cross-platform IO implementations that sit atop of uring/gcd/iocp. 3) This is a good question and one we're still working on. I think one of the lead developers KC has a few good ideas about instrumentation we can do to enable detecting races to global state. It's certainly going to be an issue for people porting large codebases. 4) This is the place to start: https://github.com/ocaml-multicore/multicore-opam#install-multicore-ocaml https://github.com/ocaml-multicore/multicore-opam#install-mu... .
    - wuschel 5y ago
      Thank you for the your answer! I will check out the URIs.
- intc 5y ago
  Could you explain (in simple terms if possible) how the Multicore OCaml achieves a memory model which is much simpler on more efficient than in Java or C (mentioned at https://github.com/ocaml-multicore/ocaml-multicore/wiki https://github.com/ocaml-multicore/ocaml-multicore/wiki)? Didn't see any mentions of critical sections (mutexes) with C++ examples in the documentation ("Bounding Data Races in Space and Time"). I'm not sure I understand the comparisons the writers are presenting.
  - jlouis 5y ago
    The key problem is program transformation, and in particular optimizations. Different CPUs/ISAs, and compilers, might want to transform your program to make it run faster. However, in the multi-core setting, data races pose bounds and limits on how much you can trigger those optimizations. The program doesn't generally execute sequentially in a way that can be entirely reasoned about. Instructions might be reordered for the sake of the program to run faster. Programmers can't work with that. So one proposes a memory model. Follow these rules, and our optimizations won't alter the behavior of the program. They kind-of describes what happens "in between" the critical sections of the program, hence the lack of a mutex mention. The paper presents a local property and then shows, formally, that this property is enough to guarantee an efficient memory model. That is, a model in which you can perform optimizations, while programmers can still reason about the programs behavior. The crux of the paper is that the property is local. This is new, because memory models which came before it are global: to reason about correctness, you have to consider the whole program, rather than consider a small (local) subset. OCaml requires more safety than most programming languages, so this is good for the fact that you can now compose local fragments of OCaml programs, without having to worry about a global safety property. The property is also simpler for programmers to reason about. The way you "use" the paper is that you adapt your optimizations to follow the property, and you make sure that the virtual memory model is implemented the same way on different architectures. Finally, the examples: they explore the idea of a local reasoning. In particular, they show why the (existing) global properties fail if you view them under the stronger requirement of local reasoning. It's the setup for the paper, since it means you can't just use the existing models. They need to be adapted if you want a more localized property.
  - 5y ago
- hajile 5y ago
  StandardML implementations have had good multicore support for decades now despite having only a tiny fraction of the users and development time. Meanwhile, Ocaml has been promising support for years. What in Ocaml makes this so much harder to implement?
  - rwmj 5y ago
    OCaml (or its immediate predecessor[1]) had a multicore implementation, but it was dropped because of its complexity and effect on single-thread performance. The challenge is to add it back in a way that is maintainable and doesn't negatively affect current users. [1] https://www.researchgate.net/publication/2774662_Concurrent_generational_garbage_collector_for_a_multithreaded_implementation_of_ML https://www.researchgate.net/publication/2774662_Concurrent_...
  - sadiq 5y ago
    As the sibling comment mentions, the hard part is actually retrofitting multicore whilst maintaining compatibility _and_ performance. Our paper last year covers most of why this is tricky: https://arxiv.org/abs/2004.11663 https://arxiv.org/abs/2004.11663
- 4ad 5y ago
  Algebraic effects as described in the OCaml papers about effects are dynamically typed. I recently heard in some talk that this idea was abandoned, and the new algebraic effects are in fact statically typed. Do I remember this correctly? If so, where can I read about these new algebraic effects?
  - kcsrk 5y ago
    Abandoned is perhaps too strong a term :-). Effect handlers in the language are supported by fibers, lightweight stacklets managed by the runtime. The details of the implementation can be found in the upcoming research paper in PLDI'21 conference [1]. The effect handlers in Multicore OCaml today do not provide effect safety. Programs are not statically guaranteed to handle all the effects they may perform. This is only as bad as exceptions in OCaml and every other mainstream language with exceptions. We are working on developing an effect system, which will ensure effect safety i.e, the compiler ensures that all the effects performed are caught. You also get a nice inferred type that says what effects a particular function may perform; if it performs none, then it is a pure function! This implementation would still use the current fiber support in the runtime. Leo White, one of the developers of Multicore OCaml had given a talk on this new effect system a few years ago [2]. That's the best place today to learn about the new effect handlers. The plan is to first add the fiber runtime support to OCaml without the syntax extensions for effect handlers, and then introduce syntax along with the effect system. [1] https://arxiv.org/abs/2104.00250 https://arxiv.org/abs/2104.00250 [2] https://www.janestreet.com/tech-talks/effective-programming/ https://www.janestreet.com/tech-talks/effective-programming/
- agumonkey 5y ago
  How much resources are going into multicore compared to other needs / improvements in ocaml (if any) ?
  - sadiq 5y ago
    Not sure I have enough of an overview to comment on that in general, unfortunately. The overview Anil gave at the end of last year on the OCaml Platform should give you an idea of the many other strands of work that are going on: https://ocaml.org/platform/ https://ocaml.org/platform/
    - agumonkey 5y ago
      Thanks man, and Best wishes
terminalserver 5y ago
For the lay person, what is the advantage of ocaml over other languages? Why would I reach for it?
- Steltek 5y ago
  Jane Street, financial trading company, uses OCaml a ton and publishes a lot of their libraries. I played with it some back when I was dabbling in every language I could get my hands on. About when Scala was new (and constantly breaking between releases) and before Rust, Go, JS v8 were around. I really liked the functional aspect, which wasn't bolted on after the fact, and that it could be compiled to a real binary, not interpreted bytecode. In my naive youth, I viewed AOT compilation as the path to a true high level language that was also fast. While it was neat, I moved on to Lisps/Schemes and now modern JS is my very happy compromise of functional and practical.
- Taikonerd 5y ago
  OCaml is... * functional * strongly, statically typed * garbage-collected OCaml can compile to a native binary, or to JS via ReScript (formerly BuckleScript).
  - christophilus 5y ago
    Also, it compiles very quickly compared to most other statically typed languages; especially compared to more advanced ones.
  - Zababa 5y ago
    > OCaml can compile to a native binary, or to JS via ReScript (formerly BuckleScript). It can also compile to JS via js_of_ocaml!
- richeyryan 5y ago
  OCaml is statically typed functional programming language. It's a cousin of Haskell. It has some nice things like automatic type inference so you don't have to write very many type annotations. It's not as focused on purity as Haskell so its easier to mutate state where you want to but you get a lot of the niceties of ML programming languages like pattern matching, variants, structural typing in places. It also has object-oriented features, though they aren't widely used the attitude is something like OO is there if we need it and we're definitely willing to use it in places that require it. Its pretty fast for a functional language and you could probably get pretty far with it before you'd have to consider using a real low level langauge. The disadvantages I think are pretty uncontroversial: a smaller community, not as many libraries, a bit of a fractured stdlib and build situation and until now no multicore.
toolslive 5y ago
What I'm still missing is a strategy to move existing ocaml programs that use (let's say) Lwt for concurrency and a bit of C for things that were trivial to parallelize to move to multicore ocaml AND benefit from it.
- kcsrk 5y ago
  (One of the Multicore OCaml devs here) We have prototyped offloading CPU intensive computations in Lwt programs using Multicore OCaml [1]. We're currently working with Lwt maintainers to upstream it. [1] https://sudha247.github.io/2020/10/01/lwt-multicore/ https://sudha247.github.io/2020/10/01/lwt-multicore/
  - toolslive 5y ago
    thx! Another question. Suppose you're starting from scratch, is it worth going the effects based async io route ? https://github.com/kayceesrk/ocaml-aeio https://github.com/kayceesrk/ocaml-aeio seems very interesting...
    - kcsrk 5y ago
      Yes, exactly. The reason why monadic concurrency libraries such as Lwt and Async is that the OCaml language does not support concurrency natively. If it did, we would have built something similar to the `ocaml-aeio` library. Btw there is a modern instantiation of `ocaml-aeio` called `eieio` [1] which supports Linux's io-uring. Eventually, this will be extended to support all the modern I/O stacks on different platforms, and also support performing I/O on multiple cores. [1] https://github.com/ocaml-multicore/eioio https://github.com/ocaml-multicore/eioio
  - anuragsoni 5y ago
    > We have prototyped offloading CPU intensive computations in Lwt programs using Multicore OCaml This is very interesting! I'm wondering if you are aware of any discussions with the async maintainers about what their plans are with the multicore runtime?
philzook 5y ago
It seems like great work. Something I don't understand is that I hear people mentioning a lack of multicore support as blocker to using a programming language. I don't think this is something I have ever felt. What problem domains require multicore?
- blacktriangle 5y ago
  I feel the same way and I can't help but feel like I am missing something. Concurrency is still inherently more complicated than serial processing no matter what language features you add, and it's amazing how far you can get just by scaling across OS processes. I know there are plenty of domains that do want multicore (ML, graphics), but it seems like even the people writing line of business apps think lack of multicore is a deal breaker for them.
  - hderms 5y ago
    Yeah I really don't know because OCaml already has async constructs. It doesn't seem like it should be a dealbreaker for most. That being said, it has a reputation for being extremely performant so it might just be that people feel like it's so close to being usable in a lot of situations it currently isn't so there is more aggregate desire to allow for multicore execution
  - toolslive 5y ago
    Often, it's just about finding a stick. They don't want to use OCaml and the absence of multi-core features gives them the excuse they were looking for. Now they can inform management: "OCaml is not an option. we'll stick with <insert PL here>"
    - sshine 5y ago
      It often is. But sometimes it is also about finding more reasons to use one of your favorite languages. I’m doing pseudo-async PHP at work, and having proper support would certainly make these solutions less brittle.
dang 5y ago
Past related threads: Multicore OCaml: Feb 2021 with new preprint on Effect Handlers - https://news.ycombinator.com/item?id=26424785 https://news.ycombinator.com/item?id=26424785 - March 2021 (29 comments) Multicore OCaml: October 2020 - https://news.ycombinator.com/item?id=25034538 https://news.ycombinator.com/item?id=25034538 - Nov 2020 (9 comments) Multicore OCaml: September 2020 - https://news.ycombinator.com/item?id=24719124 https://news.ycombinator.com/item?id=24719124 - Oct 2020 (43 comments) Parallel Programming in Multicore OCaml - https://news.ycombinator.com/item?id=23740869 https://news.ycombinator.com/item?id=23740869 - July 2020 (15 comments) Multicore OCaml: May 2020 update - https://news.ycombinator.com/item?id=23380370 https://news.ycombinator.com/item?id=23380370 - June 2020 (17 comments) Multicore OCaml: March 2020 update - https://news.ycombinator.com/item?id=22727975 https://news.ycombinator.com/item?id=22727975 - March 2020 (37 comments) Multicore OCaml: Feb 2020 update - https://news.ycombinator.com/item?id=22443428 https://news.ycombinator.com/item?id=22443428 - Feb 2020 (80 comments) State of Multicore OCaml [pdf] - https://news.ycombinator.com/item?id=17416797 https://news.ycombinator.com/item?id=17416797 - June 2018 (103 comments) OCaml-multicore now at 4.04.2 - https://news.ycombinator.com/item?id=16646181 https://news.ycombinator.com/item?id=16646181 - March 2018 (4 comments) A deep dive into Multicore OCaml garbage collector - https://news.ycombinator.com/item?id=14780159 https://news.ycombinator.com/item?id=14780159 - July 2017 (89 comments) Lock-free programming for the masses - https://news.ycombinator.com/item?id=11907584 https://news.ycombinator.com/item?id=11907584 - June 2016 (29 comments) Lock-free programming for the masses - https://news.ycombinator.com/item?id=11893911 https://news.ycombinator.com/item?id=11893911 - June 2016 (4 comments) OCaml 4.03 will, “if all goes well”, support multicore - https://news.ycombinator.com/item?id=9582980 https://news.ycombinator.com/item?id=9582980 - May 2015 (113 comments) Multicore OCaml - https://news.ycombinator.com/item?id=8003699 https://news.ycombinator.com/item?id=8003699 - July 2014 (1 comment)

145 points by sadiq 5y ago

sadiq 5y ago

Happy to answer any questions.

NanoCoaster 5y ago
I'm sorry if these are dumb questions, I'd be totally fine with some pointers in the right directions if you don't want to waste your time :) I'm under the impression that the implementation is based on algebraic effects. Does that include extending the language in a way that lets users define their own effects & handlers? Also, how's the performance? Last time I looked this stuff up (and played around with Eff), which was quite a while ago, I was told that regular usage of effects may impact performance quite noticeably.
- sadiq 5y ago
  Not dumb questions at all. Multicore adds parallelism via Domains (which are essentially heavyweight threads) and concurrency via Effects and fibers. There's a multicore GC that supports both of those. We plan to upstream things in two parts. First domains-only parallelism and then effects as a follow-up. When the latter lands users will be able to define their own effects and handlers, yes. Performance is pretty good, you can see our PLDI2021 paper for a proper performance evaluation and loads more details: https://arxiv.org/abs/2104.00250 https://arxiv.org/abs/2104.00250
  - NanoCoaster 5y ago
    Thank you, interesting stuff. Very much looking forward to user-definable effects! I like Ocaml in principal, but haven't yet strayed beyond learning the very basics some time ago. Seeing a usable implementation of algebraic effects in a somewhat popular language...that I gotta see :)
    - sadiq 5y ago
      Worth pointing out you don't just need to stop at seeing it, you can get started playing around with it today: https://github.com/ocaml-multicore/multicore-opam#install-multicore-ocaml https://github.com/ocaml-multicore/multicore-opam#install-mu...
      - NanoCoaster 5y ago
        Cool, thanks for the link. Will definitely play around with it as soon as I find the time. Too many programming languages, too little free time, sadly.
mirekrusin 5y ago
Is it currently possible to play with multicore ocaml on M1?
- mseri 5y ago
  I think not yet: https://github.com/ocaml-multicore/ocaml-multicore/issues/86#issuecomment-822000783 https://github.com/ocaml-multicore/ocaml-multicore/issues/86... Fwiw, OCaml itself can be compiled natively on M1.
  - sadiq 5y ago
    Yes, arm64 support is non-functional but we're planning to get it back in to a working state very soon.
wuschel 5y ago
Hello sadiq, I have some more general questions for you: As understand that currently, the situation is analogous to Python. The GIL allows for process based concurrency, with the well known disadvantage regarding memory consumption. Also, my guess is that OCaml relies library based solutions at the moment? 1) What does the introduction of MultiCore potential mean for Ocaml? Will Ocaml be a much better fit to run a webserver backend? Could you perhaps give a comparison to other programming languages? 2) How will Ocaml stand out with Multicore in the PL world, and what solutions would be it uniquely suited for? 3) What tools are going to be present to deal with bugs introduced by your new runtime e.g. race conditions? 4) If one wants to have a go at Multicore and play around with it, where/how do I start? Many thanks!
- sadiq 5y ago
  Good questions! 1) As I mentioned in https://news.ycombinator.com/item?id=27142502 https://news.ycombinator.com/item?id=27142502 there is support for parallelism and concurrency. Giving an example of where these might be useful in a webservice. The addition of shared-memory parallelism is beneficial where you might have a great deal of shared state that needs to be used to service requests. An in-memory cache is a good example - with a processed-based approach managing read/writes and avoiding significant overhead from marshalling the data is difficult. Concurrency via effects at a minimum can make writing network-based services much more pleasant (and debuggable!). See the examples in https://arxiv.org/abs/2104.00250 https://arxiv.org/abs/2104.00250 where programs can be written in a direct-style similar to blocking IO but using effects are transformed to use asynchronous interfaces. There's work going on in the project at the moment to build fast cross-platform IO implementations that sit atop of uring/gcd/iocp. 3) This is a good question and one we're still working on. I think one of the lead developers KC has a few good ideas about instrumentation we can do to enable detecting races to global state. It's certainly going to be an issue for people porting large codebases. 4) This is the place to start: https://github.com/ocaml-multicore/multicore-opam#install-multicore-ocaml https://github.com/ocaml-multicore/multicore-opam#install-mu... .
  - wuschel 5y ago
    Thank you for the your answer! I will check out the URIs.
intc 5y ago
Could you explain (in simple terms if possible) how the Multicore OCaml achieves a memory model which is much simpler on more efficient than in Java or C (mentioned at https://github.com/ocaml-multicore/ocaml-multicore/wiki https://github.com/ocaml-multicore/ocaml-multicore/wiki)? Didn't see any mentions of critical sections (mutexes) with C++ examples in the documentation ("Bounding Data Races in Space and Time"). I'm not sure I understand the comparisons the writers are presenting.
- jlouis 5y ago
  The key problem is program transformation, and in particular optimizations. Different CPUs/ISAs, and compilers, might want to transform your program to make it run faster. However, in the multi-core setting, data races pose bounds and limits on how much you can trigger those optimizations. The program doesn't generally execute sequentially in a way that can be entirely reasoned about. Instructions might be reordered for the sake of the program to run faster. Programmers can't work with that. So one proposes a memory model. Follow these rules, and our optimizations won't alter the behavior of the program. They kind-of describes what happens "in between" the critical sections of the program, hence the lack of a mutex mention. The paper presents a local property and then shows, formally, that this property is enough to guarantee an efficient memory model. That is, a model in which you can perform optimizations, while programmers can still reason about the programs behavior. The crux of the paper is that the property is local. This is new, because memory models which came before it are global: to reason about correctness, you have to consider the whole program, rather than consider a small (local) subset. OCaml requires more safety than most programming languages, so this is good for the fact that you can now compose local fragments of OCaml programs, without having to worry about a global safety property. The property is also simpler for programmers to reason about. The way you "use" the paper is that you adapt your optimizations to follow the property, and you make sure that the virtual memory model is implemented the same way on different architectures. Finally, the examples: they explore the idea of a local reasoning. In particular, they show why the (existing) global properties fail if you view them under the stronger requirement of local reasoning. It's the setup for the paper, since it means you can't just use the existing models. They need to be adapted if you want a more localized property.
- 5y ago
hajile 5y ago
StandardML implementations have had good multicore support for decades now despite having only a tiny fraction of the users and development time. Meanwhile, Ocaml has been promising support for years. What in Ocaml makes this so much harder to implement?
- rwmj 5y ago
  OCaml (or its immediate predecessor[1]) had a multicore implementation, but it was dropped because of its complexity and effect on single-thread performance. The challenge is to add it back in a way that is maintainable and doesn't negatively affect current users. [1] https://www.researchgate.net/publication/2774662_Concurrent_generational_garbage_collector_for_a_multithreaded_implementation_of_ML https://www.researchgate.net/publication/2774662_Concurrent_...
- sadiq 5y ago
  As the sibling comment mentions, the hard part is actually retrofitting multicore whilst maintaining compatibility _and_ performance. Our paper last year covers most of why this is tricky: https://arxiv.org/abs/2004.11663 https://arxiv.org/abs/2004.11663
4ad 5y ago
Algebraic effects as described in the OCaml papers about effects are dynamically typed. I recently heard in some talk that this idea was abandoned, and the new algebraic effects are in fact statically typed. Do I remember this correctly? If so, where can I read about these new algebraic effects?
- kcsrk 5y ago
  Abandoned is perhaps too strong a term :-). Effect handlers in the language are supported by fibers, lightweight stacklets managed by the runtime. The details of the implementation can be found in the upcoming research paper in PLDI'21 conference [1]. The effect handlers in Multicore OCaml today do not provide effect safety. Programs are not statically guaranteed to handle all the effects they may perform. This is only as bad as exceptions in OCaml and every other mainstream language with exceptions. We are working on developing an effect system, which will ensure effect safety i.e, the compiler ensures that all the effects performed are caught. You also get a nice inferred type that says what effects a particular function may perform; if it performs none, then it is a pure function! This implementation would still use the current fiber support in the runtime. Leo White, one of the developers of Multicore OCaml had given a talk on this new effect system a few years ago [2]. That's the best place today to learn about the new effect handlers. The plan is to first add the fiber runtime support to OCaml without the syntax extensions for effect handlers, and then introduce syntax along with the effect system. [1] https://arxiv.org/abs/2104.00250 https://arxiv.org/abs/2104.00250 [2] https://www.janestreet.com/tech-talks/effective-programming/ https://www.janestreet.com/tech-talks/effective-programming/
agumonkey 5y ago
How much resources are going into multicore compared to other needs / improvements in ocaml (if any) ?
- sadiq 5y ago
  Not sure I have enough of an overview to comment on that in general, unfortunately. The overview Anil gave at the end of last year on the OCaml Platform should give you an idea of the many other strands of work that are going on: https://ocaml.org/platform/ https://ocaml.org/platform/
  - agumonkey 5y ago
    Thanks man, and Best wishes

terminalserver 5y ago

For the lay person, what is the advantage of ocaml over other languages? Why would I reach for it?

Steltek 5y ago
Jane Street, financial trading company, uses OCaml a ton and publishes a lot of their libraries. I played with it some back when I was dabbling in every language I could get my hands on. About when Scala was new (and constantly breaking between releases) and before Rust, Go, JS v8 were around. I really liked the functional aspect, which wasn't bolted on after the fact, and that it could be compiled to a real binary, not interpreted bytecode. In my naive youth, I viewed AOT compilation as the path to a true high level language that was also fast. While it was neat, I moved on to Lisps/Schemes and now modern JS is my very happy compromise of functional and practical.
Taikonerd 5y ago
OCaml is... * functional * strongly, statically typed * garbage-collected OCaml can compile to a native binary, or to JS via ReScript (formerly BuckleScript).
- christophilus 5y ago
  Also, it compiles very quickly compared to most other statically typed languages; especially compared to more advanced ones.
- Zababa 5y ago
  > OCaml can compile to a native binary, or to JS via ReScript (formerly BuckleScript). It can also compile to JS via js_of_ocaml!
richeyryan 5y ago
OCaml is statically typed functional programming language. It's a cousin of Haskell. It has some nice things like automatic type inference so you don't have to write very many type annotations. It's not as focused on purity as Haskell so its easier to mutate state where you want to but you get a lot of the niceties of ML programming languages like pattern matching, variants, structural typing in places. It also has object-oriented features, though they aren't widely used the attitude is something like OO is there if we need it and we're definitely willing to use it in places that require it. Its pretty fast for a functional language and you could probably get pretty far with it before you'd have to consider using a real low level langauge. The disadvantages I think are pretty uncontroversial: a smaller community, not as many libraries, a bit of a fractured stdlib and build situation and until now no multicore.

toolslive 5y ago

What I'm still missing is a strategy to move existing ocaml programs that use (let's say) Lwt for concurrency and a bit of C for things that were trivial to parallelize to move to multicore ocaml AND benefit from it.

kcsrk 5y ago
(One of the Multicore OCaml devs here) We have prototyped offloading CPU intensive computations in Lwt programs using Multicore OCaml [1]. We're currently working with Lwt maintainers to upstream it. [1] https://sudha247.github.io/2020/10/01/lwt-multicore/ https://sudha247.github.io/2020/10/01/lwt-multicore/
- toolslive 5y ago
  thx! Another question. Suppose you're starting from scratch, is it worth going the effects based async io route ? https://github.com/kayceesrk/ocaml-aeio https://github.com/kayceesrk/ocaml-aeio seems very interesting...
  - kcsrk 5y ago
    Yes, exactly. The reason why monadic concurrency libraries such as Lwt and Async is that the OCaml language does not support concurrency natively. If it did, we would have built something similar to the `ocaml-aeio` library. Btw there is a modern instantiation of `ocaml-aeio` called `eieio` [1] which supports Linux's io-uring. Eventually, this will be extended to support all the modern I/O stacks on different platforms, and also support performing I/O on multiple cores. [1] https://github.com/ocaml-multicore/eioio https://github.com/ocaml-multicore/eioio
- anuragsoni 5y ago
  > We have prototyped offloading CPU intensive computations in Lwt programs using Multicore OCaml This is very interesting! I'm wondering if you are aware of any discussions with the async maintainers about what their plans are with the multicore runtime?

philzook 5y ago

It seems like great work. Something I don't understand is that I hear people mentioning a lack of multicore support as blocker to using a programming language. I don't think this is something I have ever felt. What problem domains require multicore?

blacktriangle 5y ago
I feel the same way and I can't help but feel like I am missing something. Concurrency is still inherently more complicated than serial processing no matter what language features you add, and it's amazing how far you can get just by scaling across OS processes. I know there are plenty of domains that do want multicore (ML, graphics), but it seems like even the people writing line of business apps think lack of multicore is a deal breaker for them.
- hderms 5y ago
  Yeah I really don't know because OCaml already has async constructs. It doesn't seem like it should be a dealbreaker for most. That being said, it has a reputation for being extremely performant so it might just be that people feel like it's so close to being usable in a lot of situations it currently isn't so there is more aggregate desire to allow for multicore execution
- toolslive 5y ago
  Often, it's just about finding a stick. They don't want to use OCaml and the absence of multi-core features gives them the excuse they were looking for. Now they can inform management: "OCaml is not an option. we'll stick with <insert PL here>"
  - sshine 5y ago
    It often is. But sometimes it is also about finding more reasons to use one of your favorite languages. I’m doing pseudo-async PHP at work, and having proper support would certainly make these solutions less brittle.

dang 5y ago

Past related threads: Multicore OCaml: Feb 2021 with new preprint on Effect Handlers - https://news.ycombinator.com/item?id=26424785 https://news.ycombinator.com/item?id=26424785 - March 2021 (29 comments) Multicore OCaml: October 2020 - https://news.ycombinator.com/item?id=25034538 https://news.ycombinator.com/item?id=25034538 - Nov 2020 (9 comments) Multicore OCaml: September 2020 - https://news.ycombinator.com/item?id=24719124 https://news.ycombinator.com/item?id=24719124 - Oct 2020 (43 comments) Parallel Programming in Multicore OCaml - https://news.ycombinator.com/item?id=23740869 https://news.ycombinator.com/item?id=23740869 - July 2020 (15 comments) Multicore OCaml: May 2020 update - https://news.ycombinator.com/item?id=23380370 https://news.ycombinator.com/item?id=23380370 - June 2020 (17 comments) Multicore OCaml: March 2020 update - https://news.ycombinator.com/item?id=22727975 https://news.ycombinator.com/item?id=22727975 - March 2020 (37 comments) Multicore OCaml: Feb 2020 update - https://news.ycombinator.com/item?id=22443428 https://news.ycombinator.com/item?id=22443428 - Feb 2020 (80 comments) State of Multicore OCaml [pdf] - https://news.ycombinator.com/item?id=17416797 https://news.ycombinator.com/item?id=17416797 - June 2018 (103 comments) OCaml-multicore now at 4.04.2 - https://news.ycombinator.com/item?id=16646181 https://news.ycombinator.com/item?id=16646181 - March 2018 (4 comments) A deep dive into Multicore OCaml garbage collector - https://news.ycombinator.com/item?id=14780159 https://news.ycombinator.com/item?id=14780159 - July 2017 (89 comments) Lock-free programming for the masses - https://news.ycombinator.com/item?id=11907584 https://news.ycombinator.com/item?id=11907584 - June 2016 (29 comments) Lock-free programming for the masses - https://news.ycombinator.com/item?id=11893911 https://news.ycombinator.com/item?id=11893911 - June 2016 (4 comments) OCaml 4.03 will, “if all goes well”, support multicore - https://news.ycombinator.com/item?id=9582980 https://news.ycombinator.com/item?id=9582980 - May 2015 (113 comments) Multicore OCaml - https://news.ycombinator.com/item?id=8003699 https://news.ycombinator.com/item?id=8003699 - July 2014 (1 comment)