7 ms·

Adapting the OCaml Ecosystem for Multicore OCaml

115 points by rbjorklin 5y ago

dwohnitmok 5y ago
What's the OCaml community's perception on when multicore will fully land in trunk?
- yawaramin 5y ago
  A few myears. Maybe a few yonths.
  - deleted 5y ago
    [deleted]
  - scns 5y ago
    Metayears?
- rbjorklin 5y ago
  My understanding is that 4.14 is scheduled for Q1 2022 and 5.0 with multicore _might_ be released roughly in parallel.
  - yawaramin 5y ago
    Heh.
  - amelius 5y ago
    Are there any benchmarks? Comparisons to other multicore runtimes, like GoLang's?
    - Zababa 5y ago
      I remember seeing a graph where http/af, a http server, was close to Go net/http. I'll try to run https://github.com/ocaml-multicore/retro-httpaf-bench https://github.com/ocaml-multicore/retro-httpaf-bench and report back. I also found another graph, but I'm too colorblind to read it properly https://user-images.githubusercontent.com/554131/131155247-f4604ccc-8f14-4846-9e56-a021467c38ec.png https://user-images.githubusercontent.com/554131/131155247-f.... Edit: the benchmark fails to build
      - erk__ 5y ago
        Yeah the colors are really not great for color blind people, I rather have some symbols or similar on them
        Zababa 5y ago
        That's one of the small things I like about more complex charts (more complex than a picture): I can usually hover over the chart to see the label.
      - amelius 5y ago
        Top two lines are httpaf_eio and rust_hyper. Next two lines are httpaf_effects and httpaf_lwt. Bottom two lines are nethttp_go and cohttp_lwt_unix. All in order from top to bottom.
        Zababa 5y ago
        Thank you very much!
    - yawaramin 5y ago
      Yes, check out https://watch.ocaml.org/videos/watch/playlist/7a4ad26a-b8c5-4588-bf2a-4b981fed87f2?playlistPosition=12&resume=true https://watch.ocaml.org/videos/watch/playlist/7a4ad26a-b8c5-... for some interesting comparisons.
    - kcsrk 5y ago
      We don't compare against Go pervasively. Benchmarking across languages is hard generally, but here is a result on a specific benchmark comparing several versions of OCaml benchmarks against Go and Rust on a Http server benchmark: https://github.com/ocaml-multicore/retro-httpaf-bench/pull/15 https://github.com/ocaml-multicore/retro-httpaf-bench/pull/1.... If there are suggestions to make the Go and Rust versions, please feel free to tell us how in the issue tracker.
    - sadiq 5y ago
      Our paper from ICFP last year has extensive benchmarks against stock OCaml: https://arxiv.org/abs/2004.11663 https://arxiv.org/abs/2004.11663 In short there should be very little performance difference against stock. As KC has linked in a sibling comment there's a project going to explore using effects to write high performance IO libraries and that has some early benchmarks.
Zababa 5y ago
A nice detail: this video is hosted on the new OCaml website (https://v3.ocaml.org/en https://v3.ocaml.org/en), which includes its own peertube instance.
- maddyboo 5y ago
  Also interesting: the new OCaml website is built with ReScript, the rebranding of ReasonML.
  - Zababa 5y ago
    That's right. Here's the forum post presenting the new websites and the choices made: https://discuss.ocaml.org/t/v3-ocaml-org-a-roadmap-for-ocamls-online-presence/8368 https://discuss.ocaml.org/t/v3-ocaml-org-a-roadmap-for-ocaml....
dmitriid 5y ago
> Adapting the OCaml Ecosystem for Multicore That's the thing that is missing in discussions around many languages. You can't just add multicore or actors to a language post-factum. Because by the time you get around to doing it everything in your language, from standard lib to third-party packages, has already been written with no concept of multi-anything. And still, time and again, languages are being designed with these concerns as an afterthought.
- rkangel 5y ago
  > And still, time and again, languages are being designed with these concerns as an afterthought. Is that still happening? My impression is that any languages designed since multicore became common have thought very hard about their approach. The problem we have is that a lot of our languages have been around a while. Even Python is 30 years old at this point! Erlang here is the miracle to me. Its programming model was designed for concurrency on a single core, but when SMP processers appeared all it needed was a VM change that was completely transparent to the programs above it and suddenly all Erlang programs were extremely parallel!
  - 4ad 5y ago
    Erlang was not designed for concurrency on a single core, it was designed for parallelism on multiple machines. Because of this, erlang processes do not share memory and SMP-level parallelism was easy to add.
  - melony 5y ago
    Many languages side step the problem. With JS-inspired languages like Dart, you are limited to message passing workers. Many high performance concurrency applications are difficult without shared memory support.
niralnetworks 5y ago
Control your fleet and the staff in real-time with the best fleet management solutions from Trakiga, managing School Bus, Cabs and trucks https://www.trakiga.com/fleet-management/ https://www.trakiga.com/fleet-management/

115 points by rbjorklin 5y ago

dwohnitmok 5y ago

What's the OCaml community's perception on when multicore will fully land in trunk?

yawaramin 5y ago
A few myears. Maybe a few yonths.
- deleted 5y ago
  [deleted]
- scns 5y ago
  Metayears?
rbjorklin 5y ago
My understanding is that 4.14 is scheduled for Q1 2022 and 5.0 with multicore _might_ be released roughly in parallel.
- yawaramin 5y ago
  Heh.
- amelius 5y ago
  Are there any benchmarks? Comparisons to other multicore runtimes, like GoLang's?
  - Zababa 5y ago
    I remember seeing a graph where http/af, a http server, was close to Go net/http. I'll try to run https://github.com/ocaml-multicore/retro-httpaf-bench https://github.com/ocaml-multicore/retro-httpaf-bench and report back. I also found another graph, but I'm too colorblind to read it properly https://user-images.githubusercontent.com/554131/131155247-f4604ccc-8f14-4846-9e56-a021467c38ec.png https://user-images.githubusercontent.com/554131/131155247-f.... Edit: the benchmark fails to build
    - erk__ 5y ago
      Yeah the colors are really not great for color blind people, I rather have some symbols or similar on them
      - Zababa 5y ago
        That's one of the small things I like about more complex charts (more complex than a picture): I can usually hover over the chart to see the label.
    - amelius 5y ago
      Top two lines are httpaf_eio and rust_hyper. Next two lines are httpaf_effects and httpaf_lwt. Bottom two lines are nethttp_go and cohttp_lwt_unix. All in order from top to bottom.
      - Zababa 5y ago
        Thank you very much!
  - yawaramin 5y ago
    Yes, check out https://watch.ocaml.org/videos/watch/playlist/7a4ad26a-b8c5-4588-bf2a-4b981fed87f2?playlistPosition=12&resume=true https://watch.ocaml.org/videos/watch/playlist/7a4ad26a-b8c5-... for some interesting comparisons.
  - kcsrk 5y ago
    We don't compare against Go pervasively. Benchmarking across languages is hard generally, but here is a result on a specific benchmark comparing several versions of OCaml benchmarks against Go and Rust on a Http server benchmark: https://github.com/ocaml-multicore/retro-httpaf-bench/pull/15 https://github.com/ocaml-multicore/retro-httpaf-bench/pull/1.... If there are suggestions to make the Go and Rust versions, please feel free to tell us how in the issue tracker.
  - sadiq 5y ago
    Our paper from ICFP last year has extensive benchmarks against stock OCaml: https://arxiv.org/abs/2004.11663 https://arxiv.org/abs/2004.11663 In short there should be very little performance difference against stock. As KC has linked in a sibling comment there's a project going to explore using effects to write high performance IO libraries and that has some early benchmarks.

Zababa 5y ago

A nice detail: this video is hosted on the new OCaml website (https://v3.ocaml.org/en https://v3.ocaml.org/en), which includes its own peertube instance.

maddyboo 5y ago
Also interesting: the new OCaml website is built with ReScript, the rebranding of ReasonML.
- Zababa 5y ago
  That's right. Here's the forum post presenting the new websites and the choices made: https://discuss.ocaml.org/t/v3-ocaml-org-a-roadmap-for-ocamls-online-presence/8368 https://discuss.ocaml.org/t/v3-ocaml-org-a-roadmap-for-ocaml....

dmitriid 5y ago

> Adapting the OCaml Ecosystem for Multicore That's the thing that is missing in discussions around many languages. You can't just add multicore or actors to a language post-factum. Because by the time you get around to doing it everything in your language, from standard lib to third-party packages, has already been written with no concept of multi-anything. And still, time and again, languages are being designed with these concerns as an afterthought.

rkangel 5y ago
> And still, time and again, languages are being designed with these concerns as an afterthought. Is that still happening? My impression is that any languages designed since multicore became common have thought very hard about their approach. The problem we have is that a lot of our languages have been around a while. Even Python is 30 years old at this point! Erlang here is the miracle to me. Its programming model was designed for concurrency on a single core, but when SMP processers appeared all it needed was a VM change that was completely transparent to the programs above it and suddenly all Erlang programs were extremely parallel!
- 4ad 5y ago
  Erlang was not designed for concurrency on a single core, it was designed for parallelism on multiple machines. Because of this, erlang processes do not share memory and SMP-level parallelism was easy to add.
- melony 5y ago
  Many languages side step the problem. With JS-inspired languages like Dart, you are limited to message passing workers. Many high performance concurrency applications are difficult without shared memory support.

niralnetworks 5y ago

Control your fleet and the staff in real-time with the best fleet management solutions from Trakiga, managing School Bus, Cabs and trucks https://www.trakiga.com/fleet-management/ https://www.trakiga.com/fleet-management/