5 ms·

I never managed to understand why so many people were apparently longering for multicore support. I just can't believe many are impatiently waiting for this fea

by rixed 9y ago

I never managed to understand why so many people were apparently longering for multicore support. I just can't believe many are impatiently waiting for this feature to start using the ocaml language for some projects requiring multicore support, and I became impatient myself just to see what those projects are. In recent years where servers are to scale to many machines I found myself using less and less kennel threads. Also it seams GUIs are mostly web theses days. Most remaining fields I can imagine where you would want kennel threads, ocaml would not fit anyway because of GC and lack of control over memory layout. So I really wonder...

girvo 9y ago
So about the only reason I have for it is some certain parallel algorithms I'd love to use, that are mostly best done at the thread level. But, these days using unix sockets with some cute flags letting multiple proceeds binding to the same listening sockets means it's sort of less needed! Though, doing a lot of work with Nim lately and exposing thread pools via Futures makes for some pretty lovely easy parallel code, so I dunno. Not a deal breaker but a nice to have, and having threads in OCaml will make things just that bit nicer
rbehrends 9y ago
The most common reasons for requiring shared memory concurrency (instead of message passing) are: 1. It's just plain easier to write a lot of stuff with a shared memory approach. 2. There are algorithms where you get the best speedup using shared memory. But yes, there are also downsides. 1. Shared memory concurrency can be a lot harder on the runtime. In particular, writing an efficient single-threaded GC is an order of magnitude easier than an efficient concurrent GC. 2. You eventually run into scaling issues with shared memory only (number of cores, limited memory bandwidth). That said, a hybrid shared memory/message passing solution can still be superior to a pure message passing one, and for "desktop" parallelism that's not an issue. 3. Shared memory concurrency requires programming language support (at least if you want to keep a modicum of sanity), whereas message passing concurrency can be done as a library. And many languages have really, really screwed up their handling of shared memory concurrency where it's extremely difficult to reason about correctness. My perceptions may be colored here because I did cut my teeth learning about parallel programming with distributed computing in the 1990s, but I also find that lack of shared memory concurrency less of an issue in actual practice (though, obviously, I'd hardly pass up on having the option).
- rixed 9y ago
  Indeed I welcome the option as well. If it doesn't make the runtime noticably slower for non concurrent programs, of course, but I'm confident the authors will make all this a no-op in that case.
  - kcsrk 9y ago
    Performance backwards compatibility has been a key goal in the Multicore OCaml project. Currently, the overhead for running legacy sequential OCaml programs on the multicore GC is a few percentage points on average. The overhead is low enough that we may not need to provide a "sequential-only" compilation flag.
- fpoling 9y ago
  Erlang essentially using that hybrid model. The runtime uses a shared heap for strings and some other immutable data that can be reference counted. On top of that each thread has own GC-heap and messages copies everything private heap. I am puzzled why other languages do not even try to explore this. This will be a nice fit for Ocaml or nodejs. And even with Go I wish for be an option to run several Go processes with a shared heap with explicit API to write/read things there and channels working across processes.
kcsrk 9y ago
Multicore support isn't for every one, but there are some important use cases where multicore support is very useful. One example is Facebook's Hack language type checker [0] which offers good opportunity for parallelism, but the lack of multicore support in OCaml means that they had to implement a custom off-heap lock free hash table shared by multiple processes. There are many industrial uses of OCaml including static type checkers (FB's Hack, Flow, Infer, etc), program verification tools (Coq, why3), math libraries (Owl) [1] etc., all of which benefit from multicore support. [0]: https://www.youtube.com/watch?v=uXuYVUdFY48 https://www.youtube.com/watch?v=uXuYVUdFY48 [1]: https://news.ycombinator.com/item?id=14751236 https://news.ycombinator.com/item?id=14751236
lobster_johnson 9y ago
In Go you can just do "go doWork()" and the scheduler runs the function as a coroutine on some kernel thread, thus allowing your program to make use of all cores. The ability to just throw goroutines at the scheduler changes how you approach writing programs. For example, say you want to process an input that consists of lots of individual records. You can run the input stream in one goroutine, then spawn a bunch of worker goroutines that each receives records on a channel for processing. Since Go uses real threads and doesn't have a global lock, you can get nearly linear improvement in throughout here. You wouldn't do this in, say, Ruby or Python, where the concurrency situation is about the same as in OCaml. Ruby has threads, like OCaml, but their slowness means they're not really usable in the same way as goroutines. Parallelizing apps by forking child processes and communicating input and results via pipes is something I've done a lot in Ruby, and it's a really awkward, heavy-handed concurrency model.
- rbehrends 9y ago
  Go doesn't really have a good shared memory story, either. Shared memory support exists, but it's not really any safer than in C or Java. If that were good enough, then Ocamlnet's netmulticore library would already fit the bill. But most people who want shared memory support want something that's both safer and higher level. Go really is designed around message-passing as its primary means of coordination; the key part here is having nice abstractions, which is where Go's strengths lie and which revolve primarily around `select`. But writing higher-level abstractions over processes and FIFO channels isn't terribly hard, as long as you have serialization built in (which Python, Ruby, and OCaml all do).
- ice109 9y ago
  I whole heartedly agree with you on the convenience of go routines. I haven't been following the ocaml concurrency sorry; are they implementing a similar concurrency model?
  - kcsrk 9y ago
    Multicore OCaml comes with native support for concurrency through algebraic effect handlers which generalize common control flow abstractions such as exceptions, async/await, generators, non-determinism, backtracking. All of these mechanisms can be implemented directly in OCaml. [0] introduces the model, [1,2] has examples, and for further reading see [3]. [0]: http://kcsrk.info/ocaml/multicore/2015/05/20/effects-multicore/ http://kcsrk.info/ocaml/multicore/2015/05/20/effects-multico... [1]: http://kcsrk.info/ocaml/multicore/effects/2015/05/27/more-effects/ http://kcsrk.info/ocaml/multicore/effects/2015/05/27/more-ef... [2]: https://github.com/kayceesrk/effects-examples https://github.com/kayceesrk/effects-examples [3]: See recent pubs here http://kcsrk.info/ http://kcsrk.info/.
logicchains 9y ago
Even if a particular application doesn't require any multicore support, usually at work I won't be working on just one application as part of a project, there'll be multiple applications involved, at least some of which will require shared memory parallelism. To me it doesn't make sense to pick a language that can only be used for a few applications not all, as this leads to unnecessary duplication in library code and the like compared to just using one language (in my case, C++) for all of them (similar to how people prefer to use the same language for frontend and backend to reduce duplication). Especially when the lack of multicore doesn't bring any compelling advantages: it saves a few percentage points of single-threaded performance at the expense of completely ruling out most use-cases that require shared memory parallelism (or at least rules out any ways of doing them that aren't incredibly un-ergonomic), which seems like an absolutely terrible trade-off to me. Here's a concrete example: comparing the Isabelle proof assistant, written in PolyML (a multi-core supporting SML implementation developed mostly by one guy) with Coq (which is written in OCaml). Interactive theorem proving in the former is a lot nicer as it takes advantage of multithreading not only for faster concurrent processing of proofs, but also to do things like running Quickcheck and Nitpick in a separate thread automatically to identify trivially falsifiable lemmas, and automatically finding stdlib lemmas that exactly solve a particular proof. I think PolyML demonstrates that a lack of manpower isn't what stopped OCaml implementing multithreading support. Over the years there have actually been a few proposals/branches implementing some form of support, but all were rejected/abandoned. There was even one that just made the runtime reentrant (passing the runtime around directly instead of having it as a global variable), meaning the OCaml runtime could now be stopped and restarted when embedded in another application (e.g. C calling OCaml), a feature already present in Haskell, but this relatively simple improvement was also rejected (which personally bothers me a lot on a subjective level as I hate globals so it seems like a worthy improvement for its own sake). I remember waiting excitedly four or five years ago for it to be merged, a small but significant step on the path to multicore support, only for that hope to fade away as contributions to the branch slowed to a trickle then dried up completely. Without such extreme focus on avoiding decreases in single-core performance, it would have been much easier for a change like this to have been merged. I think in recent times there is even less justification for rejecting multicore for affecting single-threaded performance. For two reasons: firstly, the recent addition of FLambda has brought performance improvements in many cases of over 10%, easily enough to compensate for any loss from multi-threading. Secondly, HFT now basically requires FPGA to compete, so single-threaded performance would presumably be of less concern to Jane Street (OCaml's biggest industrial user) now as they really shouldn't be using a software execution engine anyway (disclaimer: I say that as someone working at a competing HFT firm). And if Jane Street isn't doing HFT, then a drop of a few percentage points in single-threaded performance shouldn't affect them much anyway. Finally, OCaml could always do what Haskell did and add a flag to toggle whether or not multi-threading is enabled, allowing single-threaded users to avoid any performance regression.
_ph_ 9y ago
The short answer is: single-core cpu performance is not going up much in the recent years. The core count is still increasing every year. That is why graphics cards still see a strong rise in total performance, their usual work load is inherently parallel. For typical server tasks, like web serving, we can just run several processes. Unix is doing this for decades. But even web serving is often cpu bound, if your SQL query runs only on one cpu. The obvious thing to make programs faster is, to utilize as many cpus as possible to perform the computations. There is no silver bullet, but the recent years brought quite some progress. Go makes it easy and cheap to spawn off goroutines, the channels and GC help to make this a reasonable concept. Functional programming languages are even a more obvious candidate for parallel execution, as they have no side effects.
- pencilhappen 9y ago
  > but the recent years brought quite some progress. Go makes it easy ... It actually feels like the recent years have been pretty disappointing. Go is one of the few places where the language developers are making a real effort around multicore. Erlang came with a bunch of opinions about how to do this in the 90s. But every other mainstream language is taking a head-in-the-sand attitude about this. Python squandered their backwards compatibility break without addressing this at all. The JVM developers are making no effort to get away from the global heap. Ruby and Node are completely ignoring it entirely. PHP from it's request/response web history actually does well, except its non-web story is as dismal as ever. Lua has coroutines, which are a reasonable abstraction, but no multicore VM story...
  - fpoling 9y ago
    Given that Go is memory-unsafe on multicore, I do not think that language developers are made that strong efforts. The answer in Go is to use runtime checkers, but even those do not catch all memory safety issues.

by rixed 9y ago

girvo 9y ago

So about the only reason I have for it is some certain parallel algorithms I'd love to use, that are mostly best done at the thread level. But, these days using unix sockets with some cute flags letting multiple proceeds binding to the same listening sockets means it's sort of less needed! Though, doing a lot of work with Nim lately and exposing thread pools via Futures makes for some pretty lovely easy parallel code, so I dunno. Not a deal breaker but a nice to have, and having threads in OCaml will make things just that bit nicer

rbehrends 9y ago

The most common reasons for requiring shared memory concurrency (instead of message passing) are: 1. It's just plain easier to write a lot of stuff with a shared memory approach. 2. There are algorithms where you get the best speedup using shared memory. But yes, there are also downsides. 1. Shared memory concurrency can be a lot harder on the runtime. In particular, writing an efficient single-threaded GC is an order of magnitude easier than an efficient concurrent GC. 2. You eventually run into scaling issues with shared memory only (number of cores, limited memory bandwidth). That said, a hybrid shared memory/message passing solution can still be superior to a pure message passing one, and for "desktop" parallelism that's not an issue. 3. Shared memory concurrency requires programming language support (at least if you want to keep a modicum of sanity), whereas message passing concurrency can be done as a library. And many languages have really, really screwed up their handling of shared memory concurrency where it's extremely difficult to reason about correctness. My perceptions may be colored here because I did cut my teeth learning about parallel programming with distributed computing in the 1990s, but I also find that lack of shared memory concurrency less of an issue in actual practice (though, obviously, I'd hardly pass up on having the option).

rixed 9y ago
Indeed I welcome the option as well. If it doesn't make the runtime noticably slower for non concurrent programs, of course, but I'm confident the authors will make all this a no-op in that case.
- kcsrk 9y ago
  Performance backwards compatibility has been a key goal in the Multicore OCaml project. Currently, the overhead for running legacy sequential OCaml programs on the multicore GC is a few percentage points on average. The overhead is low enough that we may not need to provide a "sequential-only" compilation flag.
fpoling 9y ago
Erlang essentially using that hybrid model. The runtime uses a shared heap for strings and some other immutable data that can be reference counted. On top of that each thread has own GC-heap and messages copies everything private heap. I am puzzled why other languages do not even try to explore this. This will be a nice fit for Ocaml or nodejs. And even with Go I wish for be an option to run several Go processes with a shared heap with explicit API to write/read things there and channels working across processes.

kcsrk 9y ago

Multicore support isn't for every one, but there are some important use cases where multicore support is very useful. One example is Facebook's Hack language type checker [0] which offers good opportunity for parallelism, but the lack of multicore support in OCaml means that they had to implement a custom off-heap lock free hash table shared by multiple processes. There are many industrial uses of OCaml including static type checkers (FB's Hack, Flow, Infer, etc), program verification tools (Coq, why3), math libraries (Owl) [1] etc., all of which benefit from multicore support. [0]: https://www.youtube.com/watch?v=uXuYVUdFY48 https://www.youtube.com/watch?v=uXuYVUdFY48 [1]: https://news.ycombinator.com/item?id=14751236 https://news.ycombinator.com/item?id=14751236

lobster_johnson 9y ago

In Go you can just do "go doWork()" and the scheduler runs the function as a coroutine on some kernel thread, thus allowing your program to make use of all cores. The ability to just throw goroutines at the scheduler changes how you approach writing programs. For example, say you want to process an input that consists of lots of individual records. You can run the input stream in one goroutine, then spawn a bunch of worker goroutines that each receives records on a channel for processing. Since Go uses real threads and doesn't have a global lock, you can get nearly linear improvement in throughout here. You wouldn't do this in, say, Ruby or Python, where the concurrency situation is about the same as in OCaml. Ruby has threads, like OCaml, but their slowness means they're not really usable in the same way as goroutines. Parallelizing apps by forking child processes and communicating input and results via pipes is something I've done a lot in Ruby, and it's a really awkward, heavy-handed concurrency model.

rbehrends 9y ago
Go doesn't really have a good shared memory story, either. Shared memory support exists, but it's not really any safer than in C or Java. If that were good enough, then Ocamlnet's netmulticore library would already fit the bill. But most people who want shared memory support want something that's both safer and higher level. Go really is designed around message-passing as its primary means of coordination; the key part here is having nice abstractions, which is where Go's strengths lie and which revolve primarily around `select`. But writing higher-level abstractions over processes and FIFO channels isn't terribly hard, as long as you have serialization built in (which Python, Ruby, and OCaml all do).
ice109 9y ago
I whole heartedly agree with you on the convenience of go routines. I haven't been following the ocaml concurrency sorry; are they implementing a similar concurrency model?
- kcsrk 9y ago
  Multicore OCaml comes with native support for concurrency through algebraic effect handlers which generalize common control flow abstractions such as exceptions, async/await, generators, non-determinism, backtracking. All of these mechanisms can be implemented directly in OCaml. [0] introduces the model, [1,2] has examples, and for further reading see [3]. [0]: http://kcsrk.info/ocaml/multicore/2015/05/20/effects-multicore/ http://kcsrk.info/ocaml/multicore/2015/05/20/effects-multico... [1]: http://kcsrk.info/ocaml/multicore/effects/2015/05/27/more-effects/ http://kcsrk.info/ocaml/multicore/effects/2015/05/27/more-ef... [2]: https://github.com/kayceesrk/effects-examples https://github.com/kayceesrk/effects-examples [3]: See recent pubs here http://kcsrk.info/ http://kcsrk.info/.

logicchains 9y ago

Even if a particular application doesn't require any multicore support, usually at work I won't be working on just one application as part of a project, there'll be multiple applications involved, at least some of which will require shared memory parallelism. To me it doesn't make sense to pick a language that can only be used for a few applications not all, as this leads to unnecessary duplication in library code and the like compared to just using one language (in my case, C++) for all of them (similar to how people prefer to use the same language for frontend and backend to reduce duplication). Especially when the lack of multicore doesn't bring any compelling advantages: it saves a few percentage points of single-threaded performance at the expense of completely ruling out most use-cases that require shared memory parallelism (or at least rules out any ways of doing them that aren't incredibly un-ergonomic), which seems like an absolutely terrible trade-off to me. Here's a concrete example: comparing the Isabelle proof assistant, written in PolyML (a multi-core supporting SML implementation developed mostly by one guy) with Coq (which is written in OCaml). Interactive theorem proving in the former is a lot nicer as it takes advantage of multithreading not only for faster concurrent processing of proofs, but also to do things like running Quickcheck and Nitpick in a separate thread automatically to identify trivially falsifiable lemmas, and automatically finding stdlib lemmas that exactly solve a particular proof. I think PolyML demonstrates that a lack of manpower isn't what stopped OCaml implementing multithreading support. Over the years there have actually been a few proposals/branches implementing some form of support, but all were rejected/abandoned. There was even one that just made the runtime reentrant (passing the runtime around directly instead of having it as a global variable), meaning the OCaml runtime could now be stopped and restarted when embedded in another application (e.g. C calling OCaml), a feature already present in Haskell, but this relatively simple improvement was also rejected (which personally bothers me a lot on a subjective level as I hate globals so it seems like a worthy improvement for its own sake). I remember waiting excitedly four or five years ago for it to be merged, a small but significant step on the path to multicore support, only for that hope to fade away as contributions to the branch slowed to a trickle then dried up completely. Without such extreme focus on avoiding decreases in single-core performance, it would have been much easier for a change like this to have been merged. I think in recent times there is even less justification for rejecting multicore for affecting single-threaded performance. For two reasons: firstly, the recent addition of FLambda has brought performance improvements in many cases of over 10%, easily enough to compensate for any loss from multi-threading. Secondly, HFT now basically requires FPGA to compete, so single-threaded performance would presumably be of less concern to Jane Street (OCaml's biggest industrial user) now as they really shouldn't be using a software execution engine anyway (disclaimer: I say that as someone working at a competing HFT firm). And if Jane Street isn't doing HFT, then a drop of a few percentage points in single-threaded performance shouldn't affect them much anyway. Finally, OCaml could always do what Haskell did and add a flag to toggle whether or not multi-threading is enabled, allowing single-threaded users to avoid any performance regression.

_ph_ 9y ago

The short answer is: single-core cpu performance is not going up much in the recent years. The core count is still increasing every year. That is why graphics cards still see a strong rise in total performance, their usual work load is inherently parallel. For typical server tasks, like web serving, we can just run several processes. Unix is doing this for decades. But even web serving is often cpu bound, if your SQL query runs only on one cpu. The obvious thing to make programs faster is, to utilize as many cpus as possible to perform the computations. There is no silver bullet, but the recent years brought quite some progress. Go makes it easy and cheap to spawn off goroutines, the channels and GC help to make this a reasonable concept. Functional programming languages are even a more obvious candidate for parallel execution, as they have no side effects.

pencilhappen 9y ago
> but the recent years brought quite some progress. Go makes it easy ... It actually feels like the recent years have been pretty disappointing. Go is one of the few places where the language developers are making a real effort around multicore. Erlang came with a bunch of opinions about how to do this in the 90s. But every other mainstream language is taking a head-in-the-sand attitude about this. Python squandered their backwards compatibility break without addressing this at all. The JVM developers are making no effort to get away from the global heap. Ruby and Node are completely ignoring it entirely. PHP from it's request/response web history actually does well, except its non-web story is as dismal as ever. Lua has coroutines, which are a reasonable abstraction, but no multicore VM story...
- fpoling 9y ago
  Given that Go is memory-unsafe on multicore, I do not think that language developers are made that strong efforts. The answer in Go is to use runtime checkers, but even those do not catch all memory safety issues.