6 ms·

It seems like great work. Something I don't understand is that I hear people mentioning a lack of multicore support as blocker to using a programming language.

It seems like great work. Something I don't understand is that I hear people mentioning a lack of multicore support as blocker to using a programming language. I don't think this is something I have ever felt. What problem domains require multicore?

blacktriangle 5y ago
I feel the same way and I can't help but feel like I am missing something. Concurrency is still inherently more complicated than serial processing no matter what language features you add, and it's amazing how far you can get just by scaling across OS processes. I know there are plenty of domains that do want multicore (ML, graphics), but it seems like even the people writing line of business apps think lack of multicore is a deal breaker for them.
- hderms 5y ago
  Yeah I really don't know because OCaml already has async constructs. It doesn't seem like it should be a dealbreaker for most. That being said, it has a reputation for being extremely performant so it might just be that people feel like it's so close to being usable in a lot of situations it currently isn't so there is more aggregate desire to allow for multicore execution
- toolslive 5y ago
  Often, it's just about finding a stick. They don't want to use OCaml and the absence of multi-core features gives them the excuse they were looking for. Now they can inform management: "OCaml is not an option. we'll stick with <insert PL here>"
  - sshine 5y ago
    It often is. But sometimes it is also about finding more reasons to use one of your favorite languages. I’m doing pseudo-async PHP at work, and having proper support would certainly make these solutions less brittle.
    - deleted 5y ago
      [deleted]
  - Buttons840 5y ago
    I learned Haskell instead of OCaml in part because of this. Yeah, it probably wasn't that important, it was an emotional decision, and it wasn't "fair" to OCaml. But whatever it was or wasn't, I learned Haskell instead of OCaml. Also, if I read between the lines correctly, in your scenerio it's management who is interested in OCaml and the developers want to use something else? Wonder if that has ever actually happened?
    - toolslive 5y ago
      management was open to "exploring alternatives" whenever a component needed to be redone.
- gwmnxnp_516a 5y ago
  Forking new processes is less efficient than spawning new threads as processes uses more machine resources and more memory. In addition, communication between threads is far more simple than communication between processes. Multicore may not be deal-breaker for server applications, but it matters for applications that require parallel processing.
  - gmfawcett 5y ago
    Forking a process on a modern Linux is relatively cheap these days, and for many jobs you will just prefork all the workers anyway. Passing messages via thread mailboxes or IPC pipes are about equally complex endeavours. If threads had never been invented, I suspect that 90% of modern multicore programs would have done just fine on a mixture of multi-processing and asynchronous I/O. (The other 10% would have done poorly, though.)
  - blacktriangle 5y ago
    Less efficient for the machine, making the program deal with async is less efficient for the programmer. I'm just confused when I see people working on CRUD apps ruling out tools because they lack first-class async support.
xfer 5y ago
Anytime you need some parallelism like processing chunks of data independently? Using processes to do it is not very ergonomic.
- gmfawcett 5y ago
  It depends? It's easy in Ocaml to fork child processes, set up pipes, and pass messages back and forth. There are libraries that paper over the details (or you can roll your own, as I suspect many of us have done over the years). That approach is not suitable for every workload, but fine for most map/reduce applications.
Buttons840 5y ago
When people buy a shiny new processor with 8 or more cores, I think it's very natural to want to know you can utilize them.
- slaymaker1907 5y ago
  I think the question is more about what what use cases require multithreading rather than just using multiple processes.
  - sshine 5y ago
    In functional programming, simply mapping over data lends itself to trivial parallelism. But yes, the use cases that require it are much more interesting. Basically, background processing.
    - pjmlp 5y ago
      Given that UNIX daemons are mostly processes, how is multhreading relevant for backgrond processing?
sweeneyrod 5y ago
Also, if no multicore blocks you from using OCaml, you also have to stay away from Python/Ruby/JavaScript which also lack it.
toolslive 5y ago
For us, it was IO actually: doing IO concurrently (Lwt) and letting the one core schedule the IO worked quite well for a long time, but then we started targetting NVMe devices... Now just the scheduling of the IO would fill up the core without coming even close saturating the device. We ended up splitting the device and using multiple processes but that was suboptimal.
simiones 5y ago
Well, any interactive program benefits immensely from multi-threading - at the very least, you need 1 UI thread and 1 background thread. This can be achieved with asynchronous workflows, but it's siginifcantly more complicated compared to spawning a thread to address a user action. Similarly, wanting to spin a thread to do some background work or periodically check some property is an extremely common pattern, that has nothing to do with performance, and where spawning a new process is significantly more overhead, both conceptually and in terms of used resources.

blacktriangle 5y ago

I feel the same way and I can't help but feel like I am missing something. Concurrency is still inherently more complicated than serial processing no matter what language features you add, and it's amazing how far you can get just by scaling across OS processes. I know there are plenty of domains that do want multicore (ML, graphics), but it seems like even the people writing line of business apps think lack of multicore is a deal breaker for them.

hderms 5y ago
Yeah I really don't know because OCaml already has async constructs. It doesn't seem like it should be a dealbreaker for most. That being said, it has a reputation for being extremely performant so it might just be that people feel like it's so close to being usable in a lot of situations it currently isn't so there is more aggregate desire to allow for multicore execution
toolslive 5y ago
Often, it's just about finding a stick. They don't want to use OCaml and the absence of multi-core features gives them the excuse they were looking for. Now they can inform management: "OCaml is not an option. we'll stick with <insert PL here>"
- sshine 5y ago
  It often is. But sometimes it is also about finding more reasons to use one of your favorite languages. I’m doing pseudo-async PHP at work, and having proper support would certainly make these solutions less brittle.
  - deleted 5y ago
    [deleted]
- Buttons840 5y ago
  I learned Haskell instead of OCaml in part because of this. Yeah, it probably wasn't that important, it was an emotional decision, and it wasn't "fair" to OCaml. But whatever it was or wasn't, I learned Haskell instead of OCaml. Also, if I read between the lines correctly, in your scenerio it's management who is interested in OCaml and the developers want to use something else? Wonder if that has ever actually happened?
  - toolslive 5y ago
    management was open to "exploring alternatives" whenever a component needed to be redone.
gwmnxnp_516a 5y ago
Forking new processes is less efficient than spawning new threads as processes uses more machine resources and more memory. In addition, communication between threads is far more simple than communication between processes. Multicore may not be deal-breaker for server applications, but it matters for applications that require parallel processing.
- gmfawcett 5y ago
  Forking a process on a modern Linux is relatively cheap these days, and for many jobs you will just prefork all the workers anyway. Passing messages via thread mailboxes or IPC pipes are about equally complex endeavours. If threads had never been invented, I suspect that 90% of modern multicore programs would have done just fine on a mixture of multi-processing and asynchronous I/O. (The other 10% would have done poorly, though.)
- blacktriangle 5y ago
  Less efficient for the machine, making the program deal with async is less efficient for the programmer. I'm just confused when I see people working on CRUD apps ruling out tools because they lack first-class async support.

xfer 5y ago

Anytime you need some parallelism like processing chunks of data independently? Using processes to do it is not very ergonomic.

gmfawcett 5y ago
It depends? It's easy in Ocaml to fork child processes, set up pipes, and pass messages back and forth. There are libraries that paper over the details (or you can roll your own, as I suspect many of us have done over the years). That approach is not suitable for every workload, but fine for most map/reduce applications.

Buttons840 5y ago

When people buy a shiny new processor with 8 or more cores, I think it's very natural to want to know you can utilize them.

slaymaker1907 5y ago
I think the question is more about what what use cases require multithreading rather than just using multiple processes.
- sshine 5y ago
  In functional programming, simply mapping over data lends itself to trivial parallelism. But yes, the use cases that require it are much more interesting. Basically, background processing.
  - pjmlp 5y ago
    Given that UNIX daemons are mostly processes, how is multhreading relevant for backgrond processing?

sweeneyrod 5y ago

Also, if no multicore blocks you from using OCaml, you also have to stay away from Python/Ruby/JavaScript which also lack it.

toolslive 5y ago

For us, it was IO actually: doing IO concurrently (Lwt) and letting the one core schedule the IO worked quite well for a long time, but then we started targetting NVMe devices... Now just the scheduling of the IO would fill up the core without coming even close saturating the device. We ended up splitting the device and using multiple processes but that was suboptimal.

simiones 5y ago

Well, any interactive program benefits immensely from multi-threading - at the very least, you need 1 UI thread and 1 background thread. This can be achieved with asynchronous workflows, but it's siginifcantly more complicated compared to spawning a thread to address a user action. Similarly, wanting to spin a thread to do some background work or periodically check some property is an extremely common pattern, that has nothing to do with performance, and where spawning a new process is significantly more overhead, both conceptually and in terms of used resources.