3 ms·

Does it support light-weight threads (co-routines) channels yet? (and if it does, does it multiplex them on mulitple CPUs?). I had a look a few years ago, and t

Does it support light-weight threads (co-routines) channels yet? (and if it does, does it multiplex them on mulitple CPUs?). I had a look a few years ago, and then it did not. I think this is so badly needed to get an easy route to pipeline paralellism, which is simply everywhere in today's data analysis "pipelines".

ChrisRackauckas 8y ago
Yes it does: https://docs.julialang.org/en/latest/base/multi-threading/ https://docs.julialang.org/en/latest/base/multi-threading/ https://docs.julialang.org/en/latest/manual/parallel-computing/#Multi-Threading-(Experimental)-1 https://docs.julialang.org/en/latest/manual/parallel-computi... It's listed as experimental since in a 1.x released it's planned to be changed to work on top of the task interface.
- samuell 8y ago
  Thanks. Yea, I also found this statement: > [...] it may change for future Julia versions, as it is intended to make it possible to run up to N Tasks on M Process, aka M:N Threading M:N threading is (I think) the same as the "multiplexing" I mentioned. Have seen it called "M:N multiplexing" before. (At the very end of https://docs.julialang.org/en/latest/manual/parallel-computing/#Multi-Core-or-Distributed-Processing-1 https://docs.julialang.org/en/latest/manual/parallel-computi...)
  - StefanKarpinski 8y ago
    Just to clarify: * Julia has had tasks/co-routines basically forever and uses them for all blocking operations so that no explicit non-blocking I/O or callbacks are required. * It also supports multithreading using the @threads macro. * However, it does not yet map tasks to threads, but that is very close to ready: https://github.com/JuliaLang/julia/pull/22631 https://github.com/JuliaLang/julia/pull/22631. We expect this work to be finished in a near-future 1.x release and then Julia will support M:N multiplexing.
    - samuell 8y ago
      Many thanks for the clarification!
myrryr 8y ago
Not yet, soon. It's multi threading model is mostly multi process. Mostly because people write code to run on clusters on machines, not just single ones. co-routines, channels, etc will be nice, and I expect not too far away.
deleted 8y ago
[deleted]
srean 8y ago
Thank you for asking the question I was about to ask. I am ambivalent over whether it has multiplexing or not. When I need parallelism I fall back to OS threads if available or processes. One or the other is usually always a first class citizen. Whats not that common, are real lightweight coroutines.... I want my laptop to be able to simulate the transport layer over a sizeable portion of the internet, while I watch youtube.

ChrisRackauckas 8y ago

Yes it does: https://docs.julialang.org/en/latest/base/multi-threading/ https://docs.julialang.org/en/latest/base/multi-threading/ https://docs.julialang.org/en/latest/manual/parallel-computing/#Multi-Threading-(Experimental)-1 https://docs.julialang.org/en/latest/manual/parallel-computi... It's listed as experimental since in a 1.x released it's planned to be changed to work on top of the task interface.

samuell 8y ago
Thanks. Yea, I also found this statement: > [...] it may change for future Julia versions, as it is intended to make it possible to run up to N Tasks on M Process, aka M:N Threading M:N threading is (I think) the same as the "multiplexing" I mentioned. Have seen it called "M:N multiplexing" before. (At the very end of https://docs.julialang.org/en/latest/manual/parallel-computing/#Multi-Core-or-Distributed-Processing-1 https://docs.julialang.org/en/latest/manual/parallel-computi...)
- StefanKarpinski 8y ago
  Just to clarify: * Julia has had tasks/co-routines basically forever and uses them for all blocking operations so that no explicit non-blocking I/O or callbacks are required. * It also supports multithreading using the @threads macro. * However, it does not yet map tasks to threads, but that is very close to ready: https://github.com/JuliaLang/julia/pull/22631 https://github.com/JuliaLang/julia/pull/22631. We expect this work to be finished in a near-future 1.x release and then Julia will support M:N multiplexing.
  - samuell 8y ago
    Many thanks for the clarification!

myrryr 8y ago

Not yet, soon. It's multi threading model is mostly multi process. Mostly because people write code to run on clusters on machines, not just single ones. co-routines, channels, etc will be nice, and I expect not too far away.

deleted 8y ago

[deleted]

srean 8y ago

Thank you for asking the question I was about to ask. I am ambivalent over whether it has multiplexing or not. When I need parallelism I fall back to OS threads if available or processes. One or the other is usually always a first class citizen. Whats not that common, are real lightweight coroutines.... I want my laptop to be able to simulate the transport layer over a sizeable portion of the internet, while I watch youtube.