5 ms·

Hi, the author here. One of the hidden messages of the introduction is: watch Guy Steele's talks [1][2][3] if you are interested in data parallelism! These ta

by tkf 6y ago

Hi, the author here. One of the hidden messages of the introduction is: watch Guy Steele's talks [1][2][3] if you are interested in data parallelism! These talks are not Julia-specific and the idea is applicable and very useful in any languages. My libraries are heavily inspired by these talks. Of course, if you haven't used Julia yet, it'd be great if (data) parallelism gives you an excuse to try it out! It has a fantastic foundation for composable multi-threaded parallelism [4]. [1] How to Think about Parallel Programming: Not! https://www.infoq.com/presentations/Thinking-Parallel-Programming/ https://www.infoq.com/presentations/Thinking-Parallel-Progra... [2] Four Solutions to a Trivial Problem https://www.youtube.com/watch?v=ftcIcn8AmSY https://www.youtube.com/watch?v=ftcIcn8AmSY [3] Organizing Functional Code for Parallel Execution; or, foldl and foldr Considered Slightly Harmful https://vimeo.com/6624203 https://vimeo.com/6624203 [4] Announcing composable multi-threaded parallelism in Julia https://julialang.org/blog/2019/07/multithreading/ https://julialang.org/blog/2019/07/multithreading/

nextos 6y ago
I love Julia, and in many areas both the language and standard libraries are quite polished. However, simple parallelism based on threads (e.g. a parallel map) surprisingly requires a third party library. ThreadsX, the one you posted about, is also my favorite option. Besides, the standard pipe operator (|>) doesn't support partial application. This has lead to several third-party reimplementations via macros, transducers, etc. Since this is a basic feature, fragmentation is a bit worrying.
- tkf 6y ago
  Yeah, I get the point that it's nice to have basic things in the core language and stdlib. I'm rather on the "minimal language core" side in this discussion and I think it'd be better to not start adding more things in Julia Base and stdlib. At least not right now, to avoid accumulating "dead batteries." There are some non-trivial things to figure out for a composable data-parallel interface. For example, what the data collection interface should be to support folds over them? I'm using a minimalistic interface defined in SplittablesBase.jl for this but it's not wildly used outside my packages. So I'm not super confident that it's enough yet. [1] https://github.com/JuliaFolds/SplittablesBase.jl https://github.com/JuliaFolds/SplittablesBase.jl
- eigenspace 6y ago
  Just to echo Tkf here, I think it'd be great to have something like ThreadsX.map in base someday, but when the multi-threading features in Julia first came out, it was unclear what the best interface to provide would be. Since Julia is committed to semantic versioning, this is a kinda scary prospect because it means that any high level, exported interface we decide on, we'll be stuck with until at least 2.0 and probably forever. So with things like this, it's important to be conservative about the interfaces we expose and instead, we've have people explore different approaches out in the package ecosystem. One day, if someone can make a strong enough argument, I'm sure we'll see one of these package solutions end up in Base.
  - nextos 6y ago
    Yes, I agree getting design choices right is important in order not to end up with a lot of legacy code in future versions of Julia. I think the pipe operator is probably the area that needs most urgent attention. Lots of libraries have their own slightly different macros for improved pipes with partial application, and that introduces a bit of fragmentation and hurts composability. There's an open issue for this with some background: https://github.com/JuliaLang/julia/issues/5571 https://github.com/JuliaLang/julia/issues/5571
sroussey 6y ago
Given your experience, what primitives would you advocate for JS for data parallelism?
- tkf 6y ago
  For composable nested parallelism, I think spawn and sync like Julia has would be great. If there is a variant of spawn that asserts there is no concurrency is required for the child task, I'd imagine the runtime can do more optimizations (Julia doesn't have it but there was an experimental PR [1]). But concurrency sounds like a very basic thing in JavaScript so I'm not sure if it fits well in the language. But I don't have experience in serious JavaScript programming so I don't know what fits best there. I'd imagine you can build some data parallel libraries already with web workers API. [1] https://github.com/JuliaLang/julia/pull/31086 https://github.com/JuliaLang/julia/pull/31086

by tkf 6y ago

nextos 6y ago

I love Julia, and in many areas both the language and standard libraries are quite polished. However, simple parallelism based on threads (e.g. a parallel map) surprisingly requires a third party library. ThreadsX, the one you posted about, is also my favorite option. Besides, the standard pipe operator (|>) doesn't support partial application. This has lead to several third-party reimplementations via macros, transducers, etc. Since this is a basic feature, fragmentation is a bit worrying.

tkf 6y ago
Yeah, I get the point that it's nice to have basic things in the core language and stdlib. I'm rather on the "minimal language core" side in this discussion and I think it'd be better to not start adding more things in Julia Base and stdlib. At least not right now, to avoid accumulating "dead batteries." There are some non-trivial things to figure out for a composable data-parallel interface. For example, what the data collection interface should be to support folds over them? I'm using a minimalistic interface defined in SplittablesBase.jl for this but it's not wildly used outside my packages. So I'm not super confident that it's enough yet. [1] https://github.com/JuliaFolds/SplittablesBase.jl https://github.com/JuliaFolds/SplittablesBase.jl
eigenspace 6y ago
Just to echo Tkf here, I think it'd be great to have something like ThreadsX.map in base someday, but when the multi-threading features in Julia first came out, it was unclear what the best interface to provide would be. Since Julia is committed to semantic versioning, this is a kinda scary prospect because it means that any high level, exported interface we decide on, we'll be stuck with until at least 2.0 and probably forever. So with things like this, it's important to be conservative about the interfaces we expose and instead, we've have people explore different approaches out in the package ecosystem. One day, if someone can make a strong enough argument, I'm sure we'll see one of these package solutions end up in Base.
- nextos 6y ago
  Yes, I agree getting design choices right is important in order not to end up with a lot of legacy code in future versions of Julia. I think the pipe operator is probably the area that needs most urgent attention. Lots of libraries have their own slightly different macros for improved pipes with partial application, and that introduces a bit of fragmentation and hurts composability. There's an open issue for this with some background: https://github.com/JuliaLang/julia/issues/5571 https://github.com/JuliaLang/julia/issues/5571

sroussey 6y ago

Given your experience, what primitives would you advocate for JS for data parallelism?

tkf 6y ago
For composable nested parallelism, I think spawn and sync like Julia has would be great. If there is a variant of spawn that asserts there is no concurrency is required for the child task, I'd imagine the runtime can do more optimizations (Julia doesn't have it but there was an experimental PR [1]). But concurrency sounds like a very basic thing in JavaScript so I'm not sure if it fits well in the language. But I don't have experience in serious JavaScript programming so I don't know what fits best there. I'd imagine you can build some data parallel libraries already with web workers API. [1] https://github.com/JuliaLang/julia/pull/31086 https://github.com/JuliaLang/julia/pull/31086