4 ms·

A quick introduction to data parallelism in Julia

160 points by amkkma 6y ago

eigenspace 6y ago
Takafumi has really built up some amazing infrastructure in the package ecosystem. His Transducers.jl [1] is really interesting and powerful and lately he's done a lot of work with things like FLoops.jl [2] and ThreadsX.jl [3] to try and bring the benefits of transducers to more 'regular' familiar representations so more people can enjoy the benefits. The basic idea behind all of it is that he has an efficient and modular way of describing various 'looping' constructs that can be stuck together, optimized and parallelized automatically. It'd be quite interesting to see this stuff extended to GPUs. [1] https://github.com/JuliaFolds/Transducers.jl https://github.com/JuliaFolds/Transducers.jl [2] https://github.com/JuliaFolds/FLoops.jl https://github.com/JuliaFolds/FLoops.jl [3] https://github.com/tkf/ThreadsX.jl https://github.com/tkf/ThreadsX.jl
- cbkeller 6y ago
  Tkf has so many cool packages! Less to do with transducers or parallelism, but in terms of just cool and useful stuff, Maybe [1] and BangBang [2] definitely come to mind as well. [1] https://github.com/tkf/Maybe.jl https://github.com/tkf/Maybe.jl [2] https://github.com/JuliaFolds/BangBang.jl https://github.com/JuliaFolds/BangBang.jl
  - eigenspace 6y ago
    One that I really thought was cool recently was UnderscoreOh.jl [1]. He has a real knack for finding crazy fun things you can do with julia syntax. [1] https://github.com/tkf/UnderscoreOh.jl https://github.com/tkf/UnderscoreOh.jl
  - amkkma 6y ago
    Another one that's forward looking: https://github.com/tkf/Mutabilities.jl https://github.com/tkf/Mutabilities.jl
- cmcaine 6y ago
  I really like Baselet.jl[1] which provides Tuple-specialised implementations for a bunch of Base APIs that the compiler is very good at unrolling :) [1]: https://github.com/tkf/Baselet.jl https://github.com/tkf/Baselet.jl
melling 6y ago
There's a current MIT course call Introduction to Computational Thinking that's using Julia. I've watch a handful of videos so far. Good introduction to Julia. https://computationalthinking.mit.edu/Fall20/ https://computationalthinking.mit.edu/Fall20/
- Sukera 6y ago
  One of the lecturers working on this is Grant Sanderson, of 3Blue1Brown fame. Great video series so far!
tkf 6y ago
Hi, the author here. One of the hidden messages of the introduction is: watch Guy Steele's talks [1][2][3] if you are interested in data parallelism! These talks are not Julia-specific and the idea is applicable and very useful in any languages. My libraries are heavily inspired by these talks. Of course, if you haven't used Julia yet, it'd be great if (data) parallelism gives you an excuse to try it out! It has a fantastic foundation for composable multi-threaded parallelism [4]. [1] How to Think about Parallel Programming: Not! https://www.infoq.com/presentations/Thinking-Parallel-Programming/ https://www.infoq.com/presentations/Thinking-Parallel-Progra... [2] Four Solutions to a Trivial Problem https://www.youtube.com/watch?v=ftcIcn8AmSY https://www.youtube.com/watch?v=ftcIcn8AmSY [3] Organizing Functional Code for Parallel Execution; or, foldl and foldr Considered Slightly Harmful https://vimeo.com/6624203 https://vimeo.com/6624203 [4] Announcing composable multi-threaded parallelism in Julia https://julialang.org/blog/2019/07/multithreading/ https://julialang.org/blog/2019/07/multithreading/
- nextos 6y ago
  I love Julia, and in many areas both the language and standard libraries are quite polished. However, simple parallelism based on threads (e.g. a parallel map) surprisingly requires a third party library. ThreadsX, the one you posted about, is also my favorite option. Besides, the standard pipe operator (|>) doesn't support partial application. This has lead to several third-party reimplementations via macros, transducers, etc. Since this is a basic feature, fragmentation is a bit worrying.
  - tkf 6y ago
    Yeah, I get the point that it's nice to have basic things in the core language and stdlib. I'm rather on the "minimal language core" side in this discussion and I think it'd be better to not start adding more things in Julia Base and stdlib. At least not right now, to avoid accumulating "dead batteries." There are some non-trivial things to figure out for a composable data-parallel interface. For example, what the data collection interface should be to support folds over them? I'm using a minimalistic interface defined in SplittablesBase.jl for this but it's not wildly used outside my packages. So I'm not super confident that it's enough yet. [1] https://github.com/JuliaFolds/SplittablesBase.jl https://github.com/JuliaFolds/SplittablesBase.jl
blackbear_ 6y ago
Data parallelism is one of Python's most embarrassing shortcomings which is unlikely to be solved anytime soon. This could be a big reason to switch!

160 points by amkkma 6y ago

Takafumi has really built up some amazing infrastructure in the package ecosystem. His Transducers.jl [1] is really interesting and powerful and lately he's done a lot of work with things like FLoops.jl [2] and ThreadsX.jl [3] to try and bring the benefits of transducers to more 'regular' familiar representations so more people can enjoy the benefits. The basic idea behind all of it is that he has an efficient and modular way of describing various 'looping' constructs that can be stuck together, optimized and parallelized automatically. It'd be quite interesting to see this stuff extended to GPUs. [1] https://github.com/JuliaFolds/Transducers.jl https://github.com/JuliaFolds/Transducers.jl [2] https://github.com/JuliaFolds/FLoops.jl https://github.com/JuliaFolds/FLoops.jl [3] https://github.com/tkf/ThreadsX.jl https://github.com/tkf/ThreadsX.jl

cbkeller 6y ago
Tkf has so many cool packages! Less to do with transducers or parallelism, but in terms of just cool and useful stuff, Maybe [1] and BangBang [2] definitely come to mind as well. [1] https://github.com/tkf/Maybe.jl https://github.com/tkf/Maybe.jl [2] https://github.com/JuliaFolds/BangBang.jl https://github.com/JuliaFolds/BangBang.jl
- eigenspace 6y ago
  One that I really thought was cool recently was UnderscoreOh.jl [1]. He has a real knack for finding crazy fun things you can do with julia syntax. [1] https://github.com/tkf/UnderscoreOh.jl https://github.com/tkf/UnderscoreOh.jl
- amkkma 6y ago
  Another one that's forward looking: https://github.com/tkf/Mutabilities.jl https://github.com/tkf/Mutabilities.jl
cmcaine 6y ago
I really like Baselet.jl[1] which provides Tuple-specialised implementations for a bunch of Base APIs that the compiler is very good at unrolling :) [1]: https://github.com/tkf/Baselet.jl https://github.com/tkf/Baselet.jl

melling 6y ago

There's a current MIT course call Introduction to Computational Thinking that's using Julia. I've watch a handful of videos so far. Good introduction to Julia. https://computationalthinking.mit.edu/Fall20/ https://computationalthinking.mit.edu/Fall20/

Sukera 6y ago
One of the lecturers working on this is Grant Sanderson, of 3Blue1Brown fame. Great video series so far!

tkf 6y ago

Hi, the author here. One of the hidden messages of the introduction is: watch Guy Steele's talks [1][2][3] if you are interested in data parallelism! These talks are not Julia-specific and the idea is applicable and very useful in any languages. My libraries are heavily inspired by these talks. Of course, if you haven't used Julia yet, it'd be great if (data) parallelism gives you an excuse to try it out! It has a fantastic foundation for composable multi-threaded parallelism [4]. [1] How to Think about Parallel Programming: Not! https://www.infoq.com/presentations/Thinking-Parallel-Programming/ https://www.infoq.com/presentations/Thinking-Parallel-Progra... [2] Four Solutions to a Trivial Problem https://www.youtube.com/watch?v=ftcIcn8AmSY https://www.youtube.com/watch?v=ftcIcn8AmSY [3] Organizing Functional Code for Parallel Execution; or, foldl and foldr Considered Slightly Harmful https://vimeo.com/6624203 https://vimeo.com/6624203 [4] Announcing composable multi-threaded parallelism in Julia https://julialang.org/blog/2019/07/multithreading/ https://julialang.org/blog/2019/07/multithreading/

nextos 6y ago
I love Julia, and in many areas both the language and standard libraries are quite polished. However, simple parallelism based on threads (e.g. a parallel map) surprisingly requires a third party library. ThreadsX, the one you posted about, is also my favorite option. Besides, the standard pipe operator (|>) doesn't support partial application. This has lead to several third-party reimplementations via macros, transducers, etc. Since this is a basic feature, fragmentation is a bit worrying.
- tkf 6y ago
  Yeah, I get the point that it's nice to have basic things in the core language and stdlib. I'm rather on the "minimal language core" side in this discussion and I think it'd be better to not start adding more things in Julia Base and stdlib. At least not right now, to avoid accumulating "dead batteries." There are some non-trivial things to figure out for a composable data-parallel interface. For example, what the data collection interface should be to support folds over them? I'm using a minimalistic interface defined in SplittablesBase.jl for this but it's not wildly used outside my packages. So I'm not super confident that it's enough yet. [1] https://github.com/JuliaFolds/SplittablesBase.jl https://github.com/JuliaFolds/SplittablesBase.jl

blackbear_ 6y ago

Data parallelism is one of Python's most embarrassing shortcomings which is unlikely to be solved anytime soon. This could be a big reason to switch!