4 ms·
In Julia, where the paralleization options are explicit (SIMD, AVX, threads or multiprocessing), it always depends on the load, for small operation (around 1000
by longemen3000 7y ago
In Julia, where the paralleization options are explicit (SIMD, AVX, threads or multiprocessing), it always depends on the load, for small operation (around 10000 elements) a single thread is faster only for the thread spawning time (around 1 microsecond). And there is the issue of the independent Blas threaded model, where the Blas threads sometimes interfere with Julia threads...
In a nutshell, parallelization is not a magical bullet, but is a good bullet to have at your disposal anyway
- The_rationalist 7y agoDo you know if Julia will add OpenMP support? It's clearly the way to go for offloading to hardware in a productive way.
- eigenspace 7y agoI don't know about "clearly the way to go". I think Julia's parallelism models have proven themselves to be very robust, performant and composeable, moreso than OpenMP as far as I'm aware.
- The_rationalist 7y agoHow can I annotate an existing loop to offload it on the GPU Inclusive OR on AVX IOR on cpu cores. Without this ability, in practice I use far less parallelism.
- dagw 7y agoThis is currently no official solution in Julia that I'm aware of. However there are several people working on it and a few experimental solution are under active development https://github.com/JuliaDiffEq/AutoOffload.jl https://github.com/JuliaDiffEq/AutoOffload.jl https://juliagpu.gitlab.io/GPUifyLoops.jl/ https://juliagpu.gitlab.io/GPUifyLoops.jl/
- ChrisRackauckas 7y agoAutoOffload is something different, where it's trying to do linear algebra in a way that auto-offloads to GPUs or heterogeneous. GPUifyLoops is correct for this answer, and its next incarnation is KernelAbstractions.jl. These auto-construct GPU kernels and such from loops.
- dagw 7y agoJulia is actually initially had OpenMP backed parallelism (ParallelAccelerator.jl), but they're moving away from OpenMP towards a novel and native task parallelism framework more inspired by things like Cilk[0]. [0] https://julialang.org/blog/2019/07/multithreading/ https://julialang.org/blog/2019/07/multithreading/
- The_rationalist 7y agoCilk was officially deprecated in favor of openMP. But if it's just about inspiration, OK.
- ChrisRackauckas 7y ago> And there is the issue of the independent Blas threaded model, where the Blas threads sometimes interfere with Julia threads Julia has composible multithreading, and using that model fixed composing FFTW threads with Julia's. This can be done to OpenBLAS as well, and IIRC there is a PR open for it.
- longemen3000 7y agoYeah, I'm waiting for that PR haahah