6 ms·

I really like how simply Julia unifies many computational concepts. For example, MPI is a classic solution to some of the same problems, but it is only for coar

I really like how simply Julia unifies many computational concepts. For example, MPI is a classic solution to some of the same problems, but it is only for coarse parallelism. OS-specific threading libraries are available for fine-grained parallelism, but the ligature between the two is typically messy. I have heard about Go's parallelism being awesome, does anyone know of a comparison between Go's and Julia's models?

Sukera 5y ago
According to this[1], Julias threading/task model is in part inspired by Go. I don't know about any detailed analysis on similarities and/or differences though. [1] https://julialang.org/blog/2019/07/multithreading/ https://julialang.org/blog/2019/07/multithreading/
gbrown 5y ago
I’ve been playing with GPGPU on Julia lately also, and it really seems like things have come a long way in the last few years. Check out the Juliacon 2021 talk on GPU compute if you’re interested.
- e12e 5y ago
  This one? https://live.juliacon.org/talk/P8KJSW https://live.juliacon.org/talk/P8KJSW
  - vchuravy 5y ago
    Probably more this talk about CUDA 3.0 https://live.juliacon.org/talk/UGX8YR https://live.juliacon.org/talk/UGX8YR or the workshop https://www.youtube.com/watch?v=Hz9IMJuW5hU https://www.youtube.com/watch?v=Hz9IMJuW5hU
    - gbrown 5y ago
      Yeah, I was thinking of the workshop.
- alhirzel 5y ago
  Are AMD and NVidia GPUs on par nowadays? Or is it still an NVidia-first world when it comes to compute support.
  - eigenspace 5y ago
    There was actually a question about this earlier today: https://discourse.julialang.org/t/amdgpu-jl-status/71191 https://discourse.julialang.org/t/amdgpu-jl-status/71191 TLDR: The entire GPUArrays.jl test suite now passes with AMDGPU.jl. There are still some missing features and it is not as mature as the nvidia version, but this space is progressing rapidly, and benefited from the generic GPU compilation pipeline that was initially built for CUDA.jl
    - snicker7 5y ago
      Keep in mind that AMDGPU.jl requires ROCm, which is basically dead (no recent GPUs support it and none of those that do are consumer-grade). The problem with AMD GPGPU is not software, it is that AMD literally does not care.
      - JonChesterfield 5y ago
        The gfx10 line (6800XT et al) probably work out of the box on a recent release. I think some are even officially supported. I test on a 5700XT which I don't think is officially supported. The change to 32 wide wavefronts took a while to resolve. Rocm gets releases every few months or so. The llvm project part is mirrored to GitHub in real time.
      - jpsamaroo 5y ago
        Definitely not dead; Vega is well supported, and with some tweaks, Polaris probably works too (although it definitely was broken in HIP around ROCm 4.0.0 or so). I think AMD has some work to do on non-C++/Python ecosystem engagement for sure, but they've built a foundation that's quite easy to build upon and get excellent performance and functionality; AMDGPU.jl is a testament to that.
adgjlsfhk1 5y ago
It's probably worth mentioning that Julia has MPI.jl which creates an array type that works over MPI transparently to the user.
- antoine-levitt 5y ago
  MPI.jl does not have an array wrapper type, it just exposes the bare mpi calls. Some other packages do use it to build array types.
masklinn 5y ago
> I have heard about Go's parallelism being awesome, does anyone know of a comparison between Go's and Julia's models? There's nothing fundamentally special to Go's "parallelism". It uses near-preempted (and now actually preempted I think, since a few versions back? before that they'd have implicit yield points at function calls but without function calls you could lock out the scheduler) userland threads with small stacks. The parallelism comes from the m:n scheduling. Go has no constructs for massive parallelisation, where you hand it a loop and it efficiently runs that on a hundred cores, at least not built in.
- adgjlsfhk1 5y ago
  I don't think there's anything too special with Go's constructs, it's just that they are very well optimized and widely used.
- initplus 5y ago
  Having lightweight userland threads fundamentally changes the way that you write concurrent programs. There is no need to worry about creating too many threads or of thread creation being too heavy. Old school patterns like process/thread/goroutine per connection are very natural to write in Go and result in dead simple code that performs well under load. Reasoning about threads & basic blocking calls is a simpler mental model compared to async/await style concurrency in my opinion. Go is fundamentally designed for developing backend applications, not high perf mathematical/scientific computing. Features like unrolling for-loops across hundreds of cores or offloading to GPU's would be out of place in the language.
  - dragontamer 5y ago
    > Having lightweight userland threads fundamentally changes the way that you write concurrent programs. There is no need to worry about creating too many threads or of thread creation being too heavy. Old school patterns like process/thread/goroutine per connection are very natural to write in Go and result in dead simple code that performs well under load. Well... there's still a problem in the tens-of-millions of Goroutines. But the issue is that OS-threads generally had issues at the ~100,000 of OS-threads. Making something "more lightweight" than pthreads makes sense, because 10,000,000 coroutines is a fundamentally different program design than 100,000 pthreads.

Sukera 5y ago

According to this[1], Julias threading/task model is in part inspired by Go. I don't know about any detailed analysis on similarities and/or differences though. [1] https://julialang.org/blog/2019/07/multithreading/ https://julialang.org/blog/2019/07/multithreading/

gbrown 5y ago

I’ve been playing with GPGPU on Julia lately also, and it really seems like things have come a long way in the last few years. Check out the Juliacon 2021 talk on GPU compute if you’re interested.

e12e 5y ago
This one? https://live.juliacon.org/talk/P8KJSW https://live.juliacon.org/talk/P8KJSW
- vchuravy 5y ago
  Probably more this talk about CUDA 3.0 https://live.juliacon.org/talk/UGX8YR https://live.juliacon.org/talk/UGX8YR or the workshop https://www.youtube.com/watch?v=Hz9IMJuW5hU https://www.youtube.com/watch?v=Hz9IMJuW5hU
  - gbrown 5y ago
    Yeah, I was thinking of the workshop.
alhirzel 5y ago
Are AMD and NVidia GPUs on par nowadays? Or is it still an NVidia-first world when it comes to compute support.
- eigenspace 5y ago
  There was actually a question about this earlier today: https://discourse.julialang.org/t/amdgpu-jl-status/71191 https://discourse.julialang.org/t/amdgpu-jl-status/71191 TLDR: The entire GPUArrays.jl test suite now passes with AMDGPU.jl. There are still some missing features and it is not as mature as the nvidia version, but this space is progressing rapidly, and benefited from the generic GPU compilation pipeline that was initially built for CUDA.jl
  - snicker7 5y ago
    Keep in mind that AMDGPU.jl requires ROCm, which is basically dead (no recent GPUs support it and none of those that do are consumer-grade). The problem with AMD GPGPU is not software, it is that AMD literally does not care.
    - JonChesterfield 5y ago
      The gfx10 line (6800XT et al) probably work out of the box on a recent release. I think some are even officially supported. I test on a 5700XT which I don't think is officially supported. The change to 32 wide wavefronts took a while to resolve. Rocm gets releases every few months or so. The llvm project part is mirrored to GitHub in real time.
    - jpsamaroo 5y ago
      Definitely not dead; Vega is well supported, and with some tweaks, Polaris probably works too (although it definitely was broken in HIP around ROCm 4.0.0 or so). I think AMD has some work to do on non-C++/Python ecosystem engagement for sure, but they've built a foundation that's quite easy to build upon and get excellent performance and functionality; AMDGPU.jl is a testament to that.

adgjlsfhk1 5y ago

It's probably worth mentioning that Julia has MPI.jl which creates an array type that works over MPI transparently to the user.

antoine-levitt 5y ago
MPI.jl does not have an array wrapper type, it just exposes the bare mpi calls. Some other packages do use it to build array types.

masklinn 5y ago

> I have heard about Go's parallelism being awesome, does anyone know of a comparison between Go's and Julia's models? There's nothing fundamentally special to Go's "parallelism". It uses near-preempted (and now actually preempted I think, since a few versions back? before that they'd have implicit yield points at function calls but without function calls you could lock out the scheduler) userland threads with small stacks. The parallelism comes from the m:n scheduling. Go has no constructs for massive parallelisation, where you hand it a loop and it efficiently runs that on a hundred cores, at least not built in.

adgjlsfhk1 5y ago
I don't think there's anything too special with Go's constructs, it's just that they are very well optimized and widely used.
initplus 5y ago
Having lightweight userland threads fundamentally changes the way that you write concurrent programs. There is no need to worry about creating too many threads or of thread creation being too heavy. Old school patterns like process/thread/goroutine per connection are very natural to write in Go and result in dead simple code that performs well under load. Reasoning about threads & basic blocking calls is a simpler mental model compared to async/await style concurrency in my opinion. Go is fundamentally designed for developing backend applications, not high perf mathematical/scientific computing. Features like unrolling for-loops across hundreds of cores or offloading to GPU's would be out of place in the language.
- dragontamer 5y ago
  > Having lightweight userland threads fundamentally changes the way that you write concurrent programs. There is no need to worry about creating too many threads or of thread creation being too heavy. Old school patterns like process/thread/goroutine per connection are very natural to write in Go and result in dead simple code that performs well under load. Well... there's still a problem in the tens-of-millions of Goroutines. But the issue is that OS-threads generally had issues at the ~100,000 of OS-threads. Making something "more lightweight" than pthreads makes sense, because 10,000,000 coroutines is a fundamentally different program design than 100,000 pthreads.