6 ms·

Extreme Multi-Threading: C++ and Julia 1.9 Integration

125 points by pieterr 3y ago

hsjqllzlfkf 3y ago
[flagged]
- sundarurfriend 3y ago
  This seems like an unnecessary meta comment that only detracts from discussion of the article. No one is going to be convinced by your comment, and it only makes it more likely there's further arguments about this (rather than the article) here.
- deleted 3y ago
  [deleted]
sriram_malhar 3y ago
What is extreme about this multi-threading?
- nestorD 3y ago
  Because it demonstrates a Julia library using threads created in the calling C++ code. That passing of threads between languages is highly uncommon (because many things could fail and make you crash / slow along the way and because if your language can create threads then it likely already has C-like performance so you will not introduce parallel libraries written in another language for performance reasons).
  - medo-bear 3y ago
    aren't you just embedding julia into cpp?
    - scientificcoder 3y ago
      I consider multi-threaded multi-language embedding an extreme sport. Maybe you are a software athlete and it's easy for you, but I think it's pretty scary.
      - medo-bear 3y ago
        maybe i am missing something but this is old news for embeddable common lisp. see threading section in this article from 1994[0], or is this extreme-threading different somehow? there is also a recent llvm-based implementation [1] that aims for direct lisp-cpp interface. i know my usage of the word "just" might suggest otherwise but i am not trying to deminish this feat. i use common lisp for scientific work and whenever i hear about something cool in another language i ask myself can i do this with my toolset. this is the reason i am asking this. [0] https://dl.acm.org/doi/pdf/10.1145/224139.1379849 https://dl.acm.org/doi/pdf/10.1145/224139.1379849 [1] https://clasp-developers.github.io/ https://clasp-developers.github.io/
        andyferris 3y ago
        I think it’s just that this is possible in (the recently released) Julia 1.9 and wasn’t possible in earlier versions due to how thread-local storage and the GC worked, and doing this can be useful sometimes.
        NeuroCoder 3y ago
        I think the title was intended to be more playful and fun than a serious take on the most technically intense applications of threading in C++.
        adgjlsfhk1 3y ago
        Your link 0 is cool, but is fairly different from the new stuff here. Some key differences are that the paper uses a conservative garbage collector (so enjoy random memory leaks) as a way to get around the problems of interop with C. Also, while that paper supports multithreading, as far as I can tell, it meant n-thread to 1 core (i.e. concurrent) multithreading (which is what the term usually referred to before multicore cpus were common), while Julia supports n threads to m cores (see Java's long awaited project Loom) which has a bunch of additional challenges. The last thing to mention is that the paper required a near complete rewrite of the lisp compiler. While that's an option for a simple language like Lisp, it's not as feasible for more complicated languages.
  - never_inline 3y ago
    > Julia library using threads created in the calling C++ code. I understand Julia is much more dynamic language. But Java JNI can do this too. You have to attach the thread manually, and take care of pinned references.
    - adgjlsfhk1 3y ago
      java jni is also really slow. it causes a bunch of copying and pointer indirections. Julia's C interop is zero cost.
      - never_inline 3y ago
        > it causes a bunch of copying and pointer indirections. That's correct. As you said elsewhere, lot of it pins down (!) to design of Java. There's direct byte buffer, but you have to architect your code around nio APIs to utilize it. But we were talking about working with native threads.
      - sundarurfriend 3y ago
        > java jni is also really slow. it causes a bunch of copying and pointer indirections. Are there any safety reasons for it or is it just a sign of its age?
        adgjlsfhk1 3y ago
        it's less safety and more design. java uses a moving GC, so there need to be indirections to allow the GC to move objects without C trying to access them and segfaulting. also, java only has objects, so all the data structures are wrong for C which means most uses in practice will end up copying all the memory anyway. Julia's structs are compatible with C structs, and the GC is non-moving, so you don't have random indirections. another advantage is that Julia generates machine code and then doesn't do all the fancy JIT stuff java does with multiple rounds of optimization and deoptimization, so once you've generated a function in Julia, you can just cal the pointer from C and it won't disappear from under you.
  - harha_ 3y ago
    Without knowing anything about julia; is thread creation/management better on the C++-side or why wouldn't you just create those threads on the julia side? Anyways, it is cool if it's stable and works well. I guess there's a use case for such feature, even if it sounds strange to an outsider.
    - Archit3ch 3y ago
      One possible use case is that you didn't write/control the C-++ side, yet you can still extend it with Julia.
    - sundarurfriend 3y ago
      It sounds like they already had multithreaded code in C++.
- tourgen 3y ago
  [dead]
logicchains 3y ago
This is great! What feels like ages ago I attempted to include the Julia runtime in a C++ application and it was quite difficult, as only one thread was allowed to call the runtime (not just one thread at a time, one particular thread; if it was called by any other thread than that one particular thread it or the GC would get corrupted somehow and crash).
- vchuravy 3y ago
  Yes that is precisely what was fixed, essentially the thread local storage that Julia was expecting were not setup and thus calling the runtime from a foreign thread would cause a crash. This now enables to dynamically add and remove threads.
sundarurfriend 3y ago
I was confused about why this doesn't mention libraries like CxxWrap, and how many rough edges there seem to be, but then I realized this isn't the common case of calling out to a C++ library from Julia. It's about compiling Julia code into a library that can then be linked to from C++ - which I didn't even realize was possible!
- deleted 3y ago
  [deleted]
- cookieperson 3y ago
  Cxxwrap was broken for a while there, maybe a few years? Maybe it was cxx... I forget, whichever one let you have a c++ repl. But yea I'd argue that it's a pretty common use case to embed Julia in c++. This has been possible for a long time, Julia is written in C after all, but it's hard to do meaningfully because of Julia's GC. You have to do a lot of sketchy stuff to ensure values don't get deleted as you call for them via ffi. This might surprise you but you can do this with pretty much any other languages, usually with far less RAM consumption and JIT interruptions though...
  - ihnorton 3y ago
    > Cxxwrap was broken for a while there, maybe a few years? Maybe it was cxx Yes, Cxx: https://github.com/JuliaInterop/Cxx.jl https://github.com/JuliaInterop/Cxx.jl -- it uses Clang to generate JIT'd interop thunks, and does neat things to enable that, like cross-language type inference and inlining (in addition to the REPL). CxxWrap is a different thing, which AFAIK has been actively maintained for > 5 years now: https://github.com/JuliaInterop/CxxWrap.jl https://github.com/JuliaInterop/CxxWrap.jl It is similar to Boost.Python/pybind11/nanobind: bindings are written in C++ and compiled ahead of time into a module that defines Julia entrypoints. Those entrypoints take care of signature selection, translation to/from Julia objects, and lifetime bookkeeping.
    - alhirzel 3y ago
      If you're using plain-old-C, don't overlook the excellent CBinding.jl: https://github.com/analytech-solutions/CBinding.jl https://github.com/analytech-solutions/CBinding.jl
NeutralForest 3y ago
Very cool! thanks for sharing
eigenspace 3y ago
Nice! I'm really enjoying your blogposts, and it's very cool to see julia being used at ASML for stuff like this.

125 points by pieterr 3y ago

hsjqllzlfkf 3y ago

[flagged]

sundarurfriend 3y ago
This seems like an unnecessary meta comment that only detracts from discussion of the article. No one is going to be convinced by your comment, and it only makes it more likely there's further arguments about this (rather than the article) here.
deleted 3y ago
[deleted]

sriram_malhar 3y ago

What is extreme about this multi-threading?

nestorD 3y ago
Because it demonstrates a Julia library using threads created in the calling C++ code. That passing of threads between languages is highly uncommon (because many things could fail and make you crash / slow along the way and because if your language can create threads then it likely already has C-like performance so you will not introduce parallel libraries written in another language for performance reasons).
- medo-bear 3y ago
  aren't you just embedding julia into cpp?
  - scientificcoder 3y ago
    I consider multi-threaded multi-language embedding an extreme sport. Maybe you are a software athlete and it's easy for you, but I think it's pretty scary.
    - medo-bear 3y ago
      maybe i am missing something but this is old news for embeddable common lisp. see threading section in this article from 1994[0], or is this extreme-threading different somehow? there is also a recent llvm-based implementation [1] that aims for direct lisp-cpp interface. i know my usage of the word "just" might suggest otherwise but i am not trying to deminish this feat. i use common lisp for scientific work and whenever i hear about something cool in another language i ask myself can i do this with my toolset. this is the reason i am asking this. [0] https://dl.acm.org/doi/pdf/10.1145/224139.1379849 https://dl.acm.org/doi/pdf/10.1145/224139.1379849 [1] https://clasp-developers.github.io/ https://clasp-developers.github.io/
      - andyferris 3y ago
        I think it’s just that this is possible in (the recently released) Julia 1.9 and wasn’t possible in earlier versions due to how thread-local storage and the GC worked, and doing this can be useful sometimes.
      - NeuroCoder 3y ago
        I think the title was intended to be more playful and fun than a serious take on the most technically intense applications of threading in C++.
      - adgjlsfhk1 3y ago
        Your link 0 is cool, but is fairly different from the new stuff here. Some key differences are that the paper uses a conservative garbage collector (so enjoy random memory leaks) as a way to get around the problems of interop with C. Also, while that paper supports multithreading, as far as I can tell, it meant n-thread to 1 core (i.e. concurrent) multithreading (which is what the term usually referred to before multicore cpus were common), while Julia supports n threads to m cores (see Java's long awaited project Loom) which has a bunch of additional challenges. The last thing to mention is that the paper required a near complete rewrite of the lisp compiler. While that's an option for a simple language like Lisp, it's not as feasible for more complicated languages.
- never_inline 3y ago
  > Julia library using threads created in the calling C++ code. I understand Julia is much more dynamic language. But Java JNI can do this too. You have to attach the thread manually, and take care of pinned references.
  - adgjlsfhk1 3y ago
    java jni is also really slow. it causes a bunch of copying and pointer indirections. Julia's C interop is zero cost.
    - never_inline 3y ago
      > it causes a bunch of copying and pointer indirections. That's correct. As you said elsewhere, lot of it pins down (!) to design of Java. There's direct byte buffer, but you have to architect your code around nio APIs to utilize it. But we were talking about working with native threads.
    - sundarurfriend 3y ago
      > java jni is also really slow. it causes a bunch of copying and pointer indirections. Are there any safety reasons for it or is it just a sign of its age?
      - adgjlsfhk1 3y ago
        it's less safety and more design. java uses a moving GC, so there need to be indirections to allow the GC to move objects without C trying to access them and segfaulting. also, java only has objects, so all the data structures are wrong for C which means most uses in practice will end up copying all the memory anyway. Julia's structs are compatible with C structs, and the GC is non-moving, so you don't have random indirections. another advantage is that Julia generates machine code and then doesn't do all the fancy JIT stuff java does with multiple rounds of optimization and deoptimization, so once you've generated a function in Julia, you can just cal the pointer from C and it won't disappear from under you.
- harha_ 3y ago
  Without knowing anything about julia; is thread creation/management better on the C++-side or why wouldn't you just create those threads on the julia side? Anyways, it is cool if it's stable and works well. I guess there's a use case for such feature, even if it sounds strange to an outsider.
  - Archit3ch 3y ago
    One possible use case is that you didn't write/control the C-++ side, yet you can still extend it with Julia.
  - sundarurfriend 3y ago
    It sounds like they already had multithreaded code in C++.
tourgen 3y ago
[dead]

logicchains 3y ago

This is great! What feels like ages ago I attempted to include the Julia runtime in a C++ application and it was quite difficult, as only one thread was allowed to call the runtime (not just one thread at a time, one particular thread; if it was called by any other thread than that one particular thread it or the GC would get corrupted somehow and crash).

vchuravy 3y ago
Yes that is precisely what was fixed, essentially the thread local storage that Julia was expecting were not setup and thus calling the runtime from a foreign thread would cause a crash. This now enables to dynamically add and remove threads.

sundarurfriend 3y ago

I was confused about why this doesn't mention libraries like CxxWrap, and how many rough edges there seem to be, but then I realized this isn't the common case of calling out to a C++ library from Julia. It's about compiling Julia code into a library that can then be linked to from C++ - which I didn't even realize was possible!

deleted 3y ago
[deleted]
cookieperson 3y ago
Cxxwrap was broken for a while there, maybe a few years? Maybe it was cxx... I forget, whichever one let you have a c++ repl. But yea I'd argue that it's a pretty common use case to embed Julia in c++. This has been possible for a long time, Julia is written in C after all, but it's hard to do meaningfully because of Julia's GC. You have to do a lot of sketchy stuff to ensure values don't get deleted as you call for them via ffi. This might surprise you but you can do this with pretty much any other languages, usually with far less RAM consumption and JIT interruptions though...
- ihnorton 3y ago
  > Cxxwrap was broken for a while there, maybe a few years? Maybe it was cxx Yes, Cxx: https://github.com/JuliaInterop/Cxx.jl https://github.com/JuliaInterop/Cxx.jl -- it uses Clang to generate JIT'd interop thunks, and does neat things to enable that, like cross-language type inference and inlining (in addition to the REPL). CxxWrap is a different thing, which AFAIK has been actively maintained for > 5 years now: https://github.com/JuliaInterop/CxxWrap.jl https://github.com/JuliaInterop/CxxWrap.jl It is similar to Boost.Python/pybind11/nanobind: bindings are written in C++ and compiled ahead of time into a module that defines Julia entrypoints. Those entrypoints take care of signature selection, translation to/from Julia objects, and lifetime bookkeeping.
  - alhirzel 3y ago
    If you're using plain-old-C, don't overlook the excellent CBinding.jl: https://github.com/analytech-solutions/CBinding.jl https://github.com/analytech-solutions/CBinding.jl

NeutralForest 3y ago

Very cool! thanks for sharing

eigenspace 3y ago

Nice! I'm really enjoying your blogposts, and it's very cool to see julia being used at ASML for stuff like this.