3 ms·

> Yes, julia can be called from other languages rather easily This seems false to me. StaticCompiler.jl [1] puts in their limitations that "GC-tracked allocati

> Yes, julia can be called from other languages rather easily This seems false to me. StaticCompiler.jl [1] puts in their limitations that "GC-tracked allocations and global variables do not work with compile_executable or compile_shlib. This has some interesting consequences, including that all functions within the function you want to compile must either be inlined or return only native types (otherwise Julia would have to allocate a place to put the results, which will fail)." In practice, this means that you can therefore not use the base library in your own programs. PackageCompiler.jl [2] has the same limitations if I'm not mistaken. So then you have to fall back to distributing the Julia "binary" with a full Julia runtime, which is pretty heavy. There are some packages, such as PySR [3], which do this. It seems pretty usable for research I'd say, but difficult to put in production. There is some word going around though that there is an even better static compiler in the making, but as long as that one is not publicly available I'd say that Julia cannot easily be called from other languages. [1]: https://github.com/tshort/StaticCompiler.jl https://github.com/tshort/StaticCompiler.jl [2]: https://github.com/JuliaLang/PackageCompiler.jl https://github.com/JuliaLang/PackageCompiler.jl [3]: https://github.com/MilesCranmer/PySR https://github.com/MilesCranmer/PySR

eigenspace 3y ago
I think you misunderstood me. Nothing about calling julia from other languages requires StaticCompiler.jl, that's just an experimental proof of concept for small, standalone binary generation with some heavy limitations. Calling julia from another language does not in general require AOT compilation, though it does help to make things more portable and self contained. If one wants, they can literally just spawn a julia process, define a function on the fly and then hook into that process and call that function from another language. > PackageCompiler.jl [2] has the same limitations if I'm not mistaken You are mistaken. PackageCompiler works by basically bundling an entire julia system image with your program and the whole compiler, runtime, and stdlib compiled into one bundle together. If a piece of code works in the REPL it'll work from PackageCompiler
- huijzer 3y ago
  > If one wants, they can literally just spawn a julia process, define a function on the fly and then hook into that process and call that function from another language. Okay but how do you distribute the program that depends on Julia to clients? You then also need to ensure that they have the right Julia available on their system. That's much heavier than dynamic binaries (which rely only on some shared libraries). My point was that it's not always "easy" to call Julia. That it's easy in HPC is a reasonable statement since you probably have full control over the system and it has loads of RAM available, but I wouldn't call it easy in general. > You are mistaken. I guess we're both right. I was talking about the "libraries" functionality [1]. [1]: https://julialang.github.io/PackageCompiler.jl/dev/libs.html https://julialang.github.io/PackageCompiler.jl/dev/libs.html
  - eigenspace 3y ago
    > Okay but how do you distribute the program that depends on Julia to clients? That's not the question that was asked though. The question the person asked as "Can you call Julia routines from other languages?" and the answer to that is "yes, you can easily do that". __________________________________________________ > I guess we're both right. I was talking about the "libraries" functionality I'm confused, what functionality exactly are you saying is missing from the libraries functionality? That very example you link to shows one using the julia runtime via printing. And yes, dynamic dispatch and dynamic code generation do work from libraries created by PackageCompiler.jl as well...
    - sanderjd 3y ago
      I would quibble with your use of the word "easily", but I consider it amazing that a language with such a complex runtime manages to expose an embeddable interface at all. Julia is an incredible language.
    - anewhnaccount2 3y ago
      I guess the problem is that if you create a shared-library, yes, you can call it from Python, but you are now restricted to C datatypes, and don't get any of the type-conversion or type-proxying that the other bridges provide --- as far as I can see. That said, I think the bridges are actually pretty developed compared to what exists between many pairs of high-level languages. They are clearly working for some people since you will find Python-wrapper packages for Julia code. I'm not so sure it's so important to have a single `.so` package. There are solutions which allow Python to be-in-charge and bundle Julia to various extents. There are at least: 1. https://github.com/jlapeyre/julia_project https://github.com/jlapeyre/julia_project 2. Usage of Conda and https://github.com/JuliaPy/pyjuliapkg https://github.com/JuliaPy/pyjuliapkg

eigenspace 3y ago

I think you misunderstood me. Nothing about calling julia from other languages requires StaticCompiler.jl, that's just an experimental proof of concept for small, standalone binary generation with some heavy limitations. Calling julia from another language does not in general require AOT compilation, though it does help to make things more portable and self contained. If one wants, they can literally just spawn a julia process, define a function on the fly and then hook into that process and call that function from another language. > PackageCompiler.jl [2] has the same limitations if I'm not mistaken You are mistaken. PackageCompiler works by basically bundling an entire julia system image with your program and the whole compiler, runtime, and stdlib compiled into one bundle together. If a piece of code works in the REPL it'll work from PackageCompiler

huijzer 3y ago
> If one wants, they can literally just spawn a julia process, define a function on the fly and then hook into that process and call that function from another language. Okay but how do you distribute the program that depends on Julia to clients? You then also need to ensure that they have the right Julia available on their system. That's much heavier than dynamic binaries (which rely only on some shared libraries). My point was that it's not always "easy" to call Julia. That it's easy in HPC is a reasonable statement since you probably have full control over the system and it has loads of RAM available, but I wouldn't call it easy in general. > You are mistaken. I guess we're both right. I was talking about the "libraries" functionality [1]. [1]: https://julialang.github.io/PackageCompiler.jl/dev/libs.html https://julialang.github.io/PackageCompiler.jl/dev/libs.html
- eigenspace 3y ago
  > Okay but how do you distribute the program that depends on Julia to clients? That's not the question that was asked though. The question the person asked as "Can you call Julia routines from other languages?" and the answer to that is "yes, you can easily do that". __________________________________________________ > I guess we're both right. I was talking about the "libraries" functionality I'm confused, what functionality exactly are you saying is missing from the libraries functionality? That very example you link to shows one using the julia runtime via printing. And yes, dynamic dispatch and dynamic code generation do work from libraries created by PackageCompiler.jl as well...
  - sanderjd 3y ago
    I would quibble with your use of the word "easily", but I consider it amazing that a language with such a complex runtime manages to expose an embeddable interface at all. Julia is an incredible language.
  - anewhnaccount2 3y ago
    I guess the problem is that if you create a shared-library, yes, you can call it from Python, but you are now restricted to C datatypes, and don't get any of the type-conversion or type-proxying that the other bridges provide --- as far as I can see. That said, I think the bridges are actually pretty developed compared to what exists between many pairs of high-level languages. They are clearly working for some people since you will find Python-wrapper packages for Julia code. I'm not so sure it's so important to have a single `.so` package. There are solutions which allow Python to be-in-charge and bundle Julia to various extents. There are at least: 1. https://github.com/jlapeyre/julia_project https://github.com/jlapeyre/julia_project 2. Usage of Conda and https://github.com/JuliaPy/pyjuliapkg https://github.com/JuliaPy/pyjuliapkg