5 ms·

MATLAB, Python, and R don't have the metaprogramming tools for code generation. Tools like FFTW relied heavily on code generation, and people who wrote those ki

MATLAB, Python, and R don't have the metaprogramming tools for code generation. Tools like FFTW relied heavily on code generation, and people who wrote those kinds of tools are now using Julia for generic programming. Generic programming is difficult to handle with any AOT compilation system since you get a combinatoric explosion of possibilities leading to large compile times. Generic programming works great in Julia so there's a lot of examples of utilizing types for "free features" that would be difficult to do (interactively) in any other language. Of course, most programmers don't know what generic programming even is. However, a lot of libraries these days utilize it in order to get different precision and all of that, which is why I think Julia will at least become a language of libraries since not too many people can invest so much time in C++.

celrod 8y ago
Steven G. Johnson, who wrote much of FFTW in OCaml, is now a major Julia contributor! Or OCaml and C? Wikipedia isn't especially clear on that.
- ChrisRackauckas 8y ago
  It's OCaml which generates C.
setr 8y ago
>Generic programming is difficult to handle with any AOT compilation system since you get a combinatoric explosion of possibilities leading to large compile times. I thought it was either solved by adding function signatures based on actual usage, or “boxing” the argument (which I guess is just dynamic at that point) Why would you ever see the combinatoric explosion in an AOT implementation?
- ChrisRackauckas 8y ago
  If you want to make it as a standard library so it can compile once and others link to it, then you have this issue. You need to compile on use in order to only add the function signatures based on use, so if you're doing this from types the user is giving you (say, a generic differential equation solver where the state can be any type with the right overloads) then you're out of luck. If you know of a big set of types the user might want to give you, then you can compile those ahead of time, but for the diffeq example you can have different types for the dependent and independent variable, so you'd want each combination. The solution here of course is just to make people statically compile against it if you want to use all of these features, but you need to compile as a shared library if you want to build an easy interface to a high level scripting language (Julia, Python, R, MATLAB, etc.). So what tends to happen is that libraries which do make some extra features possible are constrained when used from a scripting language and they compile a bunch of choices to mitigate that a bit (see SUNDIALS and its built-in NVector compilations). The true solution here of course is to compile on demand based on the new types you see from the user codes, and this is what Julia does automatically. As for boxing, boxing works but it's slow. The developer of LightGraphs.jl tried using Swift for awhile but went back to Julia when he realized that actually using generics in Swift is really really slow because they do this boxing (your code is generic! But...). To finish the story, he tried Go a bit but generics in Go... he went to C++ but templates but they were difficult to use and the compile times were too long to be anything close to interactive, and has done extremely well over the last few years in Julia (though it is admittedly missing some tools for (partial) AOT compilation and building binaries, but these tools are in development).
  - celrod 8y ago
    If you want another example, I saw this a couple days ago: "Also exciting - we only have 27k signatures right now, and removing all the ones taking row vectors reduces that substantially to under 10k. That seems doable pretty easily, and we can convert without performance penalty between row and column vector seamlessly (I believe)." https://discourse.mc-stan.org/t/compiling-the-math-library-into-a-shared-library-or-object-files/5461/4 https://discourse.mc-stan.org/t/compiling-the-math-library-i... Those sound like big numbers to me! That's a real problem that DynamicHMC.jl doesn't have to worry about, for example. I think Stan's autodiff is faster than ForwardDiff.jl/ReverseDiff.jl (other than Stan models regularly taking several minutes to compile), but I'm betting on Cassette-powered Capstan/Zygote getting there in the next few years.
- marmaduke 8y ago
  He means you can’t do full specialization on all possible combinations of types AOT.
marmaduke 8y ago
> most programmers don't know what generic programming even is. Where do you get this from? You’re coming off far too aggressive. Metaclasses or AST rewriting decorators are parts of Python. I know they don’t fit your use case but you don’t need to set fire to multiple ecosystems because they don’t invoke a full blown compiler for every line of code.

celrod 8y ago

Steven G. Johnson, who wrote much of FFTW in OCaml, is now a major Julia contributor! Or OCaml and C? Wikipedia isn't especially clear on that.

ChrisRackauckas 8y ago
It's OCaml which generates C.

setr 8y ago

>Generic programming is difficult to handle with any AOT compilation system since you get a combinatoric explosion of possibilities leading to large compile times. I thought it was either solved by adding function signatures based on actual usage, or “boxing” the argument (which I guess is just dynamic at that point) Why would you ever see the combinatoric explosion in an AOT implementation?

ChrisRackauckas 8y ago
If you want to make it as a standard library so it can compile once and others link to it, then you have this issue. You need to compile on use in order to only add the function signatures based on use, so if you're doing this from types the user is giving you (say, a generic differential equation solver where the state can be any type with the right overloads) then you're out of luck. If you know of a big set of types the user might want to give you, then you can compile those ahead of time, but for the diffeq example you can have different types for the dependent and independent variable, so you'd want each combination. The solution here of course is just to make people statically compile against it if you want to use all of these features, but you need to compile as a shared library if you want to build an easy interface to a high level scripting language (Julia, Python, R, MATLAB, etc.). So what tends to happen is that libraries which do make some extra features possible are constrained when used from a scripting language and they compile a bunch of choices to mitigate that a bit (see SUNDIALS and its built-in NVector compilations). The true solution here of course is to compile on demand based on the new types you see from the user codes, and this is what Julia does automatically. As for boxing, boxing works but it's slow. The developer of LightGraphs.jl tried using Swift for awhile but went back to Julia when he realized that actually using generics in Swift is really really slow because they do this boxing (your code is generic! But...). To finish the story, he tried Go a bit but generics in Go... he went to C++ but templates but they were difficult to use and the compile times were too long to be anything close to interactive, and has done extremely well over the last few years in Julia (though it is admittedly missing some tools for (partial) AOT compilation and building binaries, but these tools are in development).
- celrod 8y ago
  If you want another example, I saw this a couple days ago: "Also exciting - we only have 27k signatures right now, and removing all the ones taking row vectors reduces that substantially to under 10k. That seems doable pretty easily, and we can convert without performance penalty between row and column vector seamlessly (I believe)." https://discourse.mc-stan.org/t/compiling-the-math-library-into-a-shared-library-or-object-files/5461/4 https://discourse.mc-stan.org/t/compiling-the-math-library-i... Those sound like big numbers to me! That's a real problem that DynamicHMC.jl doesn't have to worry about, for example. I think Stan's autodiff is faster than ForwardDiff.jl/ReverseDiff.jl (other than Stan models regularly taking several minutes to compile), but I'm betting on Cassette-powered Capstan/Zygote getting there in the next few years.
marmaduke 8y ago
He means you can’t do full specialization on all possible combinations of types AOT.

marmaduke 8y ago

> most programmers don't know what generic programming even is. Where do you get this from? You’re coming off far too aggressive. Metaclasses or AST rewriting decorators are parts of Python. I know they don’t fit your use case but you don’t need to set fire to multiple ecosystems because they don’t invoke a full blown compiler for every line of code.