6 ms·

I work professionally as an applied mathematician and I've struggled to understand where Julia fits into the tools already available. In terms of prototyping a

by kxyvr 8y ago

I work professionally as an applied mathematician and I've struggled to understand where Julia fits into the tools already available. In terms of prototyping algorithms, MATLAB/Octave still seems to be the best choice. We have access to an enormous number of builtin routines that help diagnose what's going on when an algorithm breaks. That's not to say other language don't, but the ability to set a single dbstop command and then run some kind of diagnostic like plotting the distribution of eigenvalues with d=eig(A); plot(real(d),imag(d),'x') is amazing and saves time. There's also a very straightforward workflow to run the debugger and then drop into gdb in case the case we need to interact with external libraries. Now, certainly, MATLAB/Octave is weak, in my opinion, for generating larger software projects that need to interact with the rest of the world. This includes things like network connections, GUIs, database access, etc. Alternatively, sometimes a new low level driver needs to be written, which needs to be very fast. All that same, that ecosystem seems to be much better in languages like C++ and Python. Though, I've been experimenting with Rust as an alternative to C++ for this use case. At this point, if I have trouble with the algorithms, I can run it in parallel with the MATLAB/Octave to diagnose how things differ. Coming back to Julia, where is it supposed to fit in? To me, there's a better prototyping language, production language, and bare metal fast language. I will make one last quip and that's the licensing. Frankly, the big advantage of MATLAB is that it provides license cover. Mathworks has obtained the appropriate licenses for your expensive factorizations and sparse matrix methodology. Julia has not and they remain largely under GPL: https://github.com/JuliaLang/julia/blob/master/LICENSE.md https://github.com/JuliaLang/julia/blob/master/LICENSE.md Look at things like SUITESPARSE. Practically, what that means is that I can deliver MATLAB code to my clients and I don't have to disclose the source externally due to GPL requirements. Now, maybe they choose to run Octave. That's fine and then they can assume the responsibility for GPL code. However, for me, I maintain a MATLAB license and that gives me coverage for a whole host of other licenses in the context of MATLAB code and that makes my life vastly easier than if I were to develop and deliver code in another language.

ChrisRackauckas 8y ago
MATLAB, Python, and R don't have the metaprogramming tools for code generation. Tools like FFTW relied heavily on code generation, and people who wrote those kinds of tools are now using Julia for generic programming. Generic programming is difficult to handle with any AOT compilation system since you get a combinatoric explosion of possibilities leading to large compile times. Generic programming works great in Julia so there's a lot of examples of utilizing types for "free features" that would be difficult to do (interactively) in any other language. Of course, most programmers don't know what generic programming even is. However, a lot of libraries these days utilize it in order to get different precision and all of that, which is why I think Julia will at least become a language of libraries since not too many people can invest so much time in C++.
- celrod 8y ago
  Steven G. Johnson, who wrote much of FFTW in OCaml, is now a major Julia contributor! Or OCaml and C? Wikipedia isn't especially clear on that.
  - ChrisRackauckas 8y ago
    It's OCaml which generates C.
- setr 8y ago
  >Generic programming is difficult to handle with any AOT compilation system since you get a combinatoric explosion of possibilities leading to large compile times. I thought it was either solved by adding function signatures based on actual usage, or “boxing” the argument (which I guess is just dynamic at that point) Why would you ever see the combinatoric explosion in an AOT implementation?
  - ChrisRackauckas 8y ago
    If you want to make it as a standard library so it can compile once and others link to it, then you have this issue. You need to compile on use in order to only add the function signatures based on use, so if you're doing this from types the user is giving you (say, a generic differential equation solver where the state can be any type with the right overloads) then you're out of luck. If you know of a big set of types the user might want to give you, then you can compile those ahead of time, but for the diffeq example you can have different types for the dependent and independent variable, so you'd want each combination. The solution here of course is just to make people statically compile against it if you want to use all of these features, but you need to compile as a shared library if you want to build an easy interface to a high level scripting language (Julia, Python, R, MATLAB, etc.). So what tends to happen is that libraries which do make some extra features possible are constrained when used from a scripting language and they compile a bunch of choices to mitigate that a bit (see SUNDIALS and its built-in NVector compilations). The true solution here of course is to compile on demand based on the new types you see from the user codes, and this is what Julia does automatically. As for boxing, boxing works but it's slow. The developer of LightGraphs.jl tried using Swift for awhile but went back to Julia when he realized that actually using generics in Swift is really really slow because they do this boxing (your code is generic! But...). To finish the story, he tried Go a bit but generics in Go... he went to C++ but templates but they were difficult to use and the compile times were too long to be anything close to interactive, and has done extremely well over the last few years in Julia (though it is admittedly missing some tools for (partial) AOT compilation and building binaries, but these tools are in development).
celrod 8y ago
The beauty of Julia is in its design as a language. This is what gives people hope that the tooling and ecosystem will emerge -- they all seem easier to develop in Julia than in other languages, thanks to multiple dispatch, the type system, and powerful metaprogramming abilities. As far as fast bare metal languages go, you can write extremely general optimized matrix multiplication libraries in Julia. For matrices that fit in the L2 cache, I achieved performance similar to Intel MKL, but with far more generic code. https://discourse.julialang.org/t/we-can-write-an-optimized-blas-library-in-pure-julia-please-skip-op-and-jump-to-post-4/11634/4?u=elrod https://discourse.julialang.org/t/we-can-write-an-optimized-... Writing a kernel in C means using SIMD intrinsics, where function names and types differ for every vector size, and then different numbers/sets of calls for each kernel (you'd want several sizes per architecture). Compare this to Julia, where parametric typing and multiple dispatch mean you only need one set of functions, and using an @generated function will generate whatever kernels you happen to need at compile time. Looking into the future, awesome projects like Cassete speak to the potential of what's possible: https://github.com/jrevels/Cassette.jl https://github.com/jrevels/Cassette.jl Cassete promises to let you do anything from getting automatic differentiation for arbitrary Julia code, even if it is strictly typed and buried in a chain of dependencies of the library you're using and written by someone who never imagined the idea of autodiff, to injecting custom compiler passes. Also, while I haven't tried it yet, many seem to like: https://github.com/timholy/Rebugger.jl https://github.com/timholy/Rebugger.jl Julia is more promise than practice right now, but that's largely because of just how much it promises. I (and many others) think it has done a great job delivering so far. That's why there's excitement, and why many are starting to embrace it.
- marmaduke 8y ago
  > Writing a kernel in C means using SIMD intrinsics, where function names and types differ for every vector size, and then different numbers/sets of calls for each kernel (you'd want several sizes per architecture) This is not true for any recent, competent compiler. I can write a template function in C++ code with a regular loop over arrays and it gets vectorized withSIMD instructions automatically according to architecture. Sure, it’s not JIT but it still never requires intrinsics
marmaduke 8y ago
The MATLAB compiler is another part of the ecosystem which is worth its price when trying to deliver quickly. There’s nothing like it in the other ecosystems (maybe Mathematica?)
- srean 8y ago
  Entire MATLAB toolchain is a ridiculously overpriced relic, the faster it disappears the better, and it will. To a close approximation the only folks who continue to use it are those who are insulated from the market -- national labs, and universities. It annoys me because at places its a ridiculous waste of tax payer's money.
  - kxyvr 8y ago
    No it's not. I've seen it used within petrochem and other engineering firms as well. For example, the real time simulator Opal-RT uses Simulink and Opals are used for things like simulating power systems. Although you may not like it, it serves a purpose and it's one tool among many.
    - 3rdAccount 8y ago
      I actually work in power systems simulations and am not aware of anyone using the power systems toolbox in Matlab outside of some academic researchers. I'm sure some folks do, but it isn't very big. This is all mainly traditional Fortran and increasingly C++ and even C. There are some open source things like MatPower (Matlab or Python), but it is mostly seen as a neat toy and not something I would trust to actually run grid studies such as an EMS vendor, PSS/E, PowerWorld, Powergem, PSCAD...etc etc etc. I've actually seen that some vendors are looking at Julia to replace some of their tools.
      - kxyvr 8y ago
        I'm curious then, what do they use for real time simulations? From what I see, some groups use Opal-RT: https://www.opal-rt.com/ https://www.opal-rt.com/ Some groups use Typhoon: https://www.typhoon-hil.com https://www.typhoon-hil.com Opal-RT is built on MATLAB/Simulink, which is how it came up. Anyway, what real time simulators do you use?
jules 8y ago
Julia combines rapid prototyping with a well designed language and good execution speed. Matlab only has the first of those three. Julia can plot eigenvalues no problem. Almost exactly the same code works.
edwintorok 8y ago
How about FriCAS and SPAD [1]? I find it odd that most of the widespread mathematical packages are not statically typed. [1] http://www.euclideanspace.com/prog/scratchpad/spad/index.htm http://www.euclideanspace.com/prog/scratchpad/spad/index.htm

by kxyvr 8y ago

ChrisRackauckas 8y ago

MATLAB, Python, and R don't have the metaprogramming tools for code generation. Tools like FFTW relied heavily on code generation, and people who wrote those kinds of tools are now using Julia for generic programming. Generic programming is difficult to handle with any AOT compilation system since you get a combinatoric explosion of possibilities leading to large compile times. Generic programming works great in Julia so there's a lot of examples of utilizing types for "free features" that would be difficult to do (interactively) in any other language. Of course, most programmers don't know what generic programming even is. However, a lot of libraries these days utilize it in order to get different precision and all of that, which is why I think Julia will at least become a language of libraries since not too many people can invest so much time in C++.

celrod 8y ago
Steven G. Johnson, who wrote much of FFTW in OCaml, is now a major Julia contributor! Or OCaml and C? Wikipedia isn't especially clear on that.
- ChrisRackauckas 8y ago
  It's OCaml which generates C.
setr 8y ago
>Generic programming is difficult to handle with any AOT compilation system since you get a combinatoric explosion of possibilities leading to large compile times. I thought it was either solved by adding function signatures based on actual usage, or “boxing” the argument (which I guess is just dynamic at that point) Why would you ever see the combinatoric explosion in an AOT implementation?
- ChrisRackauckas 8y ago
  If you want to make it as a standard library so it can compile once and others link to it, then you have this issue. You need to compile on use in order to only add the function signatures based on use, so if you're doing this from types the user is giving you (say, a generic differential equation solver where the state can be any type with the right overloads) then you're out of luck. If you know of a big set of types the user might want to give you, then you can compile those ahead of time, but for the diffeq example you can have different types for the dependent and independent variable, so you'd want each combination. The solution here of course is just to make people statically compile against it if you want to use all of these features, but you need to compile as a shared library if you want to build an easy interface to a high level scripting language (Julia, Python, R, MATLAB, etc.). So what tends to happen is that libraries which do make some extra features possible are constrained when used from a scripting language and they compile a bunch of choices to mitigate that a bit (see SUNDIALS and its built-in NVector compilations). The true solution here of course is to compile on demand based on the new types you see from the user codes, and this is what Julia does automatically. As for boxing, boxing works but it's slow. The developer of LightGraphs.jl tried using Swift for awhile but went back to Julia when he realized that actually using generics in Swift is really really slow because they do this boxing (your code is generic! But...). To finish the story, he tried Go a bit but generics in Go... he went to C++ but templates but they were difficult to use and the compile times were too long to be anything close to interactive, and has done extremely well over the last few years in Julia (though it is admittedly missing some tools for (partial) AOT compilation and building binaries, but these tools are in development).

celrod 8y ago

The beauty of Julia is in its design as a language. This is what gives people hope that the tooling and ecosystem will emerge -- they all seem easier to develop in Julia than in other languages, thanks to multiple dispatch, the type system, and powerful metaprogramming abilities. As far as fast bare metal languages go, you can write extremely general optimized matrix multiplication libraries in Julia. For matrices that fit in the L2 cache, I achieved performance similar to Intel MKL, but with far more generic code. https://discourse.julialang.org/t/we-can-write-an-optimized-blas-library-in-pure-julia-please-skip-op-and-jump-to-post-4/11634/4?u=elrod https://discourse.julialang.org/t/we-can-write-an-optimized-... Writing a kernel in C means using SIMD intrinsics, where function names and types differ for every vector size, and then different numbers/sets of calls for each kernel (you'd want several sizes per architecture). Compare this to Julia, where parametric typing and multiple dispatch mean you only need one set of functions, and using an @generated function will generate whatever kernels you happen to need at compile time. Looking into the future, awesome projects like Cassete speak to the potential of what's possible: https://github.com/jrevels/Cassette.jl https://github.com/jrevels/Cassette.jl Cassete promises to let you do anything from getting automatic differentiation for arbitrary Julia code, even if it is strictly typed and buried in a chain of dependencies of the library you're using and written by someone who never imagined the idea of autodiff, to injecting custom compiler passes. Also, while I haven't tried it yet, many seem to like: https://github.com/timholy/Rebugger.jl https://github.com/timholy/Rebugger.jl Julia is more promise than practice right now, but that's largely because of just how much it promises. I (and many others) think it has done a great job delivering so far. That's why there's excitement, and why many are starting to embrace it.

marmaduke 8y ago
> Writing a kernel in C means using SIMD intrinsics, where function names and types differ for every vector size, and then different numbers/sets of calls for each kernel (you'd want several sizes per architecture) This is not true for any recent, competent compiler. I can write a template function in C++ code with a regular loop over arrays and it gets vectorized withSIMD instructions automatically according to architecture. Sure, it’s not JIT but it still never requires intrinsics

marmaduke 8y ago

The MATLAB compiler is another part of the ecosystem which is worth its price when trying to deliver quickly. There’s nothing like it in the other ecosystems (maybe Mathematica?)

srean 8y ago
Entire MATLAB toolchain is a ridiculously overpriced relic, the faster it disappears the better, and it will. To a close approximation the only folks who continue to use it are those who are insulated from the market -- national labs, and universities. It annoys me because at places its a ridiculous waste of tax payer's money.
- kxyvr 8y ago
  No it's not. I've seen it used within petrochem and other engineering firms as well. For example, the real time simulator Opal-RT uses Simulink and Opals are used for things like simulating power systems. Although you may not like it, it serves a purpose and it's one tool among many.
  - 3rdAccount 8y ago
    I actually work in power systems simulations and am not aware of anyone using the power systems toolbox in Matlab outside of some academic researchers. I'm sure some folks do, but it isn't very big. This is all mainly traditional Fortran and increasingly C++ and even C. There are some open source things like MatPower (Matlab or Python), but it is mostly seen as a neat toy and not something I would trust to actually run grid studies such as an EMS vendor, PSS/E, PowerWorld, Powergem, PSCAD...etc etc etc. I've actually seen that some vendors are looking at Julia to replace some of their tools.
    - kxyvr 8y ago
      I'm curious then, what do they use for real time simulations? From what I see, some groups use Opal-RT: https://www.opal-rt.com/ https://www.opal-rt.com/ Some groups use Typhoon: https://www.typhoon-hil.com https://www.typhoon-hil.com Opal-RT is built on MATLAB/Simulink, which is how it came up. Anyway, what real time simulators do you use?

jules 8y ago

Julia combines rapid prototyping with a well designed language and good execution speed. Matlab only has the first of those three. Julia can plot eigenvalues no problem. Almost exactly the same code works.

edwintorok 8y ago

How about FriCAS and SPAD [1]? I find it odd that most of the widespread mathematical packages are not statically typed. [1] http://www.euclideanspace.com/prog/scratchpad/spad/index.htm http://www.euclideanspace.com/prog/scratchpad/spad/index.htm