4 ms·

As someone who really only does numerical computing, and mostly with Julia, I'm curious. Could Julia use some help from PL and systems people to be noticably be

by vcxy 5y ago

As someone who really only does numerical computing, and mostly with Julia, I'm curious. Could Julia use some help from PL and systems people to be noticably better for people like me? Or would you guess it's already on an ideal trajectory if numerical science stuff is all I plan to use it for? I know their goal is to really be an all around good general language, so what you said makes sense for that goal regardless. I'm just curious if you have insight into gains that the numerics people like me could get that we might not know enough to know we're missing.

miguelraz 5y ago
What is the focus on numerical computing that you think Julia is missing? The ecosystem seems plenty healthy to me. (Maybe PDEs... but those are so close!) It's not uncommon for good Julia codes to smoke Fortran codebases, like the DiffEq verse or with staged programming, see Steven G. Johnson's keynote at JuliaCon 2019: https://www.youtube.com/watch?v=mSgXWpvQEHE https://www.youtube.com/watch?v=mSgXWpvQEHE
neolog 5y ago
Julia is great for numerical computing, but it's not as good as it can be, even in that area. Widening the community's base of expertise will help. I'll just list a few things that come to mind. Many Base functions are currently untyped, which means it's hard to abstract out the key properties that enable reuse, compiler optimizations, and parallelism. See [0] for a language expert's take on numerical computing, which involves defining traits like associativity that enable automatic parallelism by default. A stronger culture of functional programming could make your code faster and easier to understand. It's often (though not always) easier for tooling to optimize pure functions over immutable data structures (and arrays). Arrays are currently mutable, and there is too much emphasis on mutating functions. The iteration protocol can be made more memory-efficient for large collections and simpler, following Rust's implementation [1]. Macros are useful for high-performance computing [4], and Julia's macros can be made more composable [2]. In general, systems languages don't make language decisions lightly. They have committees, discuss how other languages do things, make proposals. This allows more perspectives on each decision. That would be an improvement over the more ad-hoc style of Julia development, as long as Julia can avoid adding every possible feature, which is a risk of expanding the decision-making body [3]. [0] https://www.youtube.com/watch?v=EZD3Scuv02g https://www.youtube.com/watch?v=EZD3Scuv02g [1] https://mikeinnes.github.io/2020/06/04/iterate.html https://mikeinnes.github.io/2020/06/04/iterate.html [2] https://github.com/JuliaLang/julia/issues/37691 https://github.com/JuliaLang/julia/issues/37691 [3] https://www.stroustrup.com/P0977-remember-the-vasa.pdf https://www.stroustrup.com/P0977-remember-the-vasa.pdf [4] https://www.youtube.com/watch?v=mSgXWpvQEHE https://www.youtube.com/watch?v=mSgXWpvQEHE
- Certhas 5y ago
  I think you are roughly right that more expertise in this area would be beneficial, but you also don't seem to understand what the core strengths and goals/short comings of Julia are. I would love to see more engagement, but if it comes in the style of "you don't know what you're doing" then it wont work, especially if it's in contrast to practitioners experience in using the language. For example base functions being untyped is exactly how good Julia library code must look. Julia wants your code to be generic. It will get specialized (and compiled) when called with concrete types. The type system is not designed to encode invariants about the library, but to pass through information from the call site to the compiler when encountering inner functions. From a design perspective it is duck typed. A fundamental difference to compiled languages is that the compiler doesn't need to reason about the types ahead of time because it only gets triggered when the function is called, at which time the concrete type information is available. From my perspective this interaction of parametric type system, JAOT compilation, and multiple dispatch/ubiquitous generic code looks like they are interacting in a way that is genuinely new and exciting. At least I don't know any language that does something similar. One way to think of it is that Julia only has template functions, with many the draw backs and problems that entails. C++ 20 introduced Concepts to improve this aspects of the language, and I really believe Julia is in need of something along those lines, but this is more for humans than for the compiler. And obviously Julia is doing very well when it comes to enabling reuse and composability for example. I mean, I can throw a Neural Network into a Differential Equation, solve it using a state of the art solver, differentiate through the whole thing to do a gradient descent, and run all that on the GPU or CPU with the same code. So it's pretty absurd to claim that the duck typing in the base library is a problem for reuse or parallelism. BTW we also prototyped our problem space with Fortran and Julia, and Julia actually ended up faster than the Fortran implementation for the same algorithms. So compiler optimization also is not constrained in this way. Finally, Rust is a language built on the principle to not look at the cutting edge of PL research but instead to look at established things and implement them in a sound and relatively conservative way.
  - neolog 5y ago
    > ... practitioners experience in using the language I use and like the language. > For example base functions being untyped is exactly how good Julia library code must look. Julia wants your code to be generic. I don't think there needs to be a conflict between being typed and being generic, especially when traits are available. There has been a lot of advancement in flexible type systems and ad-hoc polymorphism in the last 20 years. For example, there's a lot we can say about the type of `map` or `filter`, but none of that information is specified in Julia.

by vcxy 5y ago

miguelraz 5y ago

What is the focus on numerical computing that you think Julia is missing? The ecosystem seems plenty healthy to me. (Maybe PDEs... but those are so close!) It's not uncommon for good Julia codes to smoke Fortran codebases, like the DiffEq verse or with staged programming, see Steven G. Johnson's keynote at JuliaCon 2019: https://www.youtube.com/watch?v=mSgXWpvQEHE https://www.youtube.com/watch?v=mSgXWpvQEHE

neolog 5y ago

Julia is great for numerical computing, but it's not as good as it can be, even in that area. Widening the community's base of expertise will help. I'll just list a few things that come to mind. Many Base functions are currently untyped, which means it's hard to abstract out the key properties that enable reuse, compiler optimizations, and parallelism. See [0] for a language expert's take on numerical computing, which involves defining traits like associativity that enable automatic parallelism by default. A stronger culture of functional programming could make your code faster and easier to understand. It's often (though not always) easier for tooling to optimize pure functions over immutable data structures (and arrays). Arrays are currently mutable, and there is too much emphasis on mutating functions. The iteration protocol can be made more memory-efficient for large collections and simpler, following Rust's implementation [1]. Macros are useful for high-performance computing [4], and Julia's macros can be made more composable [2]. In general, systems languages don't make language decisions lightly. They have committees, discuss how other languages do things, make proposals. This allows more perspectives on each decision. That would be an improvement over the more ad-hoc style of Julia development, as long as Julia can avoid adding every possible feature, which is a risk of expanding the decision-making body [3]. [0] https://www.youtube.com/watch?v=EZD3Scuv02g https://www.youtube.com/watch?v=EZD3Scuv02g [1] https://mikeinnes.github.io/2020/06/04/iterate.html https://mikeinnes.github.io/2020/06/04/iterate.html [2] https://github.com/JuliaLang/julia/issues/37691 https://github.com/JuliaLang/julia/issues/37691 [3] https://www.stroustrup.com/P0977-remember-the-vasa.pdf https://www.stroustrup.com/P0977-remember-the-vasa.pdf [4] https://www.youtube.com/watch?v=mSgXWpvQEHE https://www.youtube.com/watch?v=mSgXWpvQEHE

Certhas 5y ago
I think you are roughly right that more expertise in this area would be beneficial, but you also don't seem to understand what the core strengths and goals/short comings of Julia are. I would love to see more engagement, but if it comes in the style of "you don't know what you're doing" then it wont work, especially if it's in contrast to practitioners experience in using the language. For example base functions being untyped is exactly how good Julia library code must look. Julia wants your code to be generic. It will get specialized (and compiled) when called with concrete types. The type system is not designed to encode invariants about the library, but to pass through information from the call site to the compiler when encountering inner functions. From a design perspective it is duck typed. A fundamental difference to compiled languages is that the compiler doesn't need to reason about the types ahead of time because it only gets triggered when the function is called, at which time the concrete type information is available. From my perspective this interaction of parametric type system, JAOT compilation, and multiple dispatch/ubiquitous generic code looks like they are interacting in a way that is genuinely new and exciting. At least I don't know any language that does something similar. One way to think of it is that Julia only has template functions, with many the draw backs and problems that entails. C++ 20 introduced Concepts to improve this aspects of the language, and I really believe Julia is in need of something along those lines, but this is more for humans than for the compiler. And obviously Julia is doing very well when it comes to enabling reuse and composability for example. I mean, I can throw a Neural Network into a Differential Equation, solve it using a state of the art solver, differentiate through the whole thing to do a gradient descent, and run all that on the GPU or CPU with the same code. So it's pretty absurd to claim that the duck typing in the base library is a problem for reuse or parallelism. BTW we also prototyped our problem space with Fortran and Julia, and Julia actually ended up faster than the Fortran implementation for the same algorithms. So compiler optimization also is not constrained in this way. Finally, Rust is a language built on the principle to not look at the cutting edge of PL research but instead to look at established things and implement them in a sound and relatively conservative way.
- neolog 5y ago
  > ... practitioners experience in using the language I use and like the language. > For example base functions being untyped is exactly how good Julia library code must look. Julia wants your code to be generic. I don't think there needs to be a conflict between being typed and being generic, especially when traits are available. There has been a lot of advancement in flexible type systems and ad-hoc polymorphism in the last 20 years. For example, there's a lot we can say about the type of `map` or `filter`, but none of that information is specified in Julia.