5 ms·

Scientific programming is heavily polymorphic. You just need to look at the interface to most well used libraries like BLAS. Fortran and C conventionally deal

Scientific programming is heavily polymorphic. You just need to look at the interface to most well used libraries like BLAS. Fortran and C conventionally deal with this by adding the relevant type signature to the function name. Scientific programming is also, regularly, polymorphic in 2, (often parametric) types. This heavily motivated the design choices in Julia esp the multiple dispatch infrastructure that is descended from Dylan.

reikonomusha 9y ago
Most scientific codes are in the end working on large arrays of floats. There are exceptions, but many scientific codes don’t at all resemble, say, general purpose template libraries. You can make them generic, but it’s not often you gain anything at all.
- lucozade 9y ago
  That's not true. At all. Almost all scientific libraries resemble template libraries but with bespoke, exposed name mangling and significant portions of the internal data structures specified in comments. They do this, not because it's inherently a good thing, but because the advantages of using Fortran or C have outweighed the disadvantages. But where that's not true, for example in end user computing, Fortran has not been popular for decades. What the Julia devs are attempting to do is to design a language that has the ergonomics of a Matlab or R to make scientific programmers productive whilst having the performance of a Fortran or C. They have a sophisticated type and dispatch system to help bring these goals together. Now, their current focus appears to be more on the R/Matlab replacement than on becoming the language of choice for HPC, say. So the steady state performance is unlikely to be competitive at scale as yet. Maybe they won't get there. Maybe the decisions they've made mean that it's effectively impossible to eek out those last few percent so we'll still have a 2 language model. I don't know but I applaud them for trying. More, I applaud the way they have gone about their development: focus on usage, actively learning from other languages, very little linguistic dogma. I hope they succeed.
  - one-more-minute 9y ago
    You should check out the Celeste project [1]. Julia is a more than viable option for HPC. https://juliacomputing.com/case-studies/celeste.html https://juliacomputing.com/case-studies/celeste.html
    - lucozade 9y ago
      I was aware, but thank you for the link. The fact that it can be used in an important use case is a necessary but not sufficient condition that it can replace the incumbents. Same goes for end user computing. Now, it gives me great confidence that, without the apparent short term focus of the core devs in this field, there has been such progress as Celeste. This is a good sign that the fundamentals are right. But I'm probably more encouraged that the community are focused on user experience and delivering an environment that scientists want to use. This is essential, IMO, and appears to be working well.
      - tavert 9y ago
        Short term focused work on Celeste, by at least one of the core devs, came at a very significant opportunity cost of not having an up-to-date, usable Julia debugger.
        chappi42 9y ago
        The Julia v0.5 debugger works. Why upgrade to v0.6 when the debugger is important? I'm sure the debugger will target v1.0. And maybe it was more fun (read interesting)to work on Celeste than to support short-lived intermediary versions?
- shele 9y ago
  A lot of Julia code is not only generic in the element type, but also with respect to the array type: a linear solver written in Julia works on classical matrices, but also on banded real matrices or sparse arrays of BigFloats.
- ChrisRackauckas 9y ago
  While I agree that things like arbitrary precision are niche, there's lots to gain in standard uses. Let me point out a few areas where that's already very clear: 1. Allowing arbitrary array types means your algorithm can already work on the GPU via GPUArrays.jl (https://github.com/JuliaGPU/GPUArrays.jl https://github.com/JuliaGPU/GPUArrays.jl). This means that all differential equation, optimization, numerical linear algebra, etc. routines which internally utilize the functions which GPUArrays implements will automatically compile a special version when encountering a GPUArray that will do all computations on the GPU without any data transfer back and forth. 2. Matrix types allow the user to specify algorithms for linear solving. Numerical linear algebra is an entire field based around coming up with good algorithms for solving linear systems (\), and the whole point of the field is to specialize on the type of matrix you have. With Julia, you can set it up so that way the user passes what type of matrix they have (Tridiagonal, sparse, PETSc for multi-node parallelism, or a special matrix type with \ overloaded to do multigrid) and the optimization, differential equation, etc. routine will use the fast linear solver routine specific to the problem. This makes it easy to expose very large amounts of performance gains and specialization opportunities to the user in ways that are usually automatic (i.e. sometimes that user doesn't even have to know!). 3. Complex numbers. Large fields of scientific computing (physicists and engineers) use complex numbers. Many libraries have pitiful support for complex numbers. Generic typing helps a lot here. Those are 3 off the top of my head, and there are many more diffeq-specific examples that have come up as users have asked to have resizing control models with discrete variables mixed in with continuous variables, etc., and this can all be handled efficiently via the type system without having to restructure the core algorithms to handle these extra features. The key thing here is that users can add new features specific to their problem into your algorithm as they need to, without having to modify your code, just by smart uses of the type system.

reikonomusha 9y ago

Most scientific codes are in the end working on large arrays of floats. There are exceptions, but many scientific codes don’t at all resemble, say, general purpose template libraries. You can make them generic, but it’s not often you gain anything at all.

lucozade 9y ago
That's not true. At all. Almost all scientific libraries resemble template libraries but with bespoke, exposed name mangling and significant portions of the internal data structures specified in comments. They do this, not because it's inherently a good thing, but because the advantages of using Fortran or C have outweighed the disadvantages. But where that's not true, for example in end user computing, Fortran has not been popular for decades. What the Julia devs are attempting to do is to design a language that has the ergonomics of a Matlab or R to make scientific programmers productive whilst having the performance of a Fortran or C. They have a sophisticated type and dispatch system to help bring these goals together. Now, their current focus appears to be more on the R/Matlab replacement than on becoming the language of choice for HPC, say. So the steady state performance is unlikely to be competitive at scale as yet. Maybe they won't get there. Maybe the decisions they've made mean that it's effectively impossible to eek out those last few percent so we'll still have a 2 language model. I don't know but I applaud them for trying. More, I applaud the way they have gone about their development: focus on usage, actively learning from other languages, very little linguistic dogma. I hope they succeed.
- one-more-minute 9y ago
  You should check out the Celeste project [1]. Julia is a more than viable option for HPC. https://juliacomputing.com/case-studies/celeste.html https://juliacomputing.com/case-studies/celeste.html
  - lucozade 9y ago
    I was aware, but thank you for the link. The fact that it can be used in an important use case is a necessary but not sufficient condition that it can replace the incumbents. Same goes for end user computing. Now, it gives me great confidence that, without the apparent short term focus of the core devs in this field, there has been such progress as Celeste. This is a good sign that the fundamentals are right. But I'm probably more encouraged that the community are focused on user experience and delivering an environment that scientists want to use. This is essential, IMO, and appears to be working well.
    - tavert 9y ago
      Short term focused work on Celeste, by at least one of the core devs, came at a very significant opportunity cost of not having an up-to-date, usable Julia debugger.
      - chappi42 9y ago
        The Julia v0.5 debugger works. Why upgrade to v0.6 when the debugger is important? I'm sure the debugger will target v1.0. And maybe it was more fun (read interesting)to work on Celeste than to support short-lived intermediary versions?
shele 9y ago
A lot of Julia code is not only generic in the element type, but also with respect to the array type: a linear solver written in Julia works on classical matrices, but also on banded real matrices or sparse arrays of BigFloats.
ChrisRackauckas 9y ago
While I agree that things like arbitrary precision are niche, there's lots to gain in standard uses. Let me point out a few areas where that's already very clear: 1. Allowing arbitrary array types means your algorithm can already work on the GPU via GPUArrays.jl (https://github.com/JuliaGPU/GPUArrays.jl https://github.com/JuliaGPU/GPUArrays.jl). This means that all differential equation, optimization, numerical linear algebra, etc. routines which internally utilize the functions which GPUArrays implements will automatically compile a special version when encountering a GPUArray that will do all computations on the GPU without any data transfer back and forth. 2. Matrix types allow the user to specify algorithms for linear solving. Numerical linear algebra is an entire field based around coming up with good algorithms for solving linear systems (\), and the whole point of the field is to specialize on the type of matrix you have. With Julia, you can set it up so that way the user passes what type of matrix they have (Tridiagonal, sparse, PETSc for multi-node parallelism, or a special matrix type with \ overloaded to do multigrid) and the optimization, differential equation, etc. routine will use the fast linear solver routine specific to the problem. This makes it easy to expose very large amounts of performance gains and specialization opportunities to the user in ways that are usually automatic (i.e. sometimes that user doesn't even have to know!). 3. Complex numbers. Large fields of scientific computing (physicists and engineers) use complex numbers. Many libraries have pitiful support for complex numbers. Generic typing helps a lot here. Those are 3 off the top of my head, and there are many more diffeq-specific examples that have come up as users have asked to have resizing control models with discrete variables mixed in with continuous variables, etc., and this can all be handled efficiently via the type system without having to restructure the core algorithms to handle these extra features. The key thing here is that users can add new features specific to their problem into your algorithm as they need to, without having to modify your code, just by smart uses of the type system.