4 ms·

If my scenario holds, at some point Swift's scientific computing ecosystem will rival and overtake Julia's. I don't see the ML ecosystem developing in isolati

If my scenario holds, at some point Swift's scientific computing ecosystem will rival and overtake Julia's. I don't see the ML ecosystem developing in isolation because there's going to be overlap, especially as more and more code can be differentiated.

ddragon 7y ago
I mean, I can't really guess the future, but I kind of feel it's not a simple case of "if you build they'll come" here. Python has a dominance over ML (and over the academy in general) that is beyond any current language, but it didn't bring it's statistics ecosystem to the level of R (and statistics is much closer to ML than pure math and physics). That's because the ML ecosystem does work very well in isolation (it's mostly a black box regressor), and the natural path of evolution is not in different way of processing things, but ways of preprocessing things (for example, how to capture and treat images and sounds before applying the models). And in this marathon Julia is a very late runner, and Swift didn't even start properly running in this direction.
- dklend122 7y ago
  I think google will invest in the scientific ecosystem since they've shown some interest for diff programming using scientific models, and these are becoming more mainstream, even in pytorch. The ML stuff is just a start.
eigenspace 7y ago
Who is going to make the scientific ecosystem? Julia and Python's scientific ecosystems are so strong precicesly because they get domain experts in those ecosystems to write the software they need for their niche. Machine learning programmers aren't about remake DifferentialEquations.jl or scipy in Swift. I've yet to meet a single scientist from a field outside of machine learning who was seriously excited for swift. This sort of machinery is hard to make and takes deep expertise, I really doubt it'll be made in Swift any time soon. Does swift even have plotting libraries yet? Swift has a good automatic differentiation story, mostly because it is very focused on machine learning use-cases, has corprate backing and all efforts are on one implementation. However, having only one automatic differentiation implementation has drawbacks. It won't be suitable for everyone. Julia on the other hand has a gigantic basket of different automatic differentiation tools all of which have strengths and weaknesses. This allows people to choose the right tool for the job and explore a very wide design space, allowing us to find which approaches work best for different circumstances. Our AD machinery is still evolving and definitely has problems, but progress has been fast and really encouraging. Even if Swift becomes the next Python and eats scientific computing, I strongly doubt this will seriously hamper Julia's community. We've been doing great living in Python's shadow. Julia doesn't need to be the most popular language in the world to be useful or successful.
- dklend122 7y ago
  > Who is going to make the scientific ecosystem? Google and apple. Apple already is working on a swift-numerics package. Look at TF python and jax. They've re-implemented chunks of scipy and numpy twice, hired people to work on plotting (altair) etc And that's with python. Their engineering time will go much further with swift, obviously.
  - eigenspace 7y ago
    Google and Apple are not going to make a full on scientific ecosystem because they don't have the domain experts or the motive. Numpy is not the same thing as scipy. DifferentialEquations.jl in julia is a great example of what it actually takes to make a real, competitive differentiation equation library. The sort of stuff that was built there requires a deep connection to the scientific and mathematics literature. Cash won't cut it. Another great example that'll resonate with physicists at least is things like ITensors.jl https://github.com/ITensor/ITensors.jl https://github.com/ITensor/ITensors.jl. Apple and Google are not going to make something like that.
    - dklend122 7y ago
      It's not JUST google and apple, I'm sure they have enough cash and expertise and will to create enough momentum to attract more domain experts in other areas. Especially once google brain and deepmind start working on things more complex than stacking layers, which is happening now. In particular, do you have another example aside from DifferentialEquations.jl ? Neural ODEs are hot enough that something like that could easily pop up in swift. ITensors is interesting, but that's only one.
      - eigenspace 7y ago
        > It's not JUST google and apple, I'm sure they have enough cash and expertise and will to create enough momentum to attract more domain experts in other areas. Maybe, but I'm doubtful. Scientific domain experts flock to languages like Python, Julia, Matlab, R, etc. because they're interactive and allow them to quickly iterate on ideas, query data, produce plots, etc. Swift is not much of an interactive language and is not built around that kind of repl driven experience. > In particular, do you have another example aside from DifferentialEquations.jl ? Sure, here's a smattering of high quality packages made by and for research scientists: https://github.com/JuliaApproximation/ApproxFun.jl https://github.com/BioJulia https://github.com/JuliaDiffEq/ModelingToolkit.jl https://github.com/crstnbr/MonteCarlo.jl https://github.com/chriselrod/LoopVectorization.jl https://github.com/JuliaNLSolvers/Optim.jl https://github.com/PainterQubits/Unitful.jl https://github.com/mcabbott/TensorCast.jl https://github.com/JuliaPhysics/Measurements.jl https://github.com/Jutho/TensorOperations.jl There are many many more, these are just the first that came to mind.

ddragon 7y ago

I mean, I can't really guess the future, but I kind of feel it's not a simple case of "if you build they'll come" here. Python has a dominance over ML (and over the academy in general) that is beyond any current language, but it didn't bring it's statistics ecosystem to the level of R (and statistics is much closer to ML than pure math and physics). That's because the ML ecosystem does work very well in isolation (it's mostly a black box regressor), and the natural path of evolution is not in different way of processing things, but ways of preprocessing things (for example, how to capture and treat images and sounds before applying the models). And in this marathon Julia is a very late runner, and Swift didn't even start properly running in this direction.

dklend122 7y ago
I think google will invest in the scientific ecosystem since they've shown some interest for diff programming using scientific models, and these are becoming more mainstream, even in pytorch. The ML stuff is just a start.

eigenspace 7y ago

Who is going to make the scientific ecosystem? Julia and Python's scientific ecosystems are so strong precicesly because they get domain experts in those ecosystems to write the software they need for their niche. Machine learning programmers aren't about remake DifferentialEquations.jl or scipy in Swift. I've yet to meet a single scientist from a field outside of machine learning who was seriously excited for swift. This sort of machinery is hard to make and takes deep expertise, I really doubt it'll be made in Swift any time soon. Does swift even have plotting libraries yet? Swift has a good automatic differentiation story, mostly because it is very focused on machine learning use-cases, has corprate backing and all efforts are on one implementation. However, having only one automatic differentiation implementation has drawbacks. It won't be suitable for everyone. Julia on the other hand has a gigantic basket of different automatic differentiation tools all of which have strengths and weaknesses. This allows people to choose the right tool for the job and explore a very wide design space, allowing us to find which approaches work best for different circumstances. Our AD machinery is still evolving and definitely has problems, but progress has been fast and really encouraging. Even if Swift becomes the next Python and eats scientific computing, I strongly doubt this will seriously hamper Julia's community. We've been doing great living in Python's shadow. Julia doesn't need to be the most popular language in the world to be useful or successful.

dklend122 7y ago
> Who is going to make the scientific ecosystem? Google and apple. Apple already is working on a swift-numerics package. Look at TF python and jax. They've re-implemented chunks of scipy and numpy twice, hired people to work on plotting (altair) etc And that's with python. Their engineering time will go much further with swift, obviously.
- eigenspace 7y ago
  Google and Apple are not going to make a full on scientific ecosystem because they don't have the domain experts or the motive. Numpy is not the same thing as scipy. DifferentialEquations.jl in julia is a great example of what it actually takes to make a real, competitive differentiation equation library. The sort of stuff that was built there requires a deep connection to the scientific and mathematics literature. Cash won't cut it. Another great example that'll resonate with physicists at least is things like ITensors.jl https://github.com/ITensor/ITensors.jl https://github.com/ITensor/ITensors.jl. Apple and Google are not going to make something like that.
  - dklend122 7y ago
    It's not JUST google and apple, I'm sure they have enough cash and expertise and will to create enough momentum to attract more domain experts in other areas. Especially once google brain and deepmind start working on things more complex than stacking layers, which is happening now. In particular, do you have another example aside from DifferentialEquations.jl ? Neural ODEs are hot enough that something like that could easily pop up in swift. ITensors is interesting, but that's only one.
    - eigenspace 7y ago
      > It's not JUST google and apple, I'm sure they have enough cash and expertise and will to create enough momentum to attract more domain experts in other areas. Maybe, but I'm doubtful. Scientific domain experts flock to languages like Python, Julia, Matlab, R, etc. because they're interactive and allow them to quickly iterate on ideas, query data, produce plots, etc. Swift is not much of an interactive language and is not built around that kind of repl driven experience. > In particular, do you have another example aside from DifferentialEquations.jl ? Sure, here's a smattering of high quality packages made by and for research scientists: https://github.com/JuliaApproximation/ApproxFun.jl https://github.com/BioJulia https://github.com/JuliaDiffEq/ModelingToolkit.jl https://github.com/crstnbr/MonteCarlo.jl https://github.com/chriselrod/LoopVectorization.jl https://github.com/JuliaNLSolvers/Optim.jl https://github.com/PainterQubits/Unitful.jl https://github.com/mcabbott/TensorCast.jl https://github.com/JuliaPhysics/Measurements.jl https://github.com/Jutho/TensorOperations.jl There are many many more, these are just the first that came to mind.