12 ms·

I have high hopes for Julia becoming the defacto open-source scientific language. Despite Python and R both having a massive head start, I'm willing to bet that

I have high hopes for Julia becoming the defacto open-source scientific language. Despite Python and R both having a massive head start, I'm willing to bet that talented engineers and scientists will be drawn to Julia to implement their next-generation frameworks owing to the powerful features that it offers. For example, the fact that an array of unions such as Array{Union{Missing,T}} is represented in memory as a much more efficient union of arrays is a perfect example of where a clever compiler can make the logical thing to do also the efficient thing to do!

mlthoughts2018 8y ago
I am a machine learning library developer and I don’t share your feelings. For example the specific example you cite, I feel, should never be something scientists or engineers actively think about, only language implementers. Once you make that distinction, then whether you write it as a Cython module exposed in Python or you can use native language features to do it in Julia, nobody cares. It’s encapsulated away from people who use numerical libraries, as it should be. I also spend time developing these “runtime overhead avoidant” backend numerical libraries, and I would say I’ve seen no significant reason to prefer Julia over Cython. Don’t getme wrong, Julia is great, just not offering anything fundamentally different. And since there’s already a critical mass of people with engineering and optimization experience in the Cython & Python extension module stack, I’d expect that community to continue dominating Julia just by attrition alone.
- ChrisRackauckas 8y ago
  It's offering something completely different because of the compatibility, compile-time controls, and ability to fully interprocedurally optimize. http://www.stochasticlifestyle.com/why-numba-and-cython-are-not-substitutes-for-julia/ http://www.stochasticlifestyle.com/why-numba-and-cython-are-...
  - mlthoughts2018 8y ago
    The article you link is severely wrong about both numba and Cython. I frequently use Cython to quickly wrap calls to other C++ implementations of tools I want to try and have a working Python module in a matter of minutes, and I have almost no knowledge of C++. Modern numba can also do a lot more for huge scale projects than what the article suggests. Julia docs also seem very smugly proud of multiple dispatch and autogenerating implementations for multiple types or signatures. But Cython fused types allow the exact same polymorphic multiple dispatch. Here’s an example pedagogical project illustrating that point. < https://github.com/spearsem/buffersort https://github.com/spearsem/buffersort > This pattern is quite easy and offers a lot of generic strategies in Cython, especially if you just want a bunch of overload options in a pure C backend with a thin entrypoint to Python.
    - ChrisRackauckas 8y ago
      The points that you mentioned are not relevant to what's in the article. Cython fused types are required to be known at the package's compile time, and that's exactly the point that makes it less composible. Not requiring this is exactly the advantage of Julia which is demonstrated. The article doesn't say Numba cannot do huge scale projects. It just talks about the difference in the compilation strategies. If you read that and think that it means Numba isn't suited for huge scale projects, then that's your interpretation of how Numba works, not an opinion directly stated by the article. The article gives an easy way for you to demonstrate it wrong. You can show 10 lines of code where you send dual numbers and uncertainty objects through SciPy's ODE solvers to generate solutions with gradients and error bars. If Cython can do this, please show it.
      - mlthoughts2018 8y ago
        The first and second paragraphs under “Point 1” are specifically saying Cython & numba work just as well as Julia for single functions, then saying the problem where they are inferior to Julia is large code bases.
        ChrisRackauckas 8y ago
  - marmaduke 8y ago
    Your article raises several valid points but is a overly optimistic. I still don’t believe that you AD through arbitrary code to produce useful gradient estimates; I’ve seen that fail miserably time after time. I don’t think Julia’s generic code is the end game for fast code though. Efforts in Python toward code gen include projects like loopy, which targets accelerators, are still developing but make it easy to decouple kernels from loop domains, target specific devices and optimizations etc.
- myrryr 8y ago
  Library devs also need to actively think about memory layout if they are after crazy amounts of speed.
- babahoyo 8y ago
  Many scientists might have mathematical ideas about how an operation should be done, but dont want to learn C++ to implement them. We create a division between scientists and programmers that hurts productivity.
  - semi-extrinsic 8y ago
    Contrarian view: having a division between "general scientists" and "programmer scientists" is a good thing. I really don't miss the bad old days of scientists doing write-only code from a cobbled-together mess based on Numerical Recipes, leaving behind spaghetti C99 or F77 for the next post-doc, commenting out routines for version control; giving us papers with numerical analysis that's not even reproducible by the same group 12 months down the line. In fact I think it's a much better thing to have a sort of division where a general scientist can put together a simulation/analysis pipeline in Python or R, but if they want to implement new algorithms they'll be sufficiently out of their league that they'll need help from someone who has actually spent time learning how to code properly and efficiently, to do testing and version control etc. In fact it's very similar to how experimental science often works: you have the general scientist who just knows how to do basic measurements with some instrument, and you have the instrument scientist who has to help if the general scientist wants to use some novel technique, who keeps the equipment tidy and working and logs everything in the big lab journal.
    - StefanKarpinski 8y ago
      I can understand how one can reasonably hypothesize that. In some sense you can see Julia as one big social experiment in what happens when you do everything possible to blur the line between users and developers of a scientific (and, of course, also general) programming language by solving the two language problem and making the language for users and developers one and the same. ... and it's been wildly successful. Some of the most prolific and talented contributors to Julia and its ecosystem are scientists by day and brilliant programmers of all kinds—crazy meta programmers, compiler hackers, generational GC writers, etc.—by night (as they say). During the keynote last night before tagging 1.0, we went through the history of major features in Julia's development history, it became a running theme that so many of these contributions have come from people who are physicists, chemists, geologists, biologists, etc. So I'd say that Julia is strong evidence that breaking down the separation between scientific users and developers of scientific libraries is a good idea.
- marmaduke 8y ago
  I see your comment grayed out, and I just want to chime in, as some who does a lot of numerical stuff (more than a decade, published stuff, support multiple lab research projects etc), I want to second this point of view. When it’s time to get real work done Python is more than good enough, and there’s plenty of strategies for acceleration where required. And when I want Julia’s promise of fast loops, I use Numba. If all the effort gone into Julia had instead been spent on fixing remaining warts in Python workflow for science, we wouldn’t even havee this conversation.
  - celrod 8y ago
    I think it is easier to write fast, complicated code in Julia than in C, C++, or Fortran. Not just the syntax, but because of great support for generic code by default and metaprogramming making it relatively simple to write code to generate the code you actually want for any given scenario. Interactive benchmarking and profiling are a boon too. An example of the value of generic code is that forward mode AD is extremely easy, and almost always just works on whatever code you run it on. Then, once that's done, multiple dispatch (and possible macros for a DSL) allows for a much cleaner user interface than Python offers for numeric code. I have a lot more experience with R than Python, but seeing more of the scientist/mathematician/researchers side of things, I have to strongly disagree with the view that they should write slow code and contact a CS guy to write fast code in another language when they need it. Do you honestly think that's practical for grad student's projects? Recently, one of my friends wrote a simulation in R. Most of the work was done by hcubature -- a package written in C -- integrating a function written in R. Could just have easily been written in Python. That function was slow, and the simulation ran for days. Before an error caused it to crash, losing days of compute time. I -- a statistics grad student -- helped him rewrite it in Julia, and it finished in 2 hours. That C/C++ code will still run slowly if they have to call your R/Python code is a problem. They also can't apply things like AD easily. A common solution, used by Stan for example, is to create a whole new modeling language and have users interface through that. Learning a new language -- albeit relatively simple/domain specific -- which they then cannot debug interactively, is another pain point. All this can be avoided by simply using Julia.
    - 8y ago
- jernfrost 8y ago
  He mentioned implementors of frameworks specifically, not users of frameworks which you seem to talk about. Julia is superior to R and Python as a language for package development. This is illustrated by the fact that almost all big popular Python libraries are made in complicated C++. In Julia all the popular packages are native Julia, because it is a high performance language. It means it is much easier to get package contributors and feedback and help from package users. This is why Julia is moving forward so much faster than Python despite having much smaller mindshare.
Athas 8y ago
> For example, the fact that an array of unions such as Array{Union{Missing,T}} is represented in memory as a much more efficient union of arrays is a perfect example of where a clever compiler can make the logical thing to do also the efficient thing to do! Can you elaborate on how Julia represents arrays of unions, or point to some documentation? I'm working on something where an automatic efficient representation would be useful, and I'd like to learn from other's experience.
- iamed2 8y ago
  There's an overview here: https://docs.julialang.org/en/v1.0.0/devdocs/isbitsunionarrays/ https://docs.julialang.org/en/v1.0.0/devdocs/isbitsunionarra... In terms of the Julia changes that were needed to support this, there were two key PRs: https://github.com/JuliaLang/julia/pull/20593 https://github.com/JuliaLang/julia/pull/20593 https://github.com/JuliaLang/julia/pull/22441 https://github.com/JuliaLang/julia/pull/22441
  - Athas 8y ago
    Thanks, this looks about as I expected. The real problem occurs when you have an array of union types, where sum of the union cases are themselves way, which it does not look like Julia does anything clever about (because it's not even clear to me that it's even possible).

mlthoughts2018 8y ago

I am a machine learning library developer and I don’t share your feelings. For example the specific example you cite, I feel, should never be something scientists or engineers actively think about, only language implementers. Once you make that distinction, then whether you write it as a Cython module exposed in Python or you can use native language features to do it in Julia, nobody cares. It’s encapsulated away from people who use numerical libraries, as it should be. I also spend time developing these “runtime overhead avoidant” backend numerical libraries, and I would say I’ve seen no significant reason to prefer Julia over Cython. Don’t getme wrong, Julia is great, just not offering anything fundamentally different. And since there’s already a critical mass of people with engineering and optimization experience in the Cython & Python extension module stack, I’d expect that community to continue dominating Julia just by attrition alone.

ChrisRackauckas 8y ago
It's offering something completely different because of the compatibility, compile-time controls, and ability to fully interprocedurally optimize. http://www.stochasticlifestyle.com/why-numba-and-cython-are-not-substitutes-for-julia/ http://www.stochasticlifestyle.com/why-numba-and-cython-are-...
- mlthoughts2018 8y ago
  The article you link is severely wrong about both numba and Cython. I frequently use Cython to quickly wrap calls to other C++ implementations of tools I want to try and have a working Python module in a matter of minutes, and I have almost no knowledge of C++. Modern numba can also do a lot more for huge scale projects than what the article suggests. Julia docs also seem very smugly proud of multiple dispatch and autogenerating implementations for multiple types or signatures. But Cython fused types allow the exact same polymorphic multiple dispatch. Here’s an example pedagogical project illustrating that point. < https://github.com/spearsem/buffersort https://github.com/spearsem/buffersort > This pattern is quite easy and offers a lot of generic strategies in Cython, especially if you just want a bunch of overload options in a pure C backend with a thin entrypoint to Python.
  - ChrisRackauckas 8y ago
    The points that you mentioned are not relevant to what's in the article. Cython fused types are required to be known at the package's compile time, and that's exactly the point that makes it less composible. Not requiring this is exactly the advantage of Julia which is demonstrated. The article doesn't say Numba cannot do huge scale projects. It just talks about the difference in the compilation strategies. If you read that and think that it means Numba isn't suited for huge scale projects, then that's your interpretation of how Numba works, not an opinion directly stated by the article. The article gives an easy way for you to demonstrate it wrong. You can show 10 lines of code where you send dual numbers and uncertainty objects through SciPy's ODE solvers to generate solutions with gradients and error bars. If Cython can do this, please show it.
    - mlthoughts2018 8y ago
      The first and second paragraphs under “Point 1” are specifically saying Cython & numba work just as well as Julia for single functions, then saying the problem where they are inferior to Julia is large code bases.
      - ChrisRackauckas 8y ago
- marmaduke 8y ago
  Your article raises several valid points but is a overly optimistic. I still don’t believe that you AD through arbitrary code to produce useful gradient estimates; I’ve seen that fail miserably time after time. I don’t think Julia’s generic code is the end game for fast code though. Efforts in Python toward code gen include projects like loopy, which targets accelerators, are still developing but make it easy to decouple kernels from loop domains, target specific devices and optimizations etc.
myrryr 8y ago
Library devs also need to actively think about memory layout if they are after crazy amounts of speed.
babahoyo 8y ago
Many scientists might have mathematical ideas about how an operation should be done, but dont want to learn C++ to implement them. We create a division between scientists and programmers that hurts productivity.
- semi-extrinsic 8y ago
  Contrarian view: having a division between "general scientists" and "programmer scientists" is a good thing. I really don't miss the bad old days of scientists doing write-only code from a cobbled-together mess based on Numerical Recipes, leaving behind spaghetti C99 or F77 for the next post-doc, commenting out routines for version control; giving us papers with numerical analysis that's not even reproducible by the same group 12 months down the line. In fact I think it's a much better thing to have a sort of division where a general scientist can put together a simulation/analysis pipeline in Python or R, but if they want to implement new algorithms they'll be sufficiently out of their league that they'll need help from someone who has actually spent time learning how to code properly and efficiently, to do testing and version control etc. In fact it's very similar to how experimental science often works: you have the general scientist who just knows how to do basic measurements with some instrument, and you have the instrument scientist who has to help if the general scientist wants to use some novel technique, who keeps the equipment tidy and working and logs everything in the big lab journal.
  - StefanKarpinski 8y ago
    I can understand how one can reasonably hypothesize that. In some sense you can see Julia as one big social experiment in what happens when you do everything possible to blur the line between users and developers of a scientific (and, of course, also general) programming language by solving the two language problem and making the language for users and developers one and the same. ... and it's been wildly successful. Some of the most prolific and talented contributors to Julia and its ecosystem are scientists by day and brilliant programmers of all kinds—crazy meta programmers, compiler hackers, generational GC writers, etc.—by night (as they say). During the keynote last night before tagging 1.0, we went through the history of major features in Julia's development history, it became a running theme that so many of these contributions have come from people who are physicists, chemists, geologists, biologists, etc. So I'd say that Julia is strong evidence that breaking down the separation between scientific users and developers of scientific libraries is a good idea.
marmaduke 8y ago
I see your comment grayed out, and I just want to chime in, as some who does a lot of numerical stuff (more than a decade, published stuff, support multiple lab research projects etc), I want to second this point of view. When it’s time to get real work done Python is more than good enough, and there’s plenty of strategies for acceleration where required. And when I want Julia’s promise of fast loops, I use Numba. If all the effort gone into Julia had instead been spent on fixing remaining warts in Python workflow for science, we wouldn’t even havee this conversation.
- celrod 8y ago
  I think it is easier to write fast, complicated code in Julia than in C, C++, or Fortran. Not just the syntax, but because of great support for generic code by default and metaprogramming making it relatively simple to write code to generate the code you actually want for any given scenario. Interactive benchmarking and profiling are a boon too. An example of the value of generic code is that forward mode AD is extremely easy, and almost always just works on whatever code you run it on. Then, once that's done, multiple dispatch (and possible macros for a DSL) allows for a much cleaner user interface than Python offers for numeric code. I have a lot more experience with R than Python, but seeing more of the scientist/mathematician/researchers side of things, I have to strongly disagree with the view that they should write slow code and contact a CS guy to write fast code in another language when they need it. Do you honestly think that's practical for grad student's projects? Recently, one of my friends wrote a simulation in R. Most of the work was done by hcubature -- a package written in C -- integrating a function written in R. Could just have easily been written in Python. That function was slow, and the simulation ran for days. Before an error caused it to crash, losing days of compute time. I -- a statistics grad student -- helped him rewrite it in Julia, and it finished in 2 hours. That C/C++ code will still run slowly if they have to call your R/Python code is a problem. They also can't apply things like AD easily. A common solution, used by Stan for example, is to create a whole new modeling language and have users interface through that. Learning a new language -- albeit relatively simple/domain specific -- which they then cannot debug interactively, is another pain point. All this can be avoided by simply using Julia.
  - 8y ago
jernfrost 8y ago
He mentioned implementors of frameworks specifically, not users of frameworks which you seem to talk about. Julia is superior to R and Python as a language for package development. This is illustrated by the fact that almost all big popular Python libraries are made in complicated C++. In Julia all the popular packages are native Julia, because it is a high performance language. It means it is much easier to get package contributors and feedback and help from package users. This is why Julia is moving forward so much faster than Python despite having much smaller mindshare.

Athas 8y ago

> For example, the fact that an array of unions such as Array{Union{Missing,T}} is represented in memory as a much more efficient union of arrays is a perfect example of where a clever compiler can make the logical thing to do also the efficient thing to do! Can you elaborate on how Julia represents arrays of unions, or point to some documentation? I'm working on something where an automatic efficient representation would be useful, and I'd like to learn from other's experience.

iamed2 8y ago
There's an overview here: https://docs.julialang.org/en/v1.0.0/devdocs/isbitsunionarrays/ https://docs.julialang.org/en/v1.0.0/devdocs/isbitsunionarra... In terms of the Julia changes that were needed to support this, there were two key PRs: https://github.com/JuliaLang/julia/pull/20593 https://github.com/JuliaLang/julia/pull/20593 https://github.com/JuliaLang/julia/pull/22441 https://github.com/JuliaLang/julia/pull/22441
- Athas 8y ago
  Thanks, this looks about as I expected. The real problem occurs when you have an array of union types, where sum of the union cases are themselves way, which it does not look like Julia does anything clever about (because it's not even clear to me that it's even possible).