3 ms·

I'd like to use Julia for audio and image processing. The language and library support is well suited for my applications. However, the current state of the P

I'd like to use Julia for audio and image processing. The language and library support is well suited for my applications. However, the current state of the PackageManager is reason for Julia to be a non-starter for me. I looked at the linked release notes with hope that PackageManager improvements would be slated for 1.9. The startup time for standalone executables is unacceptably large and baffling given the computing efforts required to compile and package them. I reached for Julia to improve upon Python and the start up costs far exceed the available system time for gains. The mere existence of the acronym TTFP indicates that the Julia community has neither an appropriate frame of reference nor prioritization of developing Julia into a truly general programming language -- Julia remains far from production ready. Computing generality goes far beyond the habitual use of: input, analysis, plot. Swift and Rust ought to be benchmarks for executable startup. From my perspective, Julia runtimes ought to be able to be constructed and frozen in dynamic shared libraries such that startup can approach that of Swift and Rust. Julia developers perhaps can look to GraalVM for inspiration. A long wait for Julia 2.0, particularly if standalone executable compilation would be addressed, can not possibly help the adoption of Julia from my perspective.

ChrisRackauckas 4y ago
> A long wait for Julia 2.0, particularly if standalone executable compilation would be addressed, can not possibly help the adoption of Julia from my perspective. It's the opposite. Julia 2.0 would not bring standalone executable compilation, but instead would require compiler work for language changes. The whole point of saying "we don't need to do a 2.0 right now" is that folks on the compiler team believe working on things like standalone compilation and improved caching of LLVM code (i.e. fixing TTFP) are a better use of time than changes to language semantics (i.e. a 2.0).
- waffletower 4y ago
  If PackageManager changes can be addressed in 1.10, 1.11+ etc., I would be elated. Again, TTFP does not need to 'addressed' -- the acronym instead needs to be eliminated from usage -- a shift to use of 'startup-time' can help a change of mindset to open the Julia community up to a broader range of applications and adopters. Otherwise, Julia simply continues to feed its currently narrow niche of applications.
  - ChrisRackauckas 4y ago
    I think by PackageManager here you mean package compiler, and yes these improvements do not need a 2.0. v1.8 included a few things to in the near future allow for building binaries without big dependencies like LLVM, and finishing this work is indeed slated for the v1.x releases. Saying "we are not doing a 2.0" is precisely saying that this is more important than things which change the user-facing language semantics. And TTFP does need to be addressed. It's a current shortcoming of the compiler that native and LLVM code is not cached during the precompilation stages. If such code is able to precompile into binaries, then startup time would be dramatically decreased because then a lot of package code would no longer have to JIT compile. Tim Holy and Valentin Churavy gave a nice talk at JuliaCon 2022 about the current progress of making this work: https://www.youtube.com/watch?v=GnsONc9DYg0 https://www.youtube.com/watch?v=GnsONc9DYg0 . This is all tied up with startup time and are all in some sense the same issue. Currently, the only way to get LLVM code cached, and thus startup time essentially eliminated, is to build it into what's called the "system image". That system image is the binary that package compiler builds (https://github.com/JuliaLang/PackageCompiler.jl https://github.com/JuliaLang/PackageCompiler.jl). Julia then ships with a default system image that includes the standard library in order to remove the major chunk of code that "most" libraries share, which is why all of Julia Base works without JIT lag. However, that means everyone wants to have their thing, be it sparse matrices to statistics, in the standard library so that it gets the JIT-lag free build by default. This means the system image is huge, which is why PackageCompiler, which is simply a system for building binaries by appending package code to the system image, builds big binaries. What needs to happen is for packages to be able to precompile in a way that then caches LLVM and native code. Then there's no major compile time advantage to being in the system image, which will allow things to be pulled out of the system image to have a leaner Julia Base build without major drawbacks, which would then help make the system compile. That will then make it so that an LLVM and BLAS build does not have to be in every binary (which is what takes up most of the space and RAM), which would then allow Julia to much more comfortably move beyond the niche of scientific computing.
    - waffletower 4y ago
      'startup-time' vs. 'TTFP' are just semantics, but a crucial distinction. The focus on 'time-to-first-plot' is entrenched in the dominant workflow of the Julia community. If you want a broader range of adoption, the word choice matters. Having experienced a very similar dialogue in the Clojure community over startup time, and seen the new range of applications emerge as the issue was effectively addressed, I was startled by the very narrow and focussed acronym (TTFP) in the Julia community. I am happy that there is progress on the code caching front and will look to the link you shared. Thanks for elucidating this further!
joshjob42 4y ago
Can I ask what about your use case would involve starting Julia and importing packages over and over? At least for my workloads, I often write a small .jl script that imports everything I want and then runs some functions on a list of files, for instance, or I just do my work in a Jupyter notebook, and I pay the load cost once for hours or days of computation.. Annuitized over the entire time I might use the kernel the startup time is basically nothing. I'm not saying the load time should be a lot lot faster! That would be awesome! But I see folks complain about this and I'm wondering if they really need to be booting a new kernel constantly or if they could adapt how they work a little bit and get their desired result. (For instance, by running a .jl script instead of a bash script that calls Julia or something.)
- waffletower 4y ago
  Currently, the adaptation cost is too high. The ecosystem that I work in has many small, distributed, fast-startup applications. While I could create a fairly complex client server application written in Julia to work with its limitations, and write compact fast startup applications to communicate with it, the complexity and time investment is undesirable. Again, to use Julia here one must bend to the ecosystem's weaknesses. I think the community truly doesn't understand what a general purpose language is. If you compile with LLVM startup really ought to be a non-issue. Look at the startup of GraalVM, and they began with Java bytecode.
  - joshjob42 4y ago
    Ah okay well yes I see how that might be an issue. I basically do everything in Julia, including calling out to shell commands, running scripts in other languages, Python packages, etc. etc. It's effectively my shell. I think the problem is that there needs to be a way to store the compiled code to disk and reimport it, which I think there are some plans toward that path. There were some developments discussed [here](https://www.youtube.com/watch?v=GnsONc9DYg0 https://www.youtube.com/watch?v=GnsONc9DYg0) to help with this, but the holy grail of fully compiled function caching across Julia kernels is not there yet.
- jakobnissen 4y ago

ChrisRackauckas 4y ago

> A long wait for Julia 2.0, particularly if standalone executable compilation would be addressed, can not possibly help the adoption of Julia from my perspective. It's the opposite. Julia 2.0 would not bring standalone executable compilation, but instead would require compiler work for language changes. The whole point of saying "we don't need to do a 2.0 right now" is that folks on the compiler team believe working on things like standalone compilation and improved caching of LLVM code (i.e. fixing TTFP) are a better use of time than changes to language semantics (i.e. a 2.0).

waffletower 4y ago
If PackageManager changes can be addressed in 1.10, 1.11+ etc., I would be elated. Again, TTFP does not need to 'addressed' -- the acronym instead needs to be eliminated from usage -- a shift to use of 'startup-time' can help a change of mindset to open the Julia community up to a broader range of applications and adopters. Otherwise, Julia simply continues to feed its currently narrow niche of applications.
- ChrisRackauckas 4y ago
  I think by PackageManager here you mean package compiler, and yes these improvements do not need a 2.0. v1.8 included a few things to in the near future allow for building binaries without big dependencies like LLVM, and finishing this work is indeed slated for the v1.x releases. Saying "we are not doing a 2.0" is precisely saying that this is more important than things which change the user-facing language semantics. And TTFP does need to be addressed. It's a current shortcoming of the compiler that native and LLVM code is not cached during the precompilation stages. If such code is able to precompile into binaries, then startup time would be dramatically decreased because then a lot of package code would no longer have to JIT compile. Tim Holy and Valentin Churavy gave a nice talk at JuliaCon 2022 about the current progress of making this work: https://www.youtube.com/watch?v=GnsONc9DYg0 https://www.youtube.com/watch?v=GnsONc9DYg0 . This is all tied up with startup time and are all in some sense the same issue. Currently, the only way to get LLVM code cached, and thus startup time essentially eliminated, is to build it into what's called the "system image". That system image is the binary that package compiler builds (https://github.com/JuliaLang/PackageCompiler.jl https://github.com/JuliaLang/PackageCompiler.jl). Julia then ships with a default system image that includes the standard library in order to remove the major chunk of code that "most" libraries share, which is why all of Julia Base works without JIT lag. However, that means everyone wants to have their thing, be it sparse matrices to statistics, in the standard library so that it gets the JIT-lag free build by default. This means the system image is huge, which is why PackageCompiler, which is simply a system for building binaries by appending package code to the system image, builds big binaries. What needs to happen is for packages to be able to precompile in a way that then caches LLVM and native code. Then there's no major compile time advantage to being in the system image, which will allow things to be pulled out of the system image to have a leaner Julia Base build without major drawbacks, which would then help make the system compile. That will then make it so that an LLVM and BLAS build does not have to be in every binary (which is what takes up most of the space and RAM), which would then allow Julia to much more comfortably move beyond the niche of scientific computing.
  - waffletower 4y ago
    'startup-time' vs. 'TTFP' are just semantics, but a crucial distinction. The focus on 'time-to-first-plot' is entrenched in the dominant workflow of the Julia community. If you want a broader range of adoption, the word choice matters. Having experienced a very similar dialogue in the Clojure community over startup time, and seen the new range of applications emerge as the issue was effectively addressed, I was startled by the very narrow and focussed acronym (TTFP) in the Julia community. I am happy that there is progress on the code caching front and will look to the link you shared. Thanks for elucidating this further!

joshjob42 4y ago

Can I ask what about your use case would involve starting Julia and importing packages over and over? At least for my workloads, I often write a small .jl script that imports everything I want and then runs some functions on a list of files, for instance, or I just do my work in a Jupyter notebook, and I pay the load cost once for hours or days of computation.. Annuitized over the entire time I might use the kernel the startup time is basically nothing. I'm not saying the load time should be a lot lot faster! That would be awesome! But I see folks complain about this and I'm wondering if they really need to be booting a new kernel constantly or if they could adapt how they work a little bit and get their desired result. (For instance, by running a .jl script instead of a bash script that calls Julia or something.)

waffletower 4y ago
Currently, the adaptation cost is too high. The ecosystem that I work in has many small, distributed, fast-startup applications. While I could create a fairly complex client server application written in Julia to work with its limitations, and write compact fast startup applications to communicate with it, the complexity and time investment is undesirable. Again, to use Julia here one must bend to the ecosystem's weaknesses. I think the community truly doesn't understand what a general purpose language is. If you compile with LLVM startup really ought to be a non-issue. Look at the startup of GraalVM, and they began with Java bytecode.
- joshjob42 4y ago
  Ah okay well yes I see how that might be an issue. I basically do everything in Julia, including calling out to shell commands, running scripts in other languages, Python packages, etc. etc. It's effectively my shell. I think the problem is that there needs to be a way to store the compiled code to disk and reimport it, which I think there are some plans toward that path. There were some developments discussed [here](https://www.youtube.com/watch?v=GnsONc9DYg0 https://www.youtube.com/watch?v=GnsONc9DYg0) to help with this, but the holy grail of fully compiled function caching across Julia kernels is not there yet.
jakobnissen 4y ago