29 ms·

I think this conversation is absolutely worth having. It's great to see that 1.10 is ~3x faster here, and there's probably another ~2 to 3x improvement or so fr

I think this conversation is absolutely worth having. It's great to see that 1.10 is ~3x faster here, and there's probably another ~2 to 3x improvement or so from profiling DynamicalSystems to see where the time is going. That said, for sub second startup time with lots of packages, I highly recommend using PackageCompiler to make a custom system image which brings the using time down to roughly 0.

jampekka 3y ago
Is shaving seconds-per-package import a feasible way to get the startup time fast enough for script-type workflows? Before long you end up importing 10 packages and it will be too slow even if each package is 1s. I appreciate that the devs seem to want to do it "the right way" but that seems have taken quite a while. Wouldn't something like slow interpreter option for "glue code" be feasible? Most code doesn't need to be fast, and waiting for the "hot functions" to compile on change wouldn't be too bad. And you could still AOT for maximum speed when needed. This is essentially how Javascript JITs manage to start up so fast. Another fine "hack" for many cases would be to keep the Julia process running (like a daemon) but making sure the state is clean when a script is run on it. Something like PackageCompiler is workable, at least for the short term. Long compiles are OK if they don't have to be done all the time. I indeed manage to get < 1s TTFP with DynamicalSystems with it (last time I tried a while ago I didn't manage). The UX for creating sysimages could be nicer, but doesn't seem anything that a quick script can't fix. Maybe I'll give Julia another go in my next analysis. From what I gather there's now a new approach for caching the compilation results. I really hope it will succeed, but I'm a bit jaded from the many "Julia 1.x fixes the TTFX" news.
- sgt101 3y ago
  Ahh I getcha a bit. What is the usecase though - which script-type workflows? (genuine curiosity, not just asking!)
  - jampekka 3y ago
    Typically modeling timeseries data and comparing it to data collected from humans (e.g. car telemetry, motion tracking or eye tracking signals). The models are stochastic dynamical systems which have to be optimized against the data. It's more or less exactly in Julia's niche. The script typically loads the data files and runs simulations (a lot of times when their parameters are optimized). This means the models have to be fast, but they are also recurrent in nature so they can't be vectorized to numpy. So depending on the case it's usually numba, cython or C++, all of which can be quite painful.
    - sgt101 3y ago
      So what you are saying is that you would like a process that was live - with everything loaded, and ready to respond to any code that you wanted it to run, as opposed to having to bring up a new Juila process for each step in the pipeline. Can't you compile the pipeline into a single script and then run it in a single instance of whatever?
      - jampekka 3y ago
        I want the opposite. To have different processes for each step. Potentially the different steps being in different languages. Or at least semantics of that. The steps should be independently callable and each call should result in the same output with same parameters, i.e. the state would reset for each run. I don't care so much how this is exactly implemented as long as it fullfills these. The data doesn't need to stay in the process memory. It's can be e.g. mmapped and is cached by the OS anyway. And deserializing tens or even hundreds of megabytes takes usually less time than Julia takes to import Plots. This is the thing that isn't feasible with Julia due to the TTFP (each step incurs the startup latency).
- ChrisRackauckas 3y ago
  The next step is to enable more post-invalidation precompile for Plots. This is something we recently setup with Symbolics.jl since that is another package which has some "invalidation by design" patterns, where certain assumptions made in the compiler (such as (x==y)::Bool) are invalidated by symbolic numbers acting differently than a lot of numbers and thus some over-eager optimizations require being removed upon using. PrecompileTools.@recompile_invalidations is a recent (v1.9 era) tool that allows for forcing the compile at package precompile time to consider the environment post-invalidation and thus re-precompile, effectively reducing the TTFX in these cases where invalidation is always going to occur by design. Plots.jl is a case like this because its recipe system is effectively allowing people to extend the plotting pipeline well after Plots is loaded, which is its main killer feature but effectively means that one should expect invalidations from it. I think that recompilation post invalidation should make a strong dent in that and plan to see if that's the case in a few months when I get the time to dive in. > Wouldn't something like slow interpreter option for "glue code" be feasible? Most code doesn't need to be fast, and waiting for the "hot functions" to compile on change wouldn't be too bad. And you could still AOT for maximum speed when needed. This is essentially how Javascript JITs manage to start up so fast. That is something I've been meaning to investigate more. A lot of the Plots.jl code is internally dynamic because of its design to hold everything in a Dict{Any} and recurse the plot recipes. In other words, it doesn't even benefit from having the JIT at all because internally everything is uninferred boxed variables. There's probably a good way to just do "please interpret the calls in this part of the pipeline" and not see any runtime difference but chop out the core of the compilation. It's relatively straightforward to do, it's a one liner using JuliaInterpreter.jl @interpret, and I think one dig that results in a win would be a nice example of what should be a pattern we do more often in the near future.
  - jampekka 3y ago
    I did briefly look at JuliaInterpreter.jl and if something like that can be used for general purpose, the TTFX would be fixed, the complaints would stop and I'd wager Julia would gain a lot in popularity. I really hope it happens. For now the interpreter seems to focus on debugging, and if I understand correctly, it can't be really used to e.g. speed up imports or to specify hot code to be AOT'd/JITed?

jampekka 3y ago

Is shaving seconds-per-package import a feasible way to get the startup time fast enough for script-type workflows? Before long you end up importing 10 packages and it will be too slow even if each package is 1s. I appreciate that the devs seem to want to do it "the right way" but that seems have taken quite a while. Wouldn't something like slow interpreter option for "glue code" be feasible? Most code doesn't need to be fast, and waiting for the "hot functions" to compile on change wouldn't be too bad. And you could still AOT for maximum speed when needed. This is essentially how Javascript JITs manage to start up so fast. Another fine "hack" for many cases would be to keep the Julia process running (like a daemon) but making sure the state is clean when a script is run on it. Something like PackageCompiler is workable, at least for the short term. Long compiles are OK if they don't have to be done all the time. I indeed manage to get < 1s TTFP with DynamicalSystems with it (last time I tried a while ago I didn't manage). The UX for creating sysimages could be nicer, but doesn't seem anything that a quick script can't fix. Maybe I'll give Julia another go in my next analysis. From what I gather there's now a new approach for caching the compilation results. I really hope it will succeed, but I'm a bit jaded from the many "Julia 1.x fixes the TTFX" news.

sgt101 3y ago
Ahh I getcha a bit. What is the usecase though - which script-type workflows? (genuine curiosity, not just asking!)
- jampekka 3y ago
  Typically modeling timeseries data and comparing it to data collected from humans (e.g. car telemetry, motion tracking or eye tracking signals). The models are stochastic dynamical systems which have to be optimized against the data. It's more or less exactly in Julia's niche. The script typically loads the data files and runs simulations (a lot of times when their parameters are optimized). This means the models have to be fast, but they are also recurrent in nature so they can't be vectorized to numpy. So depending on the case it's usually numba, cython or C++, all of which can be quite painful.
  - sgt101 3y ago
    So what you are saying is that you would like a process that was live - with everything loaded, and ready to respond to any code that you wanted it to run, as opposed to having to bring up a new Juila process for each step in the pipeline. Can't you compile the pipeline into a single script and then run it in a single instance of whatever?
    - jampekka 3y ago
      I want the opposite. To have different processes for each step. Potentially the different steps being in different languages. Or at least semantics of that. The steps should be independently callable and each call should result in the same output with same parameters, i.e. the state would reset for each run. I don't care so much how this is exactly implemented as long as it fullfills these. The data doesn't need to stay in the process memory. It's can be e.g. mmapped and is cached by the OS anyway. And deserializing tens or even hundreds of megabytes takes usually less time than Julia takes to import Plots. This is the thing that isn't feasible with Julia due to the TTFP (each step incurs the startup latency).
ChrisRackauckas 3y ago
The next step is to enable more post-invalidation precompile for Plots. This is something we recently setup with Symbolics.jl since that is another package which has some "invalidation by design" patterns, where certain assumptions made in the compiler (such as (x==y)::Bool) are invalidated by symbolic numbers acting differently than a lot of numbers and thus some over-eager optimizations require being removed upon using. PrecompileTools.@recompile_invalidations is a recent (v1.9 era) tool that allows for forcing the compile at package precompile time to consider the environment post-invalidation and thus re-precompile, effectively reducing the TTFX in these cases where invalidation is always going to occur by design. Plots.jl is a case like this because its recipe system is effectively allowing people to extend the plotting pipeline well after Plots is loaded, which is its main killer feature but effectively means that one should expect invalidations from it. I think that recompilation post invalidation should make a strong dent in that and plan to see if that's the case in a few months when I get the time to dive in. > Wouldn't something like slow interpreter option for "glue code" be feasible? Most code doesn't need to be fast, and waiting for the "hot functions" to compile on change wouldn't be too bad. And you could still AOT for maximum speed when needed. This is essentially how Javascript JITs manage to start up so fast. That is something I've been meaning to investigate more. A lot of the Plots.jl code is internally dynamic because of its design to hold everything in a Dict{Any} and recurse the plot recipes. In other words, it doesn't even benefit from having the JIT at all because internally everything is uninferred boxed variables. There's probably a good way to just do "please interpret the calls in this part of the pipeline" and not see any runtime difference but chop out the core of the compilation. It's relatively straightforward to do, it's a one liner using JuliaInterpreter.jl @interpret, and I think one dig that results in a win would be a nice example of what should be a pattern we do more often in the near future.
- jampekka 3y ago
  I did briefly look at JuliaInterpreter.jl and if something like that can be used for general purpose, the TTFX would be fixed, the complaints would stop and I'd wager Julia would gain a lot in popularity. I really hope it happens. For now the interpreter seems to focus on debugging, and if I understand correctly, it can't be really used to e.g. speed up imports or to specify hot code to be AOT'd/JITed?