4 ms·
So what you are saying is that you would like a process that was live - with everything loaded, and ready to respond to any code that you wanted it to run, as o
by sgt101 3y ago
So what you are saying is that you would like a process that was live - with everything loaded, and ready to respond to any code that you wanted it to run, as opposed to having to bring up a new Juila process for each step in the pipeline.
Can't you compile the pipeline into a single script and then run it in a single instance of whatever?
- jampekka 3y agoI want the opposite. To have different processes for each step. Potentially the different steps being in different languages. Or at least semantics of that. The steps should be independently callable and each call should result in the same output with same parameters, i.e. the state would reset for each run. I don't care so much how this is exactly implemented as long as it fullfills these. The data doesn't need to stay in the process memory. It's can be e.g. mmapped and is cached by the OS anyway. And deserializing tens or even hundreds of megabytes takes usually less time than Julia takes to import Plots. This is the thing that isn't feasible with Julia due to the TTFP (each step incurs the startup latency).