18 ms·

Julia 1.6 addresses latency issues

177 points by leephillips 5y ago

gugagore 5y ago
It is amazing and frustrating to me how much latency affects my productivity. I wish I could more effortlessly switch between tasks, or just meditate and relax while I wait for something I just did on the REPL to finish. But I don't. More often than not, a 30-second delay to e.g. plot something destroys my ability to stay in a productive zone. I have been using Julia 1.6 since the release, and I'm so grateful not only that some computations run a bit faster, but that the interactivity is so improved. Even seeing a progress bar can help me stay focused, because it can be fun to watch (parallel precompilation is especially fun). When a command just hangs, I feel left in the dark about how much boredom I'll have to endure.
- ssivark 5y ago
  Very interesting UX observations on interactive programming. Kinda like mirrors in waiting areas, I wonder whether judicious logging messages about the compilation process (not a wall of text, but just enough) will serve to keep users engaged while also educating them about the compilation happening on the backend. That will help users feel more agency, and also improve their mental models of how to structure their code/activity for faster compilation.
  - bobthepanda 5y ago
    This is the theory behind spinners over separate page loads.
  - thecupisblue 5y ago
    That actually helps. Talking from years of experience doing Android development, where in the early days (and still on some projects) you would have +5 minute rebuild time, and it's supper annoying to check minor things. Having more logs actually helped it seem faster, even tho it wouldn't be, but also you could know where you are stuck and why, which helps identify bottlenecks in build processes. Sometimes it would be enough to just google what the long task does and see "oh wait I don't actually need that step for my daily development" (e.g. resource crunching, crash reporting initalization etc) or point you in the direction of "why isn't this caching itself". Quite a useful thing, and should be available as an option at least. In a REPL it might be context noise, but should still be there as a --verbose option.
- Mauricebranagh 5y ago
  Well as one of the aims as I understand it for Julia was as an alternate to Fortran. You should consider your self lucky when I started the compile link process took a while and that is assuming you had real time access. I recall one of my colleagues who was running her code on an ICL at AWE (Underwater Weapons Establishment) coming into the terminal room logging in and sighing "48 jobs in GEORGE ahead of hers. A year later we brought a Pr1me Super Mini which made things a lot faster.
- globular-toast 5y ago
  This is why git was such a game changer. Making a commit to SVN took a considerable amount of time, especially back in the day on slow internet connections. Making a commit with git is generally instant and doesn't break flow. It changes the way you work. People used to avoid committing with SVN, sometimes doing it only once a day. That seems completely insane now. I have very little tolerance for latency in my tools. It's one of the top deciding factors for me when choosing which tools to use. It's that important.
Buttons840 5y ago
I'm a big fan of Julia. It does live up to its speed claims. I've implemented the board game Go in Python, Rust, and Julia and Julia is definitely closer to Rust in speed. Same algorithms were used for all implementations. Julia's time to first plot still has some problems. The Plots library can build animations, but the time to first animation on my computer is like 10 minutes, and the time to second animation is another 10 minutes. Probably just a bug, I haven't found any other case that takes so long. I've also mentioned before that a reinforcement learning algorithm I ported from Python/PyTorch to Flux was faster, not because of training times, but because of all the other stuff (and RL has more "other stuff" than supervised learning) that goes on outside the core training loop is so much faster.
- throwaway894345 5y ago
  > I've implemented the board game Go in Python, Rust, and Julia and Julia Oof. I reread this several times consecutively as "I've implemented the board game in Go, Python, Rust ...".
  - TchoBeer 5y ago
    Same, and I literally have a correspondence game going on in another tab
    - xiaodai 5y ago
      what's your ogs id? I challenge u
      - TchoBeer 5y ago
        ID is Ada Countess of Lovelace. What's yours.
- dunefox 5y ago
  > but the time to first animation on my computer is like 10 minutes Have you tried 1.6 already? I find it's substantially faster.
gugagore 5y ago
Speeding up compilation itself is one approach to the latency issue. And there's also the idea of blending compilation and interpretation using e.g. https://github.com/JuliaDebug/JuliaInterpreter.jl https://github.com/JuliaDebug/JuliaInterpreter.jl .
- Zababa 5y ago
  V8 reduced the start up times of WebAssembly this way but with a single pass compiler instead of an interpreter. Here's the article: https://v8.dev/blog/liftoff https://v8.dev/blog/liftoff
  - ufo 5y ago
    One difference is that web assembly is typed and is designed to make these compilers possible. In the context of Javascript, V8 did the opposite. Originally they had a baseline compiler for Javascript but now they use an interpreter, which reduces the startup latency.
adsharma 5y ago
For me the issue manifested as a 10 sec latency to format a Julia file using Format.jl Solved via flags to disable JIT and brought it down to a couple of secs. Native binary would be much nicer.
- enriquto 5y ago
  Two seconds to process a tiny text file enters well into the realm of "completely unusable" in my eyes. It wouldn't be so bad if the Julia developers acknowledged that this is a valid concern (that they are not dealing with it right now for whatever reasons) and that the ecosystem will not be considered complete until this fundamental problem is solved. But this is infuriatingly not the case. Instead, they tell you that you are "holding it wrong" and that this is not really a problem, that your usage is "niche", that the interpreter is alright as it is, and that the time to first plot is unlikely to ever go below a hundred milliseconds. I find it really depressing for the language itself is incredibly beautiful. My only hope is that an independent, fast and unix-friendly implementation of the language arises, thus freeing the reference implementation of the efficiency burden and allowing it to be simpler. Something like the lua/luajit split.
  - krastanov 5y ago
    > the time to first plot is unlikely to ever go below a hundred milliseconds How is that controversial or disappointing!? Why would anyone bother optimizing this? Nor is matlab/octave/python any faster. `python3 -c "import matplotlib.pyplot as plt; plt.plot([1,2,3]);"` takes 600ms on my (powerful) workstation and that does not even include creating the plot window. To be clear, I do believe there is much more work to be done to decrease latency in Julia, but your targets are ridiculous. And as a regular on their bugtracker and forum, the devs definitely acknowledge these issues and have many times said it is one of their main priorities. By the way, if you want streaming live updated plots, this latency to first frame is not a problem. It is already straightforward to make such fast live plots in Julia (although it does not fit my personal taste for how to do it).
    - enriquto 5y ago
krastanov 5y ago
I am very confused by the claim in the first sentence of the article: "On March 24, version 1.6.0 of the Julia programming language was released. This is the first feature release since 1.0 came out in 2018". How is 1.6 a "feature release", but 1.1-1.5 are not!? Especially given the enormous new set of multi-threading features in 1.3. Edit: ah, thanks for the response, it seem I just do not know the difference between "feature" and "timed" release.
- leephillips 5y ago
  That’s the terminology used in the development process. The releases > 1.0 and < 1.6 are called “timed releases”. It doesn’t mean they don’t contain any new features.
up6w6 5y ago
I use and love Julia but I really wanted to see the general purpose language that is claimed. On one hand, you see amazing scientific libs like DifferentialEquations.jl, on the other side, things like the PackageCompiler.jl mentioned just sucks at generating binaries for daily basis.
- krastanov 5y ago
  Isn't "generating binaries" just as bad for other interpreted (interpeted-ish) languages? If you generate a "python binary", you need to package python with your binary. Same for perl/ruby. It just seems weird that people expect julia to be able to do that. It is cute that PackageCompiler.jl exists and it is cute that more AOT compilation work is being currently done, but it seems crazy to expect Julia to be good at making binaries (and I would say that about python and perl too). And by extension, it seems weird to me to complain that Julia is not a general purpose language because it can not generate binaries. What stops me from making the same statement about python, which is definitely general purpose?
  - reikonomusha 5y ago
    It’s not at all bad for Common Lisp, which is a superlatively interactive language.
    - krastanov 5y ago
      Could you elaborate or give examples? I guess all the complaints about binaries come from people used to something like Common Lisp, but while I have a general understanding of what a lisp is, this type of "provide a binary for your interpreted-ish language" is an incredibly foreign idea to me.
      - reikonomusha 5y ago
        Common Lisp was designed to be interactive and have a REPL. You can redefine functions, classes, etc. on the fly with strictly-defined semantics. (You don’t have to guess what happens if you, say, re-name a field of your class.) This is insanely useful during development, where you absolutely want to avoid doing full recompiles every time you make a little change you want to test. Some people call this “interactive and incremental development”. (Of course, you always have the option to just re-compile everything from scratch if you so please.) Common Lisp was also designed to be compiled. Most implementations these days compile to machine code. Compilation is incremental, but ahead-of-time. That means you can start running your program without having yet compiled all the features or libraries you want. You can—while you’re in the REPL or even while your program is running—compile and load extra libraries later. Compilation is cached across sessions, so you won’t ever have to recompile something that doesn’t change. Despite Lisp being mega-interactive, incremental, and dynamic, just about every implementation of Lisp allows you to just write out a compiled binary. In the implementation called “Steel Bank Common Lisp” (SBCL), from the REPL, you just write: (sb-ext:save-lisp-and-die "mycoolprog" :executable t :entry-point 'main) which will produce a statically linked executable binary called “mycoolprog” using the function called “main” as the entry point. Unless you’ve specifically programmed it in, there will be no JIT lag, no runtime compilation, etc. It will all just be compiled machine code running at the speed of your processor. (It’s possible and even easy to invoke the compiler at run-time, even in your static binary, which people rarely do, and when they do, they know exactly what they’re doing and why.) All of this is a complete non-issue in Lisp, and hasn’t been for about 35 years (or more).
smabie 5y ago
So I used to be a big proponent of Julia, and in some ways, I still am. But I very recently tried to write a high performance production system in it, and was sorely disappointed. The tooling is just so buggy and it's clear that the community isn't really interested in using it for anything besides modeling/research in a Jupyter notebook. Things that kind of suck about using Julia for production: 1. Never could get Revise to work, had to restart my REPL everytime I changed any code. Even though Julia 1.6 was a lot faster than 1.5, it still took too long. 2. Couldn't find a static type checker that actually worked (I tried JET and StaticLint). I feel like static typing is just so important for a production system, but of course the community isn't really interested because of the research focus. 3. Editor tooling. The LSP server absolutely sucks. I first tried using it with emacs (both lsp-mode and eglot mode), but it would crash constantly. I think switched to VSCode (much to my chagrin), and that worked marginally better though still very poorly. It was clear that the LSP server had no idea what was going on in my macro heavy code. It couldn't jump to definitions or usages much of the time. It could never correctly determine whether a variable was unused or misspelled either. Coupled with the lack of static type checking, this was extremely frustrating. 4. Never felt like the community could answer any of my questions. If you have some research or stats question, they were great, but anything else, forget about it. Will all of that being said, I do still use Julia for research and I find it works really well. The language is very nicely designed. All and all, I decided to ditch Julia and decided to go with Rust (after some consideration of OCaml, but unfortunately the multi-core story still isn't there yet) and am a lot happier.
- Tainnor 5y ago
  I would also add: 5. The module system is very primitive. 6. The testing framework is extremely barebones. I agree with your assessment, Julia is great for crunching numbers etc., but I wouldn't write a whole application in it.
  - kruxigt 5y ago
    I agree with you guys. I have been coding in Julia for 2 years and very little has happened in tooling. For example, the graphical profiler is buggy. I love the language though. Hoping things will change!

177 points by leephillips 5y ago

It is amazing and frustrating to me how much latency affects my productivity. I wish I could more effortlessly switch between tasks, or just meditate and relax while I wait for something I just did on the REPL to finish. But I don't. More often than not, a 30-second delay to e.g. plot something destroys my ability to stay in a productive zone. I have been using Julia 1.6 since the release, and I'm so grateful not only that some computations run a bit faster, but that the interactivity is so improved. Even seeing a progress bar can help me stay focused, because it can be fun to watch (parallel precompilation is especially fun). When a command just hangs, I feel left in the dark about how much boredom I'll have to endure.

ssivark 5y ago
Very interesting UX observations on interactive programming. Kinda like mirrors in waiting areas, I wonder whether judicious logging messages about the compilation process (not a wall of text, but just enough) will serve to keep users engaged while also educating them about the compilation happening on the backend. That will help users feel more agency, and also improve their mental models of how to structure their code/activity for faster compilation.
- bobthepanda 5y ago
  This is the theory behind spinners over separate page loads.
- thecupisblue 5y ago
  That actually helps. Talking from years of experience doing Android development, where in the early days (and still on some projects) you would have +5 minute rebuild time, and it's supper annoying to check minor things. Having more logs actually helped it seem faster, even tho it wouldn't be, but also you could know where you are stuck and why, which helps identify bottlenecks in build processes. Sometimes it would be enough to just google what the long task does and see "oh wait I don't actually need that step for my daily development" (e.g. resource crunching, crash reporting initalization etc) or point you in the direction of "why isn't this caching itself". Quite a useful thing, and should be available as an option at least. In a REPL it might be context noise, but should still be there as a --verbose option.
Mauricebranagh 5y ago
Well as one of the aims as I understand it for Julia was as an alternate to Fortran. You should consider your self lucky when I started the compile link process took a while and that is assuming you had real time access. I recall one of my colleagues who was running her code on an ICL at AWE (Underwater Weapons Establishment) coming into the terminal room logging in and sighing "48 jobs in GEORGE ahead of hers. A year later we brought a Pr1me Super Mini which made things a lot faster.
globular-toast 5y ago
This is why git was such a game changer. Making a commit to SVN took a considerable amount of time, especially back in the day on slow internet connections. Making a commit with git is generally instant and doesn't break flow. It changes the way you work. People used to avoid committing with SVN, sometimes doing it only once a day. That seems completely insane now. I have very little tolerance for latency in my tools. It's one of the top deciding factors for me when choosing which tools to use. It's that important.

Buttons840 5y ago

I'm a big fan of Julia. It does live up to its speed claims. I've implemented the board game Go in Python, Rust, and Julia and Julia is definitely closer to Rust in speed. Same algorithms were used for all implementations. Julia's time to first plot still has some problems. The Plots library can build animations, but the time to first animation on my computer is like 10 minutes, and the time to second animation is another 10 minutes. Probably just a bug, I haven't found any other case that takes so long. I've also mentioned before that a reinforcement learning algorithm I ported from Python/PyTorch to Flux was faster, not because of training times, but because of all the other stuff (and RL has more "other stuff" than supervised learning) that goes on outside the core training loop is so much faster.

throwaway894345 5y ago
> I've implemented the board game Go in Python, Rust, and Julia and Julia Oof. I reread this several times consecutively as "I've implemented the board game in Go, Python, Rust ...".
- TchoBeer 5y ago
  Same, and I literally have a correspondence game going on in another tab
  - xiaodai 5y ago
    what's your ogs id? I challenge u
    - TchoBeer 5y ago
      ID is Ada Countess of Lovelace. What's yours.
dunefox 5y ago
> but the time to first animation on my computer is like 10 minutes Have you tried 1.6 already? I find it's substantially faster.

gugagore 5y ago

Speeding up compilation itself is one approach to the latency issue. And there's also the idea of blending compilation and interpretation using e.g. https://github.com/JuliaDebug/JuliaInterpreter.jl https://github.com/JuliaDebug/JuliaInterpreter.jl .

Zababa 5y ago
V8 reduced the start up times of WebAssembly this way but with a single pass compiler instead of an interpreter. Here's the article: https://v8.dev/blog/liftoff https://v8.dev/blog/liftoff
- ufo 5y ago
  One difference is that web assembly is typed and is designed to make these compilers possible. In the context of Javascript, V8 did the opposite. Originally they had a baseline compiler for Javascript but now they use an interpreter, which reduces the startup latency.

adsharma 5y ago

For me the issue manifested as a 10 sec latency to format a Julia file using Format.jl Solved via flags to disable JIT and brought it down to a couple of secs. Native binary would be much nicer.

enriquto 5y ago
Two seconds to process a tiny text file enters well into the realm of "completely unusable" in my eyes. It wouldn't be so bad if the Julia developers acknowledged that this is a valid concern (that they are not dealing with it right now for whatever reasons) and that the ecosystem will not be considered complete until this fundamental problem is solved. But this is infuriatingly not the case. Instead, they tell you that you are "holding it wrong" and that this is not really a problem, that your usage is "niche", that the interpreter is alright as it is, and that the time to first plot is unlikely to ever go below a hundred milliseconds. I find it really depressing for the language itself is incredibly beautiful. My only hope is that an independent, fast and unix-friendly implementation of the language arises, thus freeing the reference implementation of the efficiency burden and allowing it to be simpler. Something like the lua/luajit split.
- krastanov 5y ago
  > the time to first plot is unlikely to ever go below a hundred milliseconds How is that controversial or disappointing!? Why would anyone bother optimizing this? Nor is matlab/octave/python any faster. `python3 -c "import matplotlib.pyplot as plt; plt.plot([1,2,3]);"` takes 600ms on my (powerful) workstation and that does not even include creating the plot window. To be clear, I do believe there is much more work to be done to decrease latency in Julia, but your targets are ridiculous. And as a regular on their bugtracker and forum, the devs definitely acknowledge these issues and have many times said it is one of their main priorities. By the way, if you want streaming live updated plots, this latency to first frame is not a problem. It is already straightforward to make such fast live plots in Julia (although it does not fit my personal taste for how to do it).
  - enriquto 5y ago

krastanov 5y ago

I am very confused by the claim in the first sentence of the article: "On March 24, version 1.6.0 of the Julia programming language was released. This is the first feature release since 1.0 came out in 2018". How is 1.6 a "feature release", but 1.1-1.5 are not!? Especially given the enormous new set of multi-threading features in 1.3. Edit: ah, thanks for the response, it seem I just do not know the difference between "feature" and "timed" release.

leephillips 5y ago
That’s the terminology used in the development process. The releases > 1.0 and < 1.6 are called “timed releases”. It doesn’t mean they don’t contain any new features.

up6w6 5y ago

I use and love Julia but I really wanted to see the general purpose language that is claimed. On one hand, you see amazing scientific libs like DifferentialEquations.jl, on the other side, things like the PackageCompiler.jl mentioned just sucks at generating binaries for daily basis.

krastanov 5y ago
Isn't "generating binaries" just as bad for other interpreted (interpeted-ish) languages? If you generate a "python binary", you need to package python with your binary. Same for perl/ruby. It just seems weird that people expect julia to be able to do that. It is cute that PackageCompiler.jl exists and it is cute that more AOT compilation work is being currently done, but it seems crazy to expect Julia to be good at making binaries (and I would say that about python and perl too). And by extension, it seems weird to me to complain that Julia is not a general purpose language because it can not generate binaries. What stops me from making the same statement about python, which is definitely general purpose?
- reikonomusha 5y ago
  It’s not at all bad for Common Lisp, which is a superlatively interactive language.
  - krastanov 5y ago
    Could you elaborate or give examples? I guess all the complaints about binaries come from people used to something like Common Lisp, but while I have a general understanding of what a lisp is, this type of "provide a binary for your interpreted-ish language" is an incredibly foreign idea to me.
    - reikonomusha 5y ago
      Common Lisp was designed to be interactive and have a REPL. You can redefine functions, classes, etc. on the fly with strictly-defined semantics. (You don’t have to guess what happens if you, say, re-name a field of your class.) This is insanely useful during development, where you absolutely want to avoid doing full recompiles every time you make a little change you want to test. Some people call this “interactive and incremental development”. (Of course, you always have the option to just re-compile everything from scratch if you so please.) Common Lisp was also designed to be compiled. Most implementations these days compile to machine code. Compilation is incremental, but ahead-of-time. That means you can start running your program without having yet compiled all the features or libraries you want. You can—while you’re in the REPL or even while your program is running—compile and load extra libraries later. Compilation is cached across sessions, so you won’t ever have to recompile something that doesn’t change. Despite Lisp being mega-interactive, incremental, and dynamic, just about every implementation of Lisp allows you to just write out a compiled binary. In the implementation called “Steel Bank Common Lisp” (SBCL), from the REPL, you just write: (sb-ext:save-lisp-and-die "mycoolprog" :executable t :entry-point 'main) which will produce a statically linked executable binary called “mycoolprog” using the function called “main” as the entry point. Unless you’ve specifically programmed it in, there will be no JIT lag, no runtime compilation, etc. It will all just be compiled machine code running at the speed of your processor. (It’s possible and even easy to invoke the compiler at run-time, even in your static binary, which people rarely do, and when they do, they know exactly what they’re doing and why.) All of this is a complete non-issue in Lisp, and hasn’t been for about 35 years (or more).

smabie 5y ago

So I used to be a big proponent of Julia, and in some ways, I still am. But I very recently tried to write a high performance production system in it, and was sorely disappointed. The tooling is just so buggy and it's clear that the community isn't really interested in using it for anything besides modeling/research in a Jupyter notebook. Things that kind of suck about using Julia for production: 1. Never could get Revise to work, had to restart my REPL everytime I changed any code. Even though Julia 1.6 was a lot faster than 1.5, it still took too long. 2. Couldn't find a static type checker that actually worked (I tried JET and StaticLint). I feel like static typing is just so important for a production system, but of course the community isn't really interested because of the research focus. 3. Editor tooling. The LSP server absolutely sucks. I first tried using it with emacs (both lsp-mode and eglot mode), but it would crash constantly. I think switched to VSCode (much to my chagrin), and that worked marginally better though still very poorly. It was clear that the LSP server had no idea what was going on in my macro heavy code. It couldn't jump to definitions or usages much of the time. It could never correctly determine whether a variable was unused or misspelled either. Coupled with the lack of static type checking, this was extremely frustrating. 4. Never felt like the community could answer any of my questions. If you have some research or stats question, they were great, but anything else, forget about it. Will all of that being said, I do still use Julia for research and I find it works really well. The language is very nicely designed. All and all, I decided to ditch Julia and decided to go with Rust (after some consideration of OCaml, but unfortunately the multi-core story still isn't there yet) and am a lot happier.

Tainnor 5y ago
I would also add: 5. The module system is very primitive. 6. The testing framework is extremely barebones. I agree with your assessment, Julia is great for crunching numbers etc., but I wouldn't write a whole application in it.
- kruxigt 5y ago
  I agree with you guys. I have been coding in Julia for 2 years and very little has happened in tooling. For example, the graphical profiler is buggy. I love the language though. Hoping things will change!