4 ms·

Every time I try Julia on my computer, there is so much lag between input and output, I just go back to R.

clircle 4y ago
I cannot edit my note, but to explain a bit more, I do know that Julia is a compiled language, and is fast for some workflows. But for my data science and statistics workflow, it is slower than R. And I have a pretty standard workflow: clean data, plot data, maybe fit a linear regression model or something like that.
moelf 4y ago
it's called JIT, comparing to R, it probably would speed up some moderately complex calculation by 100x to 1000x, but if you don't have the type of workload, feel free to use R -- people should use whatever suits them the best!
bobbylarrybobby 4y ago
The "right" way to use Julia is to connect to a REPL. Once you pay the cost of compiling, it's insanely fast. As long as you keep your REPL open your compilations will remain valid and cached, and you won't have to repay the cost of compilation until you restart your REPL (which you should do very rarely). There's a well-known concept in Julia, [time-to-first-plot](https://discourse.julialang.org/t/time-to-first-plot-clarification/58534 https://discourse.julialang.org/t/time-to-first-plot-clarifi...), which encapsulates the large lag the first time you call a function. But there is no such thing as time-to-second-plot — the second plot is always fast.
- clircle 4y ago
  Yes, I have heard that. I'm used to frequently restarting R to ensure that codes are reproducible and can run in a clean environment. That's a best practice in technical research that seems at odds with the Julia workflow.
  - martinsmit 4y ago
    As someone who does work in similar domains as yourself, I find that making a main function which contains all of the code I want to be reproducible is a good way to achieve both high speed and running code in a clean environment. Writing .jl files in VSCode and sending code to the integrated REPL with shift-return for playing around is great and then when I want to run code properly I just comment out my testing code, wrap my important code in a main() function (which is sometimes everything apart from the imports) and then just make sure that I'm not referring to any variables or function methods defined that are now commented. This is made relatively easy as VSCode will immediately complain about possible method errors. It's not a perfect substitution for having fast startup and when I go back to Python or R for things like the holy grail called Tidyverse then it feels like a weight off my back, but the benefit of doing things the Julia way is that your code has a clear starting point and it helps readability as you can see what every script is supposed to do.
  - krastanov 4y ago
    I disagree with the conclusion. I agree on the best practices you mentioned, and I follow them when working in Julia (and the Julia Manifest.toml files provides for much better reproducibility compared to e.g. conda). Avoiding recompilation of code that has not changed is not at odds with rerunning your computational pipeline often to ensure reproducibility. It is true though that you can not translate the muscle memory you have in R to Julia. Also, I would say actual best practices would be to have CI, not to be rerunning your code manually.
bluenose69 4y ago
Startup time is not the only issue with Julia. Its error messages are long-winded and cryptic, and its manpages can be quite sketchy. Even so, Julia is a good tool for intensive computation, and I think it will continue to replace Fortran for some such work. Balance is the key. I use R all day long for data analysis, and don't really see Julia as competitive for such work. However, when it comes to numerical modelling, Julia competes well with Fortran and C/C++. I think Julia should be thought of as a Fortran replacement for overnight/overweek jobs, not an R replacement for interactive work.
- krastanov 4y ago
  YMMV. For me, Julia is a wonderful replacement of Python/numpy/scipy/pandas/Jax/Tensorflow/cython/sympy that I use mainly for interactive work. I am a fairly competent Python developer and know how to do idiomatic high-performance Python. I needed to learn new idioms and workflows when switching to Julia, but I am drastically more productive in it, both in interactive and in batch work.
  - p1esk 4y ago
    Can you give an example?
    - krastanov 4y ago
      I will attempt to, but do let me know if this is not the type of examples you meant: Julia idioms: Multimethods and multiple dispatch instead of python/java style OOP; Loops are fast, with zero-cost-abstractions and automatic SIMD, unlike python's need for vectorized libraries; "type-inferability" is incredibly important for good performance, which is quite different from python's duck typing (but the language is dynamic, so it is still an option); measuring how often your code allocates is the most trivial way to increase its performance by an order of magnitude. Julia workflow: you *have* to change your workflow, compared to coming from python, otherwise the latency of the compiler would cause you deep enraging frustration: e.g. use Revise.jl/Pluto.jl/REPL, organize your code so that small pieces can be recompiled without having to restart your session; keep your workspace clean and your computational pipeline organized, so that reproducibility testing is easy.
sundarurfriend 4y ago
That is one of the things this update specifically addresses (and the next 1.9 will do more of). There's a tangible difference in the REPL, things feel snappier. I'm sure there will still be lags in the odd place, and there's still a first time compilation, but that compiled result is cached a lot more, and so when you restart session as you mention below, that cache will be preserved.

I cannot edit my note, but to explain a bit more, I do know that Julia is a compiled language, and is fast for some workflows. But for my data science and statistics workflow, it is slower than R. And I have a pretty standard workflow: clean data, plot data, maybe fit a linear regression model or something like that.

moelf 4y ago

it's called JIT, comparing to R, it probably would speed up some moderately complex calculation by 100x to 1000x, but if you don't have the type of workload, feel free to use R -- people should use whatever suits them the best!

bobbylarrybobby 4y ago

The "right" way to use Julia is to connect to a REPL. Once you pay the cost of compiling, it's insanely fast. As long as you keep your REPL open your compilations will remain valid and cached, and you won't have to repay the cost of compilation until you restart your REPL (which you should do very rarely). There's a well-known concept in Julia, [time-to-first-plot](https://discourse.julialang.org/t/time-to-first-plot-clarification/58534 https://discourse.julialang.org/t/time-to-first-plot-clarifi...), which encapsulates the large lag the first time you call a function. But there is no such thing as time-to-second-plot — the second plot is always fast.

clircle 4y ago
Yes, I have heard that. I'm used to frequently restarting R to ensure that codes are reproducible and can run in a clean environment. That's a best practice in technical research that seems at odds with the Julia workflow.
- martinsmit 4y ago
  As someone who does work in similar domains as yourself, I find that making a main function which contains all of the code I want to be reproducible is a good way to achieve both high speed and running code in a clean environment. Writing .jl files in VSCode and sending code to the integrated REPL with shift-return for playing around is great and then when I want to run code properly I just comment out my testing code, wrap my important code in a main() function (which is sometimes everything apart from the imports) and then just make sure that I'm not referring to any variables or function methods defined that are now commented. This is made relatively easy as VSCode will immediately complain about possible method errors. It's not a perfect substitution for having fast startup and when I go back to Python or R for things like the holy grail called Tidyverse then it feels like a weight off my back, but the benefit of doing things the Julia way is that your code has a clear starting point and it helps readability as you can see what every script is supposed to do.
- krastanov 4y ago
  I disagree with the conclusion. I agree on the best practices you mentioned, and I follow them when working in Julia (and the Julia Manifest.toml files provides for much better reproducibility compared to e.g. conda). Avoiding recompilation of code that has not changed is not at odds with rerunning your computational pipeline often to ensure reproducibility. It is true though that you can not translate the muscle memory you have in R to Julia. Also, I would say actual best practices would be to have CI, not to be rerunning your code manually.

bluenose69 4y ago

Startup time is not the only issue with Julia. Its error messages are long-winded and cryptic, and its manpages can be quite sketchy. Even so, Julia is a good tool for intensive computation, and I think it will continue to replace Fortran for some such work. Balance is the key. I use R all day long for data analysis, and don't really see Julia as competitive for such work. However, when it comes to numerical modelling, Julia competes well with Fortran and C/C++. I think Julia should be thought of as a Fortran replacement for overnight/overweek jobs, not an R replacement for interactive work.

krastanov 4y ago
YMMV. For me, Julia is a wonderful replacement of Python/numpy/scipy/pandas/Jax/Tensorflow/cython/sympy that I use mainly for interactive work. I am a fairly competent Python developer and know how to do idiomatic high-performance Python. I needed to learn new idioms and workflows when switching to Julia, but I am drastically more productive in it, both in interactive and in batch work.
- p1esk 4y ago
  Can you give an example?
  - krastanov 4y ago
    I will attempt to, but do let me know if this is not the type of examples you meant: Julia idioms: Multimethods and multiple dispatch instead of python/java style OOP; Loops are fast, with zero-cost-abstractions and automatic SIMD, unlike python's need for vectorized libraries; "type-inferability" is incredibly important for good performance, which is quite different from python's duck typing (but the language is dynamic, so it is still an option); measuring how often your code allocates is the most trivial way to increase its performance by an order of magnitude. Julia workflow: you *have* to change your workflow, compared to coming from python, otherwise the latency of the compiler would cause you deep enraging frustration: e.g. use Revise.jl/Pluto.jl/REPL, organize your code so that small pieces can be recompiled without having to restart your session; keep your workspace clean and your computational pipeline organized, so that reproducibility testing is easy.

sundarurfriend 4y ago

That is one of the things this update specifically addresses (and the next 1.9 will do more of). There's a tangible difference in the REPL, things feel snappier. I'm sure there will still be lags in the odd place, and there's still a first time compilation, but that compiled result is cached a lot more, and so when you restart session as you mention below, that cache will be preserved.