14 ms·

So, looks like Julia would be an easier transition for a lot of academic scientists. What am I missing? I mostly use R and Python. Can anyone tell me briefly wh

So, looks like Julia would be an easier transition for a lot of academic scientists. What am I missing? I mostly use R and Python. Can anyone tell me briefly why I should use Julia over Python?

pierre_d528 7y ago
"Why Does Julia Work So Well? There is an obvious reason to choose Julia: it's faster than other scripting languages, allowing you to have the rapid development of Python/MATLAB/R while producing code that is as fast as C/Fortran" https://ucidatascienceinitiative.github.io/IntroToJulia/Html/WhyJulia https://ucidatascienceinitiative.github.io/IntroToJulia/Html...
- enriquto 7y ago
  yet the startup time remains slower...
  - tomkwong 7y ago
    Startup feels instant to me. Perhaps you are talking about precompilation. In that case, it’s insignificant for most computationally intensive applications.
    - enriquto 7y ago
      I don't know the difference between startup and precompilation, and I do not really care, but if I launch a julia script from the command line it is unbearably slow for no apparent reason. Octave, on the other hand, launches instantly and starts making computations. This is understandable, because julia is not intended to be used that way. You are supposed to "live" inside the repl. However, I prefer tools that are flexible enough that can be used comfortably in non-intended ways.
      - FridgeSeal 7y ago
        That’s compilation time. Currently yeah, it’s not fantastic, but I believe that’s being actively worked on.
  - subroutine 7y ago
    Also if you have any need to generate plots & graphs - RIP Julia. I was excited to try Julia, since it seemed to integrate some of the best features of each MATLAB, R, and Python. I was truly disappointed to discover that Julia would take minutes to render the exact same plots I was generating in Octave almost instantly.
    - dnautics 7y ago
      to be fair, that typically only happens the first time you make your graph; replots are typically lightning fast. but this was a major frustration for me, I haven't had occasion to use it more recently, but I understand maybe things are a bit better for graphing in julialand in the last few months?
      - improbable22 7y ago
        I thought it had got better, but also I've just adjusted to work around it. Here are some timings today, Julia 1.1, cold start to first plot: $ julia -e '@time (using GR; plot(rand(20)))' 3.931433 seconds (10.38 M allocations: 521.292 MiB, 6.44% gc time) $ julia -e '@time (using Plots; plot(rand(20)))' 19.498644 seconds (57.26 M allocations: 2.844 GiB, 8.07% gc time) Running with less compilation: $ julia --compile=min -e '@time (using GR; plot(rand(20)))' 0.375836 seconds (368.83 k allocations: 20.190 MiB, 1.65% gc time) $ julia --compile=min -e '@time (using Plots; plot(rand(20)))' 4.302867 seconds (6.41 M allocations: 371.485 MiB, 5.07% gc time) But, as you say, it's much much quicker once started, like 1-5ms per plot.
        dnautics 7y ago
        4.3 seconds seems great. I remember when it felt like a minute or two.
    - enriquto 7y ago
      this is not really an issue, as long as you ignore the julia plotting capabilities; which should have never been there anyway. You can easily dump your numbers (and functions) on a text file and gnuplot them.
new4thaccount 7y ago
Julia JITs to native LLVM code and is really fast for a lot of use cases. For things where the JIT warming up takes less time than the full job, it can be better than Python. It was also developed with numerical computation in mind, so it was designed for that performance wise and has so many brilliant people working on the language and library. You can use macros for awesome DSLs and run on the GPU and in parallel a lot easier than Python in some cases. It has a first class package manager, great REPL and doesn't need an installer.
- patrick5415 7y ago
  I didn’t find the REPL that great, because it takes forever to jit anything. My laptop is not that old and creating an array with 5 elements takes over a second. If I type a syntax error, it takes tens of seconds to produce an error. This yields a very frustrating experience and doesn’t lend itself to an effective prototyping environment. For now at least I’ll be sticking with matlab.
  - improbable22 7y ago
    There are indeed frustrating lags due to JIT, but they have got better lately & are being worked on -- if I understand right this is now one of the priorities, after focusing on getting the breaking changes done before 1.0. Here's how long a vector of 5 random numbers takes, after a cold start, on 1.1: $ julia -e '@time rand(5)' 0.054343 seconds (121.03 k allocations: 6.219 MiB)
  - eigenspace 7y ago
    That sounds really bizarre. What version of which OS are you using? I've heard that older versions of Windows (I think 7) have problems with the REPL that cause extreme latency. I don't think the latency you're experiencing is normal.
    - patrick5415 7y ago
      I was running Debian 9, with Julia from julialang.org (not the Debian repo). This was six months ago so maybe things have improved since then.
eigenspace 7y ago
Well written pure julia programs tend to be about as fast as C, or in other words about 100 times faster than Python or R. Before that gets you too excited or sceptical, let me say that end users will not really see many significant speed boosts. When I say end users, I mean people who are just loading packages and writing scripts where they plug variables into functions from the package. The reason these end users won't see much difference is that a well made Python or R package (including things like Numpy) are actually Python and R wrappers around C, C++, Fortran or Julia code. However, if you are trying to do something that your packages weren't directly designed to do and writing non-trivial code, then you're going to start seeing the speed advantages of Julia. With all that in mind, I'd say the main reason for end users to use Julia is not the speed, it's the expressiveness and the top notch libraries. Expressiveness is kinda a vague concept, but what I'm basically saying is that through things like multiple dispatch and macros, julia is able to provide ways of writing programs in almost any domain that feel incredibly natural and various programs will compose with eachother in ways you won't see in any language except Common Lisp. Finally, because of Julia's native speed and expressiveness, it doesn't get in package developers way like Python and R do and so we're seeing that even though julia has a much smaller community than Python or R, we have a package ecosystem that's quite comparable and in certain specific regions, flat out superior. As adoption grows, we're going to see the julia package ecosystem accelerate and Julia will make sense for more end users. For now, whether or not Julia makes sense for you really depends on what field you're working in (ie. if there are good packages already for the tasks you're interested in) and how advanced a programmer you are / want to be (ie. if you're going to be trying to do something that's not already covered by existing packages).
- mehrdadn 7y ago
  > However, if you are trying to do something that your packages weren't directly designed to do and writing non-trivial code, then you're going to start seeing the speed advantages of Julia. Or you use Numba. :-) But yeah, the speed can be pretty bad compare to native code if you can't compile everything to native with Numba. To put some concrete numbers here, the one time I had some complicated but heavily-optimized Python and C++ scicomp codebases computing exactly the same thing, I saw [1] running times like the following depending on the algorithm: 1. 10.6s with NumPy, 7.5s with NumPy + Numba, and 0.6s in direct C++ (1.4x and 12.5x respectively) 2. 68s with NumPy, 6.5s with NumPy + Numba, 0.5s in direct C++ (10x and 13x respectively) The first algorithm was simpler and more vectorizable, while the second algorithm was more complicated and less vectorizable but with a lower time complexity. In either case, the algorithm still had to do a fair bit of work in Python, so it wasn't running native code 100% of the time (which I think is fairly realistic). In either case, you can see the difference between C++ and Python was around 13x when using Numba... not quite as bad as 100x, but I imagine Julia probably does better. [1] Actually, originally I saw only a 5x improvement with Python 2.7 on my older laptop, but I re-ran these on my current laptop under Python 3.7 and the difference is 13x now.
adamnemecek 7y ago
The whole development experience is so much better than Python.
- agumonkey 7y ago
  any video or article about it ?
- FridgeSeal 7y ago
  Right?! Not having a packaging system that is a stapled-on-afterthought is so nice.
dnautics 7y ago
if you want the flexibility to do something truly bizzare with confidence, you really ought to use Julia, and don't look back. A while back I was prototyping let's just say... unusual binary datatype representations for numbers. All i had to do was reimplement a handful of operations (+, -, x, /, one, zero) and I got everything from fourier transforms to matrix solving for free. Comparing numerical performance with standard IEEE representations was then easy, and I had confidence that my comparisons were legit, since I was literally calling the same function against both numerical types. More recently I wanted to play around with galois fields, following Mary Wootter's impressive work, and was able to test some ideas very quickly (and trivially deploy on a supercomputer cluster) in few lines of code using Julia. That's not a typical use case, but it's a thing. I am thinking about playing around with complex numbers (when I get some free time, which increasing seems like 'never') in deep learning, and for similar reasons Julia will be an obvious choice.
- dnautics 7y ago
  I should probably post evidence; this was a live demo I did of the floating point stuff at Stanford (demo begins ~53 minutes in) https://youtu.be/aP0Y1uAA-2Y https://youtu.be/aP0Y1uAA-2Y
throw20102010 7y ago
You should use Julia over Python because of the JIT compilation makes it faster, as long as you aren’t constantly writing code and running it a single time. Also as long as you are able to keep your REPL open for days at a time. It takes several minutes to load some popular plotting packages in Julia (because of the JIT), so it will take you some time to adjust your workflow to leaving your REPL open all the time. But the Juno IDE is just as good as Jupyter notebooks or RStudio, and there are probably high quality libraries for whatever you’re doing, unless you work in one of those niche areas.
ThenAsNow 7y ago
I just wrote my first bit of Julia this weekend. I'm impressed - it's clearly been designed for technical/numeric computing with modern language features baked-in. The type system and macros are some significantly distinguishing characteristics as compared to Python. The Unitful.jl package illustrates how these can be used to powerful effect. Multiple dispatch is another such characteristic. Having benefited from static type systems in other software projects, I would never again want to write significant technical code without a first-class type system supporting declarative type constraints. If I could never write Matlab again and instead use Julia, I'd be delighted.
thetwentyone 7y ago
I've moved all of my development from Python to Julia - The syntax is similar, but I think Julia's is more expressive and powerful once you start looking into the details. Macros, broadcasting (automatic vectorization), lambda functions, string interpolation, 1-based indexing (which weirded me out at first but I found to be more natural for most things I work on). It's been an almost wholly positive transition.
pjmlp 7y ago
No need for a second language, you can write everything in Julia, while enjoying a mix of productivity and code execution performance. Plus it has quite a few Lisp inspired features, like multi-methods and powerful macros.

So, looks like Julia would be an easier transition for a lot of academic scientists. What am I missing? I mostly use R and Python. Can anyone tell me briefly why I should use Julia over Python?

pierre_d528 7y ago

"Why Does Julia Work So Well? There is an obvious reason to choose Julia: it's faster than other scripting languages, allowing you to have the rapid development of Python/MATLAB/R while producing code that is as fast as C/Fortran" https://ucidatascienceinitiative.github.io/IntroToJulia/Html/WhyJulia https://ucidatascienceinitiative.github.io/IntroToJulia/Html...

enriquto 7y ago
yet the startup time remains slower...
- tomkwong 7y ago
  Startup feels instant to me. Perhaps you are talking about precompilation. In that case, it’s insignificant for most computationally intensive applications.
  - enriquto 7y ago
    I don't know the difference between startup and precompilation, and I do not really care, but if I launch a julia script from the command line it is unbearably slow for no apparent reason. Octave, on the other hand, launches instantly and starts making computations. This is understandable, because julia is not intended to be used that way. You are supposed to "live" inside the repl. However, I prefer tools that are flexible enough that can be used comfortably in non-intended ways.
    - FridgeSeal 7y ago
      That’s compilation time. Currently yeah, it’s not fantastic, but I believe that’s being actively worked on.
- subroutine 7y ago
  Also if you have any need to generate plots & graphs - RIP Julia. I was excited to try Julia, since it seemed to integrate some of the best features of each MATLAB, R, and Python. I was truly disappointed to discover that Julia would take minutes to render the exact same plots I was generating in Octave almost instantly.
  - dnautics 7y ago
    to be fair, that typically only happens the first time you make your graph; replots are typically lightning fast. but this was a major frustration for me, I haven't had occasion to use it more recently, but I understand maybe things are a bit better for graphing in julialand in the last few months?
    - improbable22 7y ago
      I thought it had got better, but also I've just adjusted to work around it. Here are some timings today, Julia 1.1, cold start to first plot: $ julia -e '@time (using GR; plot(rand(20)))' 3.931433 seconds (10.38 M allocations: 521.292 MiB, 6.44% gc time) $ julia -e '@time (using Plots; plot(rand(20)))' 19.498644 seconds (57.26 M allocations: 2.844 GiB, 8.07% gc time) Running with less compilation: $ julia --compile=min -e '@time (using GR; plot(rand(20)))' 0.375836 seconds (368.83 k allocations: 20.190 MiB, 1.65% gc time) $ julia --compile=min -e '@time (using Plots; plot(rand(20)))' 4.302867 seconds (6.41 M allocations: 371.485 MiB, 5.07% gc time) But, as you say, it's much much quicker once started, like 1-5ms per plot.
      - dnautics 7y ago
        4.3 seconds seems great. I remember when it felt like a minute or two.
  - enriquto 7y ago
    this is not really an issue, as long as you ignore the julia plotting capabilities; which should have never been there anyway. You can easily dump your numbers (and functions) on a text file and gnuplot them.

new4thaccount 7y ago

Julia JITs to native LLVM code and is really fast for a lot of use cases. For things where the JIT warming up takes less time than the full job, it can be better than Python. It was also developed with numerical computation in mind, so it was designed for that performance wise and has so many brilliant people working on the language and library. You can use macros for awesome DSLs and run on the GPU and in parallel a lot easier than Python in some cases. It has a first class package manager, great REPL and doesn't need an installer.

patrick5415 7y ago
I didn’t find the REPL that great, because it takes forever to jit anything. My laptop is not that old and creating an array with 5 elements takes over a second. If I type a syntax error, it takes tens of seconds to produce an error. This yields a very frustrating experience and doesn’t lend itself to an effective prototyping environment. For now at least I’ll be sticking with matlab.
- improbable22 7y ago
  There are indeed frustrating lags due to JIT, but they have got better lately & are being worked on -- if I understand right this is now one of the priorities, after focusing on getting the breaking changes done before 1.0. Here's how long a vector of 5 random numbers takes, after a cold start, on 1.1: $ julia -e '@time rand(5)' 0.054343 seconds (121.03 k allocations: 6.219 MiB)
- eigenspace 7y ago
  That sounds really bizarre. What version of which OS are you using? I've heard that older versions of Windows (I think 7) have problems with the REPL that cause extreme latency. I don't think the latency you're experiencing is normal.
  - patrick5415 7y ago
    I was running Debian 9, with Julia from julialang.org (not the Debian repo). This was six months ago so maybe things have improved since then.

eigenspace 7y ago

Well written pure julia programs tend to be about as fast as C, or in other words about 100 times faster than Python or R. Before that gets you too excited or sceptical, let me say that end users will not really see many significant speed boosts. When I say end users, I mean people who are just loading packages and writing scripts where they plug variables into functions from the package. The reason these end users won't see much difference is that a well made Python or R package (including things like Numpy) are actually Python and R wrappers around C, C++, Fortran or Julia code. However, if you are trying to do something that your packages weren't directly designed to do and writing non-trivial code, then you're going to start seeing the speed advantages of Julia. With all that in mind, I'd say the main reason for end users to use Julia is not the speed, it's the expressiveness and the top notch libraries. Expressiveness is kinda a vague concept, but what I'm basically saying is that through things like multiple dispatch and macros, julia is able to provide ways of writing programs in almost any domain that feel incredibly natural and various programs will compose with eachother in ways you won't see in any language except Common Lisp. Finally, because of Julia's native speed and expressiveness, it doesn't get in package developers way like Python and R do and so we're seeing that even though julia has a much smaller community than Python or R, we have a package ecosystem that's quite comparable and in certain specific regions, flat out superior. As adoption grows, we're going to see the julia package ecosystem accelerate and Julia will make sense for more end users. For now, whether or not Julia makes sense for you really depends on what field you're working in (ie. if there are good packages already for the tasks you're interested in) and how advanced a programmer you are / want to be (ie. if you're going to be trying to do something that's not already covered by existing packages).

mehrdadn 7y ago
> However, if you are trying to do something that your packages weren't directly designed to do and writing non-trivial code, then you're going to start seeing the speed advantages of Julia. Or you use Numba. :-) But yeah, the speed can be pretty bad compare to native code if you can't compile everything to native with Numba. To put some concrete numbers here, the one time I had some complicated but heavily-optimized Python and C++ scicomp codebases computing exactly the same thing, I saw [1] running times like the following depending on the algorithm: 1. 10.6s with NumPy, 7.5s with NumPy + Numba, and 0.6s in direct C++ (1.4x and 12.5x respectively) 2. 68s with NumPy, 6.5s with NumPy + Numba, 0.5s in direct C++ (10x and 13x respectively) The first algorithm was simpler and more vectorizable, while the second algorithm was more complicated and less vectorizable but with a lower time complexity. In either case, the algorithm still had to do a fair bit of work in Python, so it wasn't running native code 100% of the time (which I think is fairly realistic). In either case, you can see the difference between C++ and Python was around 13x when using Numba... not quite as bad as 100x, but I imagine Julia probably does better. [1] Actually, originally I saw only a 5x improvement with Python 2.7 on my older laptop, but I re-ran these on my current laptop under Python 3.7 and the difference is 13x now.

adamnemecek 7y ago

The whole development experience is so much better than Python.

agumonkey 7y ago
any video or article about it ?
FridgeSeal 7y ago
Right?! Not having a packaging system that is a stapled-on-afterthought is so nice.

dnautics 7y ago

if you want the flexibility to do something truly bizzare with confidence, you really ought to use Julia, and don't look back. A while back I was prototyping let's just say... unusual binary datatype representations for numbers. All i had to do was reimplement a handful of operations (+, -, x, /, one, zero) and I got everything from fourier transforms to matrix solving for free. Comparing numerical performance with standard IEEE representations was then easy, and I had confidence that my comparisons were legit, since I was literally calling the same function against both numerical types. More recently I wanted to play around with galois fields, following Mary Wootter's impressive work, and was able to test some ideas very quickly (and trivially deploy on a supercomputer cluster) in few lines of code using Julia. That's not a typical use case, but it's a thing. I am thinking about playing around with complex numbers (when I get some free time, which increasing seems like 'never') in deep learning, and for similar reasons Julia will be an obvious choice.

dnautics 7y ago
I should probably post evidence; this was a live demo I did of the floating point stuff at Stanford (demo begins ~53 minutes in) https://youtu.be/aP0Y1uAA-2Y https://youtu.be/aP0Y1uAA-2Y

throw20102010 7y ago

You should use Julia over Python because of the JIT compilation makes it faster, as long as you aren’t constantly writing code and running it a single time. Also as long as you are able to keep your REPL open for days at a time. It takes several minutes to load some popular plotting packages in Julia (because of the JIT), so it will take you some time to adjust your workflow to leaving your REPL open all the time. But the Juno IDE is just as good as Jupyter notebooks or RStudio, and there are probably high quality libraries for whatever you’re doing, unless you work in one of those niche areas.

ThenAsNow 7y ago

I just wrote my first bit of Julia this weekend. I'm impressed - it's clearly been designed for technical/numeric computing with modern language features baked-in. The type system and macros are some significantly distinguishing characteristics as compared to Python. The Unitful.jl package illustrates how these can be used to powerful effect. Multiple dispatch is another such characteristic. Having benefited from static type systems in other software projects, I would never again want to write significant technical code without a first-class type system supporting declarative type constraints. If I could never write Matlab again and instead use Julia, I'd be delighted.

thetwentyone 7y ago

I've moved all of my development from Python to Julia - The syntax is similar, but I think Julia's is more expressive and powerful once you start looking into the details. Macros, broadcasting (automatic vectorization), lambda functions, string interpolation, 1-based indexing (which weirded me out at first but I found to be more natural for most things I work on). It's been an almost wholly positive transition.

pjmlp 7y ago

No need for a second language, you can write everything in Julia, while enjoying a mix of productivity and code execution performance. Plus it has quite a few Lisp inspired features, like multi-methods and powerful macros.