8 ms·

The improvements are not minor they are massive. Citing rust shows that the advantage of Julia has not been explained well enough. Julia allows you to write as

The improvements are not minor they are massive. Citing rust shows that the advantage of Julia has not been explained well enough. Julia allows you to write as performant code as Rust with a much smaller investment in learning. You cite your concern for spending time learning something new. That makes no sense considering the high learning curve and complexity of Rust compared to Julia. Julia is quite fast to learn and has far more overlap with Python than Rust to make the transition easy. You can call python code from Julia and ise many of the same tools like the notebook.

88e282102ae2e5b 9y ago
Is Julia really that much more performant than numpy?
- wallnuss 9y ago
  In my opinion that is the wrong question to ask. The right question is: How fast will my code be. Numpy has been heavily optimised and is written in C and not Python. Take a look at the link below, comparing a simple sum in different languages (among them Python and Numpy). The power of Julia is that there is no privileged code. Your code will be as fast as the base library. http://nbviewer.jupyter.org/github/alanedelman/18.337_2017/blob/282c25c9229187b18a7c61e7167387c31f40f15e/lectures/Lecture01_0906%20AutoDiff%20%26%20JuliaIsFast/JuliaIsFast.ipynb http://nbviewer.jupyter.org/github/alanedelman/18.337_2017/b...
  - 81212w1 9y ago
    I find this benchmark a bit disorganized, but it seems that the "hand-written" Julia function is as slow as the "hand-written" C function. The C function is of course naive. I'm pretty sure that a hand-unrolled loop would be faster.
    - wallnuss 9y ago
      You are absolutely right, an optimised C Programn would be as fast or faster than the Numpy implementation.
  - improbable22 9y ago
    Yes to this. When you're calling some big fast package (be it some field-specific thing, or just matrix algebra) the language you're in doesn't matter that much. In terms of speed, but also in terms of effort -- you spend most of your time reading this field-specific package's manual. What's liberating is that for the things which aren't covered by such packages, there's no huge penalty for just writing it yourself. In speed but in effort too -- it's quicker to write the loop than to figure out how to make np.einsum do what you have in mind.
vvanders 9y ago
Have a citation on the performance vs Rust? I'm skeptical any GC'd language can approach C/Rust unless they have explicit mechanisms to do data layout for using the cache/prefetcher to the fullest degree.
- faitswulff 9y ago
  Julia's website has some benchmarks: https://julialang.org/benchmarks/ https://julialang.org/benchmarks/ C is the leftmost dot, Julia is just to the right.
  - vvanders 9y ago
    According to that JavaScript is faster than both Julia and C by a significant margin in iter_mandelbrot ;).
    - 3JPLW 9y ago
      Yup, it's really amazing what millions of hours of development time will do! That benchmark in particular is a little deceiving — and we take those benchmarks very seriously! See this issue for more details: https://github.com/JuliaLang/julia/issues/24097 https://github.com/JuliaLang/julia/issues/24097
  - pollitos 9y ago
    Just to note, these benchmarks are biased. This is a competition between highly-tuned-to-the-processor Julia with tuned BLAS vs other languages out-of-the-box binaries. Additionally, the benchmarks tests are written inefficiently in the comparison languages. But even then Javascript comes out on top on some benchmark tests.
    - dnautics 9y ago
      That's because the jit compiler in the Julia standard library selects the blas for you. As a programmer you don't have to optimize beyond using idiomatic type strategies. You could, say write a galois field type and use the \ solve operator to solve a Reed Solomon Erasure using old fashioned lu decomposition (automatically without pivoting, even), and then turn around and use the same \ operator with floats one line down and it will give you highly optimized blas.
Thaxll 9y ago
Performance is not everything, people are obsessed with it and it's a mistake. - libraries - community - tools Are much more important.
- astrobe_ 9y ago
  You're overlooking the context, that's a bigger mistake.
quintessex 9y ago
I've been using Julia for a year or two now, and over that time my enthusiasm for the language has waned, not grown. Here's the standard pitch for Julia: C-like performance with R/Python/MATLAB-like expressivity. This is undeniably true. For someone coming from R/Python/MATLAB, you will be able to program as easily in Julia, if not more easily and more cleanly, and get much better speed. However, what I've come to realize now is that you can get something similar from Kotlin, Nim, and Crystal, but those are much more general-purpose, and often are more consistent in their performance. They don't have the same numerical libraries at the moment, but they have a much bigger use-case target, and for something like Kotlin, a lot more leverage from the community. Rust is a lot lower-level, but as the numerical resources grow (and they inevitably will) and layers of abstraction are added, what you will probably get is something very similar to Julia in expressivity but more extensible downward, and probably more performant. Rust has a ton of room to grow, and has many good role models from C (e.g., Eigen). This also isn't even touching on PyPy and things of that sort. I like Julia and still reach for it when I'm starting a project. Often it's worked great, but other times I have to go back to R or Python for some library that calls C, because the Julia library, while usable in a way that pure R or Python isn't, is still not actually as fast as C. Sometimes "on the order of magnitude of C" is not actually good enough, and you need C. Also, if you poke around for benchmarks that aren't posted by the Julia developers themselves, you'll see different conclusions about the real-world performance of Julia, so there's that. I guess I read articles like the linked post, and I end up feeling like they're misleading. Is Julia better for data analysis than R or Python? Yes, almost certainly if you were starting from scratch and not calling any products of outside languages. Is Julia better than R or Python + C as it is in the real world? Not sure, because you still have to call C anyway at some point if you're being honest. Is it better than Kotlin, Nim, or Rust? I'm even less sure. If Julia wants to maintain long-term competitiveness, it has to reach out in the opposite direction of those languages, from numerics -> general purpose computing, because they are inevitably moving from general purpose computing -> numerics, and they are bringing a lot with them.
- chubot 9y ago
  What problems are you solving with Julia? The way I see it, Kotlin, Nim, and Crystal are completely different languages aimed at a different space. e.g. Kotlin is being used for Android apps, which is worlds away from what MATLAB and R users do.

88e282102ae2e5b 9y ago

Is Julia really that much more performant than numpy?

wallnuss 9y ago
In my opinion that is the wrong question to ask. The right question is: How fast will my code be. Numpy has been heavily optimised and is written in C and not Python. Take a look at the link below, comparing a simple sum in different languages (among them Python and Numpy). The power of Julia is that there is no privileged code. Your code will be as fast as the base library. http://nbviewer.jupyter.org/github/alanedelman/18.337_2017/blob/282c25c9229187b18a7c61e7167387c31f40f15e/lectures/Lecture01_0906%20AutoDiff%20%26%20JuliaIsFast/JuliaIsFast.ipynb http://nbviewer.jupyter.org/github/alanedelman/18.337_2017/b...
- 81212w1 9y ago
  I find this benchmark a bit disorganized, but it seems that the "hand-written" Julia function is as slow as the "hand-written" C function. The C function is of course naive. I'm pretty sure that a hand-unrolled loop would be faster.
  - wallnuss 9y ago
    You are absolutely right, an optimised C Programn would be as fast or faster than the Numpy implementation.
- improbable22 9y ago
  Yes to this. When you're calling some big fast package (be it some field-specific thing, or just matrix algebra) the language you're in doesn't matter that much. In terms of speed, but also in terms of effort -- you spend most of your time reading this field-specific package's manual. What's liberating is that for the things which aren't covered by such packages, there's no huge penalty for just writing it yourself. In speed but in effort too -- it's quicker to write the loop than to figure out how to make np.einsum do what you have in mind.

vvanders 9y ago

Have a citation on the performance vs Rust? I'm skeptical any GC'd language can approach C/Rust unless they have explicit mechanisms to do data layout for using the cache/prefetcher to the fullest degree.

faitswulff 9y ago
Julia's website has some benchmarks: https://julialang.org/benchmarks/ https://julialang.org/benchmarks/ C is the leftmost dot, Julia is just to the right.
- vvanders 9y ago
  According to that JavaScript is faster than both Julia and C by a significant margin in iter_mandelbrot ;).
  - 3JPLW 9y ago
    Yup, it's really amazing what millions of hours of development time will do! That benchmark in particular is a little deceiving — and we take those benchmarks very seriously! See this issue for more details: https://github.com/JuliaLang/julia/issues/24097 https://github.com/JuliaLang/julia/issues/24097
- pollitos 9y ago
  Just to note, these benchmarks are biased. This is a competition between highly-tuned-to-the-processor Julia with tuned BLAS vs other languages out-of-the-box binaries. Additionally, the benchmarks tests are written inefficiently in the comparison languages. But even then Javascript comes out on top on some benchmark tests.
  - dnautics 9y ago
    That's because the jit compiler in the Julia standard library selects the blas for you. As a programmer you don't have to optimize beyond using idiomatic type strategies. You could, say write a galois field type and use the \ solve operator to solve a Reed Solomon Erasure using old fashioned lu decomposition (automatically without pivoting, even), and then turn around and use the same \ operator with floats one line down and it will give you highly optimized blas.

Thaxll 9y ago

Performance is not everything, people are obsessed with it and it's a mistake. - libraries - community - tools Are much more important.

astrobe_ 9y ago
You're overlooking the context, that's a bigger mistake.

quintessex 9y ago

I've been using Julia for a year or two now, and over that time my enthusiasm for the language has waned, not grown. Here's the standard pitch for Julia: C-like performance with R/Python/MATLAB-like expressivity. This is undeniably true. For someone coming from R/Python/MATLAB, you will be able to program as easily in Julia, if not more easily and more cleanly, and get much better speed. However, what I've come to realize now is that you can get something similar from Kotlin, Nim, and Crystal, but those are much more general-purpose, and often are more consistent in their performance. They don't have the same numerical libraries at the moment, but they have a much bigger use-case target, and for something like Kotlin, a lot more leverage from the community. Rust is a lot lower-level, but as the numerical resources grow (and they inevitably will) and layers of abstraction are added, what you will probably get is something very similar to Julia in expressivity but more extensible downward, and probably more performant. Rust has a ton of room to grow, and has many good role models from C (e.g., Eigen). This also isn't even touching on PyPy and things of that sort. I like Julia and still reach for it when I'm starting a project. Often it's worked great, but other times I have to go back to R or Python for some library that calls C, because the Julia library, while usable in a way that pure R or Python isn't, is still not actually as fast as C. Sometimes "on the order of magnitude of C" is not actually good enough, and you need C. Also, if you poke around for benchmarks that aren't posted by the Julia developers themselves, you'll see different conclusions about the real-world performance of Julia, so there's that. I guess I read articles like the linked post, and I end up feeling like they're misleading. Is Julia better for data analysis than R or Python? Yes, almost certainly if you were starting from scratch and not calling any products of outside languages. Is Julia better than R or Python + C as it is in the real world? Not sure, because you still have to call C anyway at some point if you're being honest. Is it better than Kotlin, Nim, or Rust? I'm even less sure. If Julia wants to maintain long-term competitiveness, it has to reach out in the opposite direction of those languages, from numerics -> general purpose computing, because they are inevitably moving from general purpose computing -> numerics, and they are bringing a lot with them.

chubot 9y ago
What problems are you solving with Julia? The way I see it, Kotlin, Nim, and Crystal are completely different languages aimed at a different space. e.g. Kotlin is being used for Android apps, which is worlds away from what MATLAB and R users do.