4 ms·

First time I've heard anything like that. Most people report a massive speedup without doing anything special. Maybe try the current version?

cbkeller 6y ago
I had an experience slightly like that at first where I was surprised to only be going 1-2x faster than my old MATLAB, but then I realized my code was full of trivially avoidable type instabilities and got another 100x speedup in Julia.
zetalemur 6y ago
Probably because it is mostly compared to Python - which is arguably a low bar to beat.
- ddragon 6y ago
  Python is a high bar to beat though, since no one understands beating Python as beating some hypothetical Python-only numerical library, it's beating mature precompiled C/C++/Fortran libraries with a small overhead in Python. And there are very few dynamic languages that can even compete on this level (without needing the FFI), and even static languages will have trouble dealing with the level of optimization that went on those libraries. But everything has compromises, and in Julia case is the JIT lag from aggressively optimizing compilation, making it a slow language for simple do once tasks (which is the part that is usually written in Python) and a fast language for when runtime performance is the most important (which is the part done by the libraries in C/C++/Fortran), making it slower in some situations. Though new releases are attacking this issue, including the ability to not optimize this do once tasks (like plotting) and improving pre-compiling. But as for now, the ability to not have black boxes and not being required to twist my code to a particular style in order to not be prohibitive slow makes up for the slow startup (plus using Revise.jl to automatically compile code in a REPL, and keep the session alive for all my programming routine makes it not that much of a deal during development).
rich_sasha 6y ago
FWIW that was my experience too. And the JIT was too slow for my pace of code changes. Also really wanted to love it. Maybe I need to try the new version.
- socialdemocrat 6y ago
  You need to start using Revise.jl that is the key to using Julia effectively. If the JIT is too slow then, then I really wonder what you are doing. I have no performance issues but on a 5 year old computer, apart from the famous first time to plot or bringing in a whole new library. I don’t have issues with modifying my own code however when using Revise.jl
datanecdote 6y ago
Let me second GP’s sentiment. I find Julia really slow for my purposes. I don’t know his reasoning, but I will explain mine. None of this is surprising and is oft discussed. Julia (at least by default) is constantly recompiling everything. This is a huge pain in a REPL style setup where you want to tweak one thing and see the changes, again and again. I know the Julia ecosystem is working on better caching etc to fix this problem but it’s a problem. Also, despite the marketing claims around the language, expertly crafted C usually beats Julia in performance. So if your “python” program is spending most of its time in Numpy/PyTorch/etc, it will beat Julia, unless you’re writing a fancy “put a differential equation in a neural network in a Bayesian Monte Carlo” program that benefits from cross compiling across specialized libraries. Finally, the Julia libraries are just not as mature as python’s. Armies of developers and larger armies of users have battle tested and perfected python’s crown jewel libraries over many years. Often when someone posts a bad benchmark to the Julia forums they can “fix” it in the library implementation, proving the correctness of the theoretical case for Julia. But in reality many such problems remain to be fixed. Julia is really cool and does have many inherent advantages over python. But it’s not the silver bullet many of its proponents suggest it to be. At least not yet. Every few years I check out Julia and I hope one day it does become that perfect language. I think it will. I just fear it will take longer than many others hope.
- timholy 6y ago
  I appreciate your well-balanced critique, thanks. > Julia (at least by default) is constantly recompiling everything. This is a huge pain in a REPL style setup where you want to tweak one thing and see the changes, again and again. I know the Julia ecosystem is working on better caching etc to fix this problem but it’s a problem. Maybe try Revise.jl? There are a few changes it can't handle, but you can do a lot of development without ever restarting. (Disclaimer: I'm its main author.) > expertly crafted C usually beats Julia in performance This isn't generically true, and there are now quite a few examples of the converse. I linked to it above as well, but check out the benchmarks in LoopVectorization's documentation (https://chriselrod.github.io/LoopVectorization.jl/latest/examples/matrix_multiplication/#Matrix-Multiplication-1 https://chriselrod.github.io/LoopVectorization.jl/latest/exa...) for examples of beating MKL, one of the most carefully-engineered libraries in existence. I think an exciting area of growth for Julia will be exploiting the fact that Julia's compiler, written mostly in Julia, is more "morphable" than most and may develop its own plug-in architecture. This seems likely to provide performance opportunities that many fields seem hungry for. > the Julia libraries are just not as mature as python’s On balance I agree. While there are already many examples where Julia makes things easier than Python, as of today there are many more examples to the contrary. Julia's libraries are advancing rapidly, but I expect it will take a few more years of development until it's no longer so one-sided.
teorema 6y ago
My experience with Julia is that for many (most?) things, you do get a free speedup basically. With other things though, it's about the same. What this means in practice is that if you expand your use cases enough, and integrate enough libraries, you'll incorporate some bottleneck that slows it down. This is true of a lot of languages, python included, but I think when part of the language's selling point is something like "c-like speed with python-like syntax" it can be a little (although not entirely) misleading. Having said that, I still prefer Julia over python (at least for numerical computing, not so sure about other things). I just like the language more. I also think, with some exceptions, that I don't have the same dependency hell problems that I've run into with python. Even now, I'm in the process of switching over to Julia from python for a project because the python library I'm using depends on about 6 different other libraries, but only specific versions, that you have to run in a specific standalone conda environment to avoid using the wrong combination of packages, all of which are pre-python3, and so forth and so on. Even then, when you manage to thread the needle, it still falls apart later for unknown reasons. This is surely this particular package, but my experience with Julia is that things are much cleaner (R is similarly problem-free usually but it's a lot slower and Julia as a language is more coherent to me). I'd really prefer something like Nim to be seeing the attention that Julia is getting, something more general-purpose, but there's no consensus of momentum around something like that at the moment. Maybe in the near future ocaml will pick up steam, or maybe the next version of C++ will essentially make it look like python, or maybe there will be something not quite on people's radar at the moment, but at the moment it is what it is.

cbkeller 6y ago

I had an experience slightly like that at first where I was surprised to only be going 1-2x faster than my old MATLAB, but then I realized my code was full of trivially avoidable type instabilities and got another 100x speedup in Julia.

zetalemur 6y ago

Probably because it is mostly compared to Python - which is arguably a low bar to beat.

ddragon 6y ago
Python is a high bar to beat though, since no one understands beating Python as beating some hypothetical Python-only numerical library, it's beating mature precompiled C/C++/Fortran libraries with a small overhead in Python. And there are very few dynamic languages that can even compete on this level (without needing the FFI), and even static languages will have trouble dealing with the level of optimization that went on those libraries. But everything has compromises, and in Julia case is the JIT lag from aggressively optimizing compilation, making it a slow language for simple do once tasks (which is the part that is usually written in Python) and a fast language for when runtime performance is the most important (which is the part done by the libraries in C/C++/Fortran), making it slower in some situations. Though new releases are attacking this issue, including the ability to not optimize this do once tasks (like plotting) and improving pre-compiling. But as for now, the ability to not have black boxes and not being required to twist my code to a particular style in order to not be prohibitive slow makes up for the slow startup (plus using Revise.jl to automatically compile code in a REPL, and keep the session alive for all my programming routine makes it not that much of a deal during development).

rich_sasha 6y ago

FWIW that was my experience too. And the JIT was too slow for my pace of code changes. Also really wanted to love it. Maybe I need to try the new version.

socialdemocrat 6y ago
You need to start using Revise.jl that is the key to using Julia effectively. If the JIT is too slow then, then I really wonder what you are doing. I have no performance issues but on a 5 year old computer, apart from the famous first time to plot or bringing in a whole new library. I don’t have issues with modifying my own code however when using Revise.jl

datanecdote 6y ago

Let me second GP’s sentiment. I find Julia really slow for my purposes. I don’t know his reasoning, but I will explain mine. None of this is surprising and is oft discussed. Julia (at least by default) is constantly recompiling everything. This is a huge pain in a REPL style setup where you want to tweak one thing and see the changes, again and again. I know the Julia ecosystem is working on better caching etc to fix this problem but it’s a problem. Also, despite the marketing claims around the language, expertly crafted C usually beats Julia in performance. So if your “python” program is spending most of its time in Numpy/PyTorch/etc, it will beat Julia, unless you’re writing a fancy “put a differential equation in a neural network in a Bayesian Monte Carlo” program that benefits from cross compiling across specialized libraries. Finally, the Julia libraries are just not as mature as python’s. Armies of developers and larger armies of users have battle tested and perfected python’s crown jewel libraries over many years. Often when someone posts a bad benchmark to the Julia forums they can “fix” it in the library implementation, proving the correctness of the theoretical case for Julia. But in reality many such problems remain to be fixed. Julia is really cool and does have many inherent advantages over python. But it’s not the silver bullet many of its proponents suggest it to be. At least not yet. Every few years I check out Julia and I hope one day it does become that perfect language. I think it will. I just fear it will take longer than many others hope.

timholy 6y ago
I appreciate your well-balanced critique, thanks. > Julia (at least by default) is constantly recompiling everything. This is a huge pain in a REPL style setup where you want to tweak one thing and see the changes, again and again. I know the Julia ecosystem is working on better caching etc to fix this problem but it’s a problem. Maybe try Revise.jl? There are a few changes it can't handle, but you can do a lot of development without ever restarting. (Disclaimer: I'm its main author.) > expertly crafted C usually beats Julia in performance This isn't generically true, and there are now quite a few examples of the converse. I linked to it above as well, but check out the benchmarks in LoopVectorization's documentation (https://chriselrod.github.io/LoopVectorization.jl/latest/examples/matrix_multiplication/#Matrix-Multiplication-1 https://chriselrod.github.io/LoopVectorization.jl/latest/exa...) for examples of beating MKL, one of the most carefully-engineered libraries in existence. I think an exciting area of growth for Julia will be exploiting the fact that Julia's compiler, written mostly in Julia, is more "morphable" than most and may develop its own plug-in architecture. This seems likely to provide performance opportunities that many fields seem hungry for. > the Julia libraries are just not as mature as python’s On balance I agree. While there are already many examples where Julia makes things easier than Python, as of today there are many more examples to the contrary. Julia's libraries are advancing rapidly, but I expect it will take a few more years of development until it's no longer so one-sided.

teorema 6y ago

My experience with Julia is that for many (most?) things, you do get a free speedup basically. With other things though, it's about the same. What this means in practice is that if you expand your use cases enough, and integrate enough libraries, you'll incorporate some bottleneck that slows it down. This is true of a lot of languages, python included, but I think when part of the language's selling point is something like "c-like speed with python-like syntax" it can be a little (although not entirely) misleading. Having said that, I still prefer Julia over python (at least for numerical computing, not so sure about other things). I just like the language more. I also think, with some exceptions, that I don't have the same dependency hell problems that I've run into with python. Even now, I'm in the process of switching over to Julia from python for a project because the python library I'm using depends on about 6 different other libraries, but only specific versions, that you have to run in a specific standalone conda environment to avoid using the wrong combination of packages, all of which are pre-python3, and so forth and so on. Even then, when you manage to thread the needle, it still falls apart later for unknown reasons. This is surely this particular package, but my experience with Julia is that things are much cleaner (R is similarly problem-free usually but it's a lot slower and Julia as a language is more coherent to me). I'd really prefer something like Nim to be seeing the attention that Julia is getting, something more general-purpose, but there's no consensus of momentum around something like that at the moment. Maybe in the near future ocaml will pick up steam, or maybe the next version of C++ will essentially make it look like python, or maybe there will be something not quite on people's radar at the moment, but at the moment it is what it is.