20 ms·

My favorite things that are coming with Julia 1.0

227 points by oxinabox 8y ago

noobermin 8y ago
I'm surprised the author didn't touch on the massive changes to Pkg. Now Pkg is a lot more like npm, allowing for "project" level packages rather than requiring global packages.
- oxinabox 8y ago
  Good point, that is huge, I was intending to mention it as the start (and have done so now). It like the getfield overloading and a few other things really deserve a post of their own, so I wasn't going to try and push them into this one.
- photon-torpedo 8y ago
  Also the new Pkg is massively faster.
- ScottPJones 8y ago
  He explained that at the beginning of his post - things like the Pkg changes are getting quite a lot of press already, he wanted to highlight the things that people might not have realized were coming in v0.7/v1.0.
_jordan 8y ago
Phenomenal language. compiles applications to a single binary which makes it super convenient to move around environments
- oblio 8y ago
  How? Everything I've read about Julia seems to show that you're supposed to pass the source code files around. https://medium.com/@sdanisch/compiling-julia-binaries-ddd6d4e0caf4 https://medium.com/@sdanisch/compiling-julia-binaries-ddd6d4... -> notice that's a third party package.
  - ninjin 8y ago
    Indeed, which comes with its own advantages and disadvantages. As someone that has to always care about performance, Machine Learning, I am very happy not to have to worry about dependencies not being compiled to take advantage of the latest and greatest CPU instructions on a specific box in a very heterogeneous cluster. Now, you could just compile it all on each deployment, but something as large as say TensorFlow takes ages and not everyone gets to enjoy having great devops at your beck and call.
    - oblio 8y ago
      I'm not sure I understand what you mean: you prefer compilation on the target machines or you prefer a static binary?
      - ninjin 8y ago
        Compilation on the target machine, in my case. Performance trumps portability for me, but I can totally see why this is not the case for everyone.
        oblio 8y ago
        In theory that can be solved with the best of both worlds: have the runtime and JIT in the final binary. That way you still get easy deployment with 1 big file and high performance after the JIT warms up, which should be fast. I think it's just a problem of manpower on the Julia developer side.
        ninjin 8y ago
        I wholeheartedly agree, one could also note how this is Java-esque. My hope is that someone far more well-versed than me in this area will find the time and push for it. On that note, one wonderful thing about Julia is how easy it is to contribute to the core language since it is written in Julia. It really is amazing to see people from so many disciplines come together to push a “language for science as a whole”.
parvenu74 8y ago
What is the target market or prime use case for Julia? What makes it better than alternatives? I've _heard_ of this language but haven't studied it; I'm trying to understand if this is more of a niche language or something which could see more mainstream adoption like Go or Rust.
- oblio 8y ago
  It's meant to be primarily a scientific language and the developers are primarily focusing on this. Post 1.0 who knows? It definitely looks like something which could be used as a general purpose programming language.
- aschampion 8y ago
  It's mainly targeting MATLAB, and to a slightly lesser extent scientific and numeric programming in Python and R. It's a well thought out language that allows you to write MATLAB-like high level code with an easy gradient for progressive typing and optimization to near C level performance. It's a bit harder to sell versus Python, since Python has enormous value in the ecosystem, community, and ubiquity. Also, because it primary targets MATLAB a lot of the standard libraries try to have similar ergonomics, which is a bit of a waste of a great tool to recreate a poor interface.
  - fareesh 8y ago
    I often wonder what inspires folks to start from scratch in the face of a gigantic ecosystem like the one that Python brings with it, which will also keep improving.
    - 3JPLW 8y ago
      In the case of Julia, you can check out the original motivation back in 2012 or this recent answer on the message board. https://julialang.org/blog/2012/02/why-we-created-julia https://julialang.org/blog/2012/02/why-we-created-julia https://discourse.julialang.org/t/julia-motivation-why-werent-numpy-scipy-numba-good-enough/2236/10 https://discourse.julialang.org/t/julia-motivation-why-weren...
twic 8y ago
> It will be interesting to see exactly how nothing and missing play out in-practice. I feel like if all things go well, then in most circumstances operating with a missing should return a missing (i.e. properation), and operating with a nothing should throw an error (Much like a NullReferenceError). We all know that having a null value in your language is a mistake, but i have to admit that it had not occurred to me that the fix was to have two null values.
- masklinn 8y ago
  > We all know that having a null value in your language is a mistake, but i have to admit that it had not occurred to me that the fix was to have two null values. Javascript is even better as it has 3 (null, undefined and NaN, and while NaN is available in most every language it is pretty common in JS as its conversion APIs tend to return NaN instead of error).
  - deleted 8y ago
    [deleted]
  - petre 8y ago
    How is 3 null values better than one? Fortunately Dart has fixed this nonsense.
    - jimbokun 8y ago
      (That's the joke.)
- zimablue 8y ago
  The root of "null is a mistake" isn't that null is a mistake, it's that it's a mistake to allow null into a type system in such a way that it can cause non-compile-time type errors. So Julia is typed ( I think) but it's not a mistake in any way to have null in a dynamic language by this argument. Null is a thing and if you don't have it you reinvent it, it's the typesystem breaking that's the problem, maybe monads effectively have nulls it's just that they then get protected by the typesystem.
jb1991 8y ago
Honest question, just how fast is Julia anyway? Because performance is typically the main reason anyone would want to use it over alternatives. A lot of people justify Julia because, as noted in other comments, it is both high-level yet claims to be nearly as fast as C: > "Julia is the fastest modern open-source language for data science, machine learning and scientific computing...with the speed, capacity and performance of C, C++..." [4] That's a bold claim! And there is much evidence to the contrary. I often find that the language falls into a similar trap of many other languages where the creators are evangelists not quite sharing the whole picture. Writing fast Julia code is not always a pleasant experience as you often need to fight the easier idioms. It is marketed as fast, but really, how fast is it? [0] http://www.zverovich.net/2016/05/13/giving-up-on-julia.html http://www.zverovich.net/2016/05/13/giving-up-on-julia.html "What’s disappointing is the striking difference between the claimed performance and the observed one. For example, a trivial hello world program in Julia runs ~27x slower than Python’s version and ~187x slower than the one in C." [1] https://www.ibm.com/developerworks/community/blogs/jfp/entry/A_Comparison_Of_C_Julia_Python_Numba_Cython_Scipy_and_BLAS_on_LU_Factorization?lang=en https://www.ibm.com/developerworks/community/blogs/jfp/entry... "We can code in Julia in a way similar to Python's code. However, that code is slower than it should. One way to speed Julia is to take into account the Fortran ordering it uses by looping on j before looping on i in the second loop. We also add decorators to speed up the code." [2] https://www.codementor.io/zhuojiadai/julia-vs-r-vs-python-string-sort-performance-an-unfinished-journey-to-optimizing-julia-s-performance-f57tf9f8s https://www.codementor.io/zhuojiadai/julia-vs-r-vs-python-st... "...comparing R's sorting speeds to Julia's is not the complete story, even though on the surface R appears faster, and from a users' perspective, (once the data is loaded) R is still the king of speed." Stackoverflow has quite a few user posts [3] from folks trying to get their code to perform at the advertised speed vs popular alternative languages. It's not so easy. [3] https://stackoverflow.com/questions/20613817/julia-julia-lang-performance-compared-to-fortran-and-python https://stackoverflow.com/questions/20613817/julia-julia-lan... [4] https://juliacomputing.com/ https://juliacomputing.com/
- montalbano 8y ago
  Regarding [0], startup time is a commonly acknowledged issue which affects the 'hello world' benchmark. (Can you even call it a benchmark? Maybe if startup time is what you're comparing then it makes sense, other wise I don't agree.) I used to call Fortran code from Python for a particularly complex function (Mittag-Leffler function) from Python, I then rewrote everything in Julia and it runs approx 10-20x faster. So there you go, I'm a primary source. Edit: just for clarity: The Mittag-Leffler function itself runs just ever so slightly faster in Julia than Fortran-called-from-Python (due to the probably inefficient way I was sending and receiving the data to/from a Python subprocess, which itself is an example of the difficulty of combining two languages). It is the program as a whole which runs 10-20x faster.
wirrbel 8y ago
Julia always was one of the languages I wondered whether I should have a closer look at (do python data-science stuff mostly, so it is an interesting project). However, reading Dan Luu writing about his experiences with members of the Julia community https://danluu.com/julialang/ https://danluu.com/julialang/ I try my best to not get anywhere near it. Directly cited: Update: this post was edited a bit to remove a sentence about how friendly the Julia community is since that no longer seemed appropriate in light of recent private and semi-private communications from one of the co-creators of Julia. They were, by far, the nastiest and most dishonest responses I've ever gotten to any blog post. Some of those responses were on a private discussion channel; multiple people later talked to me about how shocked they were at the sheer meanness and dishonesty of the responses. Oh, and there's also the public mailing list. The responses there weren't in the same league, but even so, I didn't stick around long since I unsubscribed when one the Julia co-creators responded with something bad enough that it prompted someone else to to suggest sticking to the facts and avoiding attacks. That wasn't the first attack, or even the first one to prompt someone to respond and ask that people stay on topic; it just happened to be the one that made me think that we weren't going to have a productive discussion. I extended an olive branch before leaving, but who knows what happened there? Update 2, 1 year later: The same person who previously attacked me in private is now posting heavily edited and misleading excerpts in an attempt to discredit this post. I'm not going to post the full content in part because it's extremely long, but mostly because it's a gross violation of that community's norms to post internal content publicly. If you know anyone in the RC community who was there for the discussion before the edits and you want the truth, ask your RC buddy for their take. If you don't know any RC folks, consider that my debate partner's behavior was so egregious that multiple people asked him to stop, and many more people messaged me privately to talk about how inappropriate his behavior was. If you compare that to what's been publicly dredged up, you can get an idea of both how representative the public excerpts are and of how honest the other person is being._
- hellofunk 8y ago
  I've seen that post as well, and often wondered if it was fair characterization of overall Julia community (maybe it is), or is the result of a single bad apple at the top that overshadowed an otherwise pleasant community. Unfortunately, it doesn't take much negative energy to spoil a community, even if it comes from just one person. That it was a language co-creator is troubling but not necessarily a reason to avoid the language if the rest of the community is nice (which maybe it is, maybe it isn't). However one thing I have noticed is that most language communities do tend to follow an attitude set by the language creator(s). This seems to have played out quite a bit for Clojure, Python, Elm, Elixir, and other languages where, at least to me, the overall shared perspective of the community is closely aligned to the personal attitudes and opinions of the language author(s).
sidcool 8y ago
And we are struggling to connect Julia to Postgres DB.
- StefanKarpinski 8y ago
  Have you looked at https://github.com/invenia/LibPQ.jl https://github.com/invenia/LibPQ.jl? The folks at Invenia are very sharp and generally produce excellent, reliable packages, so I would expect this to be of high quality. The primary author, Eric Davies, in particular, is a well known high caliber Julia contributor :)
  - sidcool 8y ago
    I am using it, but it's quite inconvenient. The return types are wrong. Documentation is sparse. I would like contribute to it.
    - 3JPLW 8y ago
      Please do! Not only are the folks at Invenia technically talented, they're also really kind and wonderful. I'm sure I'm not speaking out of turn to say that I think they would very much appreciate bug reports, API feedback, and especially doc and code contributions!
      - sidcool 8y ago
        I have started with documentation and coffee samples that I m trying. Let's see where it goes with code contribution. I am especially trying to efficiently convert the hideous NamedTuples to DataFrames.
d--b 8y ago
Where are we compared to then (http://www.zverovich.net/2016/05/13/giving-up-on-julia.html http://www.zverovich.net/2016/05/13/giving-up-on-julia.html)?
- eigenspace 8y ago
  Theres a discussion on that post in another thread below, but suffice to say the julia community is pretty exasparated with the constant need to respond to that post when its main source of the speed claims was to measure the perofrmance of a hello world program in julia and included the compilation and startup time in the benchmark. Thats a deceptive and unhelpful 'benchmark', especially when the C++ compilation time was not included in its benchmark. Julia compilation and statup times are slow if you are expecting the feel of the Python interpreter, however, julia's compilation and statup time add a finite, constant overhead to its preformance and so are completely irrelevant for high performance numrical computing. If you were wanting to spawn many julia instances and make them execute a single command then exit, yes julia would be a terrible choice of that (unless you turn of compilation and use its interpreter or statically compile your program) but for the most part measuring startup time just isn't relevant unless it takes a minute or something absurd. In julia the only libraries I know of that take a minute to load are plotting libraries (once you make your first plot the library is blazing fast again) and thats been considered such a big issue that a new plotting library Makie is well under way which is supposed to be fully statically compiled in julia in order to slash the first time to plot.
mafm 8y ago
Upvote because I think this is the 1st time a posting from the ucc website has made the HN front page.
hellofunk 8y ago
Would anyone recommend using Julia for things like web servers? Python is used for web servers, and is slow. But Julia doesn’t really seem marketed to that application domain, despite that I would expect it to do a better job with the demands on a server due to its performance. I know web frameworks exist for Julia, but I’m wondering how practical it is to actually use this language for that purpose?
- goatlover 8y ago
  If your only requirement is performance, then you'd probably be better off using Go or Node. My guess is you would use Julia for a web server when it ties into other things you're using Julia for. But maybe I'm wrong here. It seems that "use the right tool" turns into use Python or JS or Excel for all the things.
  - oblio 8y ago
    I guess you’re not up to date: Excel is getting JS scripting soon and most likely Python scripting not long after that :D
    - goatlover 8y ago
      Haha, well then! World conquest is surely right around the corner with that combo. Maybe allow for some Rust plugins and have it run on blockchain, just to be on safe side.
- eigenspace 8y ago
  I'm not knowledgeable on this topic at all, but I can tell that the bulk of julia development hasn't involved a lot of thought towards web technologies. So I wouldn't be surprised to learn that the infrastructure just isn't really there to support web servers well in julia. That said, my personal favourite thing about julia is not actually its speed but the fact that it's an incredibly expressive, flexible and composable language (I'd say julia learned all the mot important lessons lisp had to teach). If I had to guess, I'd think that building the infrastructure for devs to work on web servers from the ground up may very well be easier in julia than in most other langauges, including python. I may be off the mark though. One other note since it was mentioned elsewhere, there actually is work being done right now to have julia compile to WebAssembly code which could be pretty cool!
abakus 8y ago
One thing I always dislike about julia is the .+ .* .< ... all those dots for element-wise operations. It is inherited from Matlab, but is a bad design because more common usage should have shorter operators.
- eigenspace 8y ago
  Theres an @. macro that will broadcast all the following operations. ie. exp.(v.^2) .+ (10 .* u) can be written as @. exp(v^2) + (10 * u) with no added runtime cost.
- ssfrr 8y ago
  Whether elementwise or linear algebra operations are more common depends on your use case. I’m personally super happy not to need to write np.dot(A, b) when I could just write A*b, or a’b for an inner product, which matches the math and will do the right thing with complex numbers.
ScottPJones 8y ago
The creators of Julia have been focusing on technical, numerical, scientific computing first, however, the language was always intended to also be a good general purpose programming language (which it is) [I was lucky enough to attend Jeff Bezanson's Ph.D. thesis defense at M.I.T. 3 years ago - and was able to ask him that question there]. I used to be a full-time C/C++ programmer, focused on performance of large systems, however, since learning Julia I haven't had to go back to writing C or C++ even once in over 3 years, since I can write even the sort of low-level code I typically do in Julia, faster and easier than in other languages, and get the same performance.
- nur0n 8y ago
  I'm strongly considering Julia as an alternative for C/C++. A few questions if you don't mind: * How does Julia's GC play into the performance characteristics of these systems? * What tools does Julia offer for fine-grained control of memory? * It appears that Julia uses a rich runtime, do you notice it "weighing down" on performance? * Do you ever feel the need to drop down to C/C++ for hot code paths?
  - ScottPJones 8y ago
    I do have to be aware of the GC, and try to use techniques to avoid lots of allocation. I haven't needed much in the way of fine-grained control of memory, what sorts of things were you looking for? A lot of that "rich runtime" (i.e. including the kitchen sink for linear algebra stuff) has been moved to stdlib, but still packaged with Julia, but even before, it really didn't seem to affect performance except for slowing down the time to build julia (esp. on the Raspberry Pi!). In over 3 years, I've never needed to write anything in C/C++ (except a couple of times I wrote something out in C just to demonstrate the Julia was generating as good or better code (and I mostly write rather low-level stuff). I have it in mind to learn Rust, which I think would be much better than either C or C++ for that sort of stuff (to make a small robust library of functions for some hot code paths), and it integrates well with Julia (using ccall), but the day that Julia's performance hasn't been good enough as not yet come, at least not for me.
adrianhel 8y ago
I had a look at Julia like 2-3 years ago and thought it seemed really nice to use. The performance is of course also awesome. So I hope we get more traction for Julia as a general purpose language. But for now I'm sticking with Node.
a-dub 8y ago
Can you run it with the plotting libraries loaded and have whos() return in less than 10 seconds yet? I was excited about v0.6 but then when I learned it took >15s to do whos(), I completely gave up...
- kescobo 8y ago
  Depending on which plotting package you use. Plots.jl itself is pretty slow to get started still, though there are ideas on how to improve that. That said, once you're up and running, plots (and everything else) are super snappy. When I'm doing plotting stuff, I'm usually doing it interactively, and when I'm scripting it, it's because I'm plotting hundreds or thousands of things (and then the startup time is vanishingly small).
  - a-dub 8y ago
    I first thought the whos() issue was some JIT thing... but no, after multiple runs it was still taking 10s of seconds. For context, I come from using MATLAB interactively. When doing that kind of work, the MATLAB 'whos' command is like /bin/ls, it's hard to do much without it. Is there a different command in julia for showing all numerical arrays/matrices/tensors in memory and their dimensions from the repl? whos() seems like another direct analog of MATLAB, but maybe there's something better that isn't unreasonably slow?
    - ivirshup 8y ago
      I'm a little late to the party, but I think something like this should do the trick: [names(Main) typeof.(eval.(names(Main)))]
      - a-dub 8y ago
        runs crying back to MATLAB
dyu- 8y ago
Great ffi perf too! From the benchmark over at github.com/dyu/ffi-overhead, julia is on par with luajit.
- ChrisRackauckas 8y ago
  Wow! Thanks for sharing! Julia and LuaJIT's JITing strategy really shine here, calling shared libraries faster than C itself!
Bromskloss 8y ago
Ooo, it must be zero-based indexing! Right?!
- oxinabox 8y ago
  Zero-based indexing was kinda mentioned in the post. About indexing with `begin`

227 points by oxinabox 8y ago

I'm surprised the author didn't touch on the massive changes to Pkg. Now Pkg is a lot more like npm, allowing for "project" level packages rather than requiring global packages.

oxinabox 8y ago
Good point, that is huge, I was intending to mention it as the start (and have done so now). It like the getfield overloading and a few other things really deserve a post of their own, so I wasn't going to try and push them into this one.
photon-torpedo 8y ago
Also the new Pkg is massively faster.
ScottPJones 8y ago
He explained that at the beginning of his post - things like the Pkg changes are getting quite a lot of press already, he wanted to highlight the things that people might not have realized were coming in v0.7/v1.0.

_jordan 8y ago

Phenomenal language. compiles applications to a single binary which makes it super convenient to move around environments

oblio 8y ago
How? Everything I've read about Julia seems to show that you're supposed to pass the source code files around. https://medium.com/@sdanisch/compiling-julia-binaries-ddd6d4e0caf4 https://medium.com/@sdanisch/compiling-julia-binaries-ddd6d4... -> notice that's a third party package.
- ninjin 8y ago
  Indeed, which comes with its own advantages and disadvantages. As someone that has to always care about performance, Machine Learning, I am very happy not to have to worry about dependencies not being compiled to take advantage of the latest and greatest CPU instructions on a specific box in a very heterogeneous cluster. Now, you could just compile it all on each deployment, but something as large as say TensorFlow takes ages and not everyone gets to enjoy having great devops at your beck and call.
  - oblio 8y ago
    I'm not sure I understand what you mean: you prefer compilation on the target machines or you prefer a static binary?
    - ninjin 8y ago
      Compilation on the target machine, in my case. Performance trumps portability for me, but I can totally see why this is not the case for everyone.
      - oblio 8y ago
        In theory that can be solved with the best of both worlds: have the runtime and JIT in the final binary. That way you still get easy deployment with 1 big file and high performance after the JIT warms up, which should be fast. I think it's just a problem of manpower on the Julia developer side.
        ninjin 8y ago
        I wholeheartedly agree, one could also note how this is Java-esque. My hope is that someone far more well-versed than me in this area will find the time and push for it. On that note, one wonderful thing about Julia is how easy it is to contribute to the core language since it is written in Julia. It really is amazing to see people from so many disciplines come together to push a “language for science as a whole”.

parvenu74 8y ago

What is the target market or prime use case for Julia? What makes it better than alternatives? I've _heard_ of this language but haven't studied it; I'm trying to understand if this is more of a niche language or something which could see more mainstream adoption like Go or Rust.

oblio 8y ago
It's meant to be primarily a scientific language and the developers are primarily focusing on this. Post 1.0 who knows? It definitely looks like something which could be used as a general purpose programming language.
aschampion 8y ago
It's mainly targeting MATLAB, and to a slightly lesser extent scientific and numeric programming in Python and R. It's a well thought out language that allows you to write MATLAB-like high level code with an easy gradient for progressive typing and optimization to near C level performance. It's a bit harder to sell versus Python, since Python has enormous value in the ecosystem, community, and ubiquity. Also, because it primary targets MATLAB a lot of the standard libraries try to have similar ergonomics, which is a bit of a waste of a great tool to recreate a poor interface.
- fareesh 8y ago
  I often wonder what inspires folks to start from scratch in the face of a gigantic ecosystem like the one that Python brings with it, which will also keep improving.
  - 3JPLW 8y ago
    In the case of Julia, you can check out the original motivation back in 2012 or this recent answer on the message board. https://julialang.org/blog/2012/02/why-we-created-julia https://julialang.org/blog/2012/02/why-we-created-julia https://discourse.julialang.org/t/julia-motivation-why-werent-numpy-scipy-numba-good-enough/2236/10 https://discourse.julialang.org/t/julia-motivation-why-weren...

twic 8y ago

> It will be interesting to see exactly how nothing and missing play out in-practice. I feel like if all things go well, then in most circumstances operating with a missing should return a missing (i.e. properation), and operating with a nothing should throw an error (Much like a NullReferenceError). We all know that having a null value in your language is a mistake, but i have to admit that it had not occurred to me that the fix was to have two null values.

masklinn 8y ago
> We all know that having a null value in your language is a mistake, but i have to admit that it had not occurred to me that the fix was to have two null values. Javascript is even better as it has 3 (null, undefined and NaN, and while NaN is available in most every language it is pretty common in JS as its conversion APIs tend to return NaN instead of error).
- deleted 8y ago
  [deleted]
- petre 8y ago
  How is 3 null values better than one? Fortunately Dart has fixed this nonsense.
  - jimbokun 8y ago
    (That's the joke.)
zimablue 8y ago
The root of "null is a mistake" isn't that null is a mistake, it's that it's a mistake to allow null into a type system in such a way that it can cause non-compile-time type errors. So Julia is typed ( I think) but it's not a mistake in any way to have null in a dynamic language by this argument. Null is a thing and if you don't have it you reinvent it, it's the typesystem breaking that's the problem, maybe monads effectively have nulls it's just that they then get protected by the typesystem.

jb1991 8y ago

Honest question, just how fast is Julia anyway? Because performance is typically the main reason anyone would want to use it over alternatives. A lot of people justify Julia because, as noted in other comments, it is both high-level yet claims to be nearly as fast as C: > "Julia is the fastest modern open-source language for data science, machine learning and scientific computing...with the speed, capacity and performance of C, C++..." [4] That's a bold claim! And there is much evidence to the contrary. I often find that the language falls into a similar trap of many other languages where the creators are evangelists not quite sharing the whole picture. Writing fast Julia code is not always a pleasant experience as you often need to fight the easier idioms. It is marketed as fast, but really, how fast is it? [0] http://www.zverovich.net/2016/05/13/giving-up-on-julia.html http://www.zverovich.net/2016/05/13/giving-up-on-julia.html "What’s disappointing is the striking difference between the claimed performance and the observed one. For example, a trivial hello world program in Julia runs ~27x slower than Python’s version and ~187x slower than the one in C." [1] https://www.ibm.com/developerworks/community/blogs/jfp/entry/A_Comparison_Of_C_Julia_Python_Numba_Cython_Scipy_and_BLAS_on_LU_Factorization?lang=en https://www.ibm.com/developerworks/community/blogs/jfp/entry... "We can code in Julia in a way similar to Python's code. However, that code is slower than it should. One way to speed Julia is to take into account the Fortran ordering it uses by looping on j before looping on i in the second loop. We also add decorators to speed up the code." [2] https://www.codementor.io/zhuojiadai/julia-vs-r-vs-python-string-sort-performance-an-unfinished-journey-to-optimizing-julia-s-performance-f57tf9f8s https://www.codementor.io/zhuojiadai/julia-vs-r-vs-python-st... "...comparing R's sorting speeds to Julia's is not the complete story, even though on the surface R appears faster, and from a users' perspective, (once the data is loaded) R is still the king of speed." Stackoverflow has quite a few user posts [3] from folks trying to get their code to perform at the advertised speed vs popular alternative languages. It's not so easy. [3] https://stackoverflow.com/questions/20613817/julia-julia-lang-performance-compared-to-fortran-and-python https://stackoverflow.com/questions/20613817/julia-julia-lan... [4] https://juliacomputing.com/ https://juliacomputing.com/

montalbano 8y ago
Regarding [0], startup time is a commonly acknowledged issue which affects the 'hello world' benchmark. (Can you even call it a benchmark? Maybe if startup time is what you're comparing then it makes sense, other wise I don't agree.) I used to call Fortran code from Python for a particularly complex function (Mittag-Leffler function) from Python, I then rewrote everything in Julia and it runs approx 10-20x faster. So there you go, I'm a primary source. Edit: just for clarity: The Mittag-Leffler function itself runs just ever so slightly faster in Julia than Fortran-called-from-Python (due to the probably inefficient way I was sending and receiving the data to/from a Python subprocess, which itself is an example of the difficulty of combining two languages). It is the program as a whole which runs 10-20x faster.

wirrbel 8y ago

Julia always was one of the languages I wondered whether I should have a closer look at (do python data-science stuff mostly, so it is an interesting project). However, reading Dan Luu writing about his experiences with members of the Julia community https://danluu.com/julialang/ https://danluu.com/julialang/ I try my best to not get anywhere near it. Directly cited: Update: this post was edited a bit to remove a sentence about how friendly the Julia community is since that no longer seemed appropriate in light of recent private and semi-private communications from one of the co-creators of Julia. They were, by far, the nastiest and most dishonest responses I've ever gotten to any blog post. Some of those responses were on a private discussion channel; multiple people later talked to me about how shocked they were at the sheer meanness and dishonesty of the responses. Oh, and there's also the public mailing list. The responses there weren't in the same league, but even so, I didn't stick around long since I unsubscribed when one the Julia co-creators responded with something bad enough that it prompted someone else to to suggest sticking to the facts and avoiding attacks. That wasn't the first attack, or even the first one to prompt someone to respond and ask that people stay on topic; it just happened to be the one that made me think that we weren't going to have a productive discussion. I extended an olive branch before leaving, but who knows what happened there? Update 2, 1 year later: The same person who previously attacked me in private is now posting heavily edited and misleading excerpts in an attempt to discredit this post. I'm not going to post the full content in part because it's extremely long, but mostly because it's a gross violation of that community's norms to post internal content publicly. If you know anyone in the RC community who was there for the discussion before the edits and you want the truth, ask your RC buddy for their take. If you don't know any RC folks, consider that my debate partner's behavior was so egregious that multiple people asked him to stop, and many more people messaged me privately to talk about how inappropriate his behavior was. If you compare that to what's been publicly dredged up, you can get an idea of both how representative the public excerpts are and of how honest the other person is being._

hellofunk 8y ago
I've seen that post as well, and often wondered if it was fair characterization of overall Julia community (maybe it is), or is the result of a single bad apple at the top that overshadowed an otherwise pleasant community. Unfortunately, it doesn't take much negative energy to spoil a community, even if it comes from just one person. That it was a language co-creator is troubling but not necessarily a reason to avoid the language if the rest of the community is nice (which maybe it is, maybe it isn't). However one thing I have noticed is that most language communities do tend to follow an attitude set by the language creator(s). This seems to have played out quite a bit for Clojure, Python, Elm, Elixir, and other languages where, at least to me, the overall shared perspective of the community is closely aligned to the personal attitudes and opinions of the language author(s).

sidcool 8y ago

And we are struggling to connect Julia to Postgres DB.

StefanKarpinski 8y ago
Have you looked at https://github.com/invenia/LibPQ.jl https://github.com/invenia/LibPQ.jl? The folks at Invenia are very sharp and generally produce excellent, reliable packages, so I would expect this to be of high quality. The primary author, Eric Davies, in particular, is a well known high caliber Julia contributor :)
- sidcool 8y ago
  I am using it, but it's quite inconvenient. The return types are wrong. Documentation is sparse. I would like contribute to it.
  - 3JPLW 8y ago
    Please do! Not only are the folks at Invenia technically talented, they're also really kind and wonderful. I'm sure I'm not speaking out of turn to say that I think they would very much appreciate bug reports, API feedback, and especially doc and code contributions!
    - sidcool 8y ago
      I have started with documentation and coffee samples that I m trying. Let's see where it goes with code contribution. I am especially trying to efficiently convert the hideous NamedTuples to DataFrames.

d--b 8y ago

Where are we compared to then (http://www.zverovich.net/2016/05/13/giving-up-on-julia.html http://www.zverovich.net/2016/05/13/giving-up-on-julia.html)?

eigenspace 8y ago
Theres a discussion on that post in another thread below, but suffice to say the julia community is pretty exasparated with the constant need to respond to that post when its main source of the speed claims was to measure the perofrmance of a hello world program in julia and included the compilation and startup time in the benchmark. Thats a deceptive and unhelpful 'benchmark', especially when the C++ compilation time was not included in its benchmark. Julia compilation and statup times are slow if you are expecting the feel of the Python interpreter, however, julia's compilation and statup time add a finite, constant overhead to its preformance and so are completely irrelevant for high performance numrical computing. If you were wanting to spawn many julia instances and make them execute a single command then exit, yes julia would be a terrible choice of that (unless you turn of compilation and use its interpreter or statically compile your program) but for the most part measuring startup time just isn't relevant unless it takes a minute or something absurd. In julia the only libraries I know of that take a minute to load are plotting libraries (once you make your first plot the library is blazing fast again) and thats been considered such a big issue that a new plotting library Makie is well under way which is supposed to be fully statically compiled in julia in order to slash the first time to plot.

mafm 8y ago

Upvote because I think this is the 1st time a posting from the ucc website has made the HN front page.

hellofunk 8y ago

Would anyone recommend using Julia for things like web servers? Python is used for web servers, and is slow. But Julia doesn’t really seem marketed to that application domain, despite that I would expect it to do a better job with the demands on a server due to its performance. I know web frameworks exist for Julia, but I’m wondering how practical it is to actually use this language for that purpose?

goatlover 8y ago
If your only requirement is performance, then you'd probably be better off using Go or Node. My guess is you would use Julia for a web server when it ties into other things you're using Julia for. But maybe I'm wrong here. It seems that "use the right tool" turns into use Python or JS or Excel for all the things.
- oblio 8y ago
  I guess you’re not up to date: Excel is getting JS scripting soon and most likely Python scripting not long after that :D
  - goatlover 8y ago
    Haha, well then! World conquest is surely right around the corner with that combo. Maybe allow for some Rust plugins and have it run on blockchain, just to be on safe side.
eigenspace 8y ago
I'm not knowledgeable on this topic at all, but I can tell that the bulk of julia development hasn't involved a lot of thought towards web technologies. So I wouldn't be surprised to learn that the infrastructure just isn't really there to support web servers well in julia. That said, my personal favourite thing about julia is not actually its speed but the fact that it's an incredibly expressive, flexible and composable language (I'd say julia learned all the mot important lessons lisp had to teach). If I had to guess, I'd think that building the infrastructure for devs to work on web servers from the ground up may very well be easier in julia than in most other langauges, including python. I may be off the mark though. One other note since it was mentioned elsewhere, there actually is work being done right now to have julia compile to WebAssembly code which could be pretty cool!

abakus 8y ago

One thing I always dislike about julia is the .+ .* .< ... all those dots for element-wise operations. It is inherited from Matlab, but is a bad design because more common usage should have shorter operators.

eigenspace 8y ago
Theres an @. macro that will broadcast all the following operations. ie. exp.(v.^2) .+ (10 .* u) can be written as @. exp(v^2) + (10 * u) with no added runtime cost.
ssfrr 8y ago
Whether elementwise or linear algebra operations are more common depends on your use case. I’m personally super happy not to need to write np.dot(A, b) when I could just write A*b, or a’b for an inner product, which matches the math and will do the right thing with complex numbers.

ScottPJones 8y ago

The creators of Julia have been focusing on technical, numerical, scientific computing first, however, the language was always intended to also be a good general purpose programming language (which it is) [I was lucky enough to attend Jeff Bezanson's Ph.D. thesis defense at M.I.T. 3 years ago - and was able to ask him that question there]. I used to be a full-time C/C++ programmer, focused on performance of large systems, however, since learning Julia I haven't had to go back to writing C or C++ even once in over 3 years, since I can write even the sort of low-level code I typically do in Julia, faster and easier than in other languages, and get the same performance.

nur0n 8y ago
I'm strongly considering Julia as an alternative for C/C++. A few questions if you don't mind: * How does Julia's GC play into the performance characteristics of these systems? * What tools does Julia offer for fine-grained control of memory? * It appears that Julia uses a rich runtime, do you notice it "weighing down" on performance? * Do you ever feel the need to drop down to C/C++ for hot code paths?
- ScottPJones 8y ago
  I do have to be aware of the GC, and try to use techniques to avoid lots of allocation. I haven't needed much in the way of fine-grained control of memory, what sorts of things were you looking for? A lot of that "rich runtime" (i.e. including the kitchen sink for linear algebra stuff) has been moved to stdlib, but still packaged with Julia, but even before, it really didn't seem to affect performance except for slowing down the time to build julia (esp. on the Raspberry Pi!). In over 3 years, I've never needed to write anything in C/C++ (except a couple of times I wrote something out in C just to demonstrate the Julia was generating as good or better code (and I mostly write rather low-level stuff). I have it in mind to learn Rust, which I think would be much better than either C or C++ for that sort of stuff (to make a small robust library of functions for some hot code paths), and it integrates well with Julia (using ccall), but the day that Julia's performance hasn't been good enough as not yet come, at least not for me.

adrianhel 8y ago

I had a look at Julia like 2-3 years ago and thought it seemed really nice to use. The performance is of course also awesome. So I hope we get more traction for Julia as a general purpose language. But for now I'm sticking with Node.

a-dub 8y ago

Can you run it with the plotting libraries loaded and have whos() return in less than 10 seconds yet? I was excited about v0.6 but then when I learned it took >15s to do whos(), I completely gave up...

kescobo 8y ago
Depending on which plotting package you use. Plots.jl itself is pretty slow to get started still, though there are ideas on how to improve that. That said, once you're up and running, plots (and everything else) are super snappy. When I'm doing plotting stuff, I'm usually doing it interactively, and when I'm scripting it, it's because I'm plotting hundreds or thousands of things (and then the startup time is vanishingly small).
- a-dub 8y ago
  I first thought the whos() issue was some JIT thing... but no, after multiple runs it was still taking 10s of seconds. For context, I come from using MATLAB interactively. When doing that kind of work, the MATLAB 'whos' command is like /bin/ls, it's hard to do much without it. Is there a different command in julia for showing all numerical arrays/matrices/tensors in memory and their dimensions from the repl? whos() seems like another direct analog of MATLAB, but maybe there's something better that isn't unreasonably slow?
  - ivirshup 8y ago
    I'm a little late to the party, but I think something like this should do the trick: [names(Main) typeof.(eval.(names(Main)))]
    - a-dub 8y ago
      runs crying back to MATLAB

dyu- 8y ago

Great ffi perf too! From the benchmark over at github.com/dyu/ffi-overhead, julia is on par with luajit.

ChrisRackauckas 8y ago
Wow! Thanks for sharing! Julia and LuaJIT's JITing strategy really shine here, calling shared libraries faster than C itself!

Bromskloss 8y ago

Ooo, it must be zero-based indexing! Right?!

oxinabox 8y ago
Zero-based indexing was kinda mentioned in the post. About indexing with `begin`