12 ms·

An introduction to the Julia language, part 1

78 points by leephillips 8y ago

anonytrary 8y ago
> Subscription required. The page you have tried to view (An introduction to the Julia language, part 1) is currently available to LWN subscribers only. Reader subscriptions are a necessary way to fund the continued existence of LWN and the quality of its content. If you are already an LWN.net subscriber, please log in with the form below to read this content. I'm pretty sure I can find a tutorial somewhere else.
- kryptiskt 8y ago
  Articles on LWN are publicly available the week after publishing.
adamnemecek 8y ago
I’ve been using Julia for a couple of months now and it’s unparalleled for scientific computation. More expressive than say python, super fast, easy ffi with python, cpp, MATLAB, Mathematica and others. But above all it’s the community that takes the cake. So many hard algorithms that I would have a hard time doing from scratch are implemented in Julia.
- ummonk 8y ago
  >So many hard algorithms that I would have a hard time doing from scratch are implemented in Julia. Well it helps that it's a lot easier to implement stuff from scratch in Julia than pretty much any other language.
  - adamnemecek 8y ago
    You are correct.
- vasili111 8y ago
  How good is symbolic math in comparison to Mathematica?
  - leephillips 8y ago
    It's not a symbolic math program, although there is a symbolic math package for it, that I haven't tried.
  - cultus 8y ago
    As leephilips said, there's a package, but it's nowhere near what Mathematica can do symbolically. Julia was more intended as a numerical Matlab replacement. However, it could have a lot of potential here. It is a fully homoiconic language with real macros, so it could be a very good fit for symbolic math, just as Maxima is built in Common Lisp. It would be great if there was a real effort towards that. Mathematica is still really the only game in town.
    - waldrews 8y ago
      It's surprising to me that there isn't a good way to call Maxima from either Python/R/Julia or even a non-CL lisp. Since Maxima runs on top of Common Lisp, could it run on a JVM-based Common Lisp, and exchange objects with Clojure? Could the CLASP project, CL on the LLVM, lead to a Julia-Maxima bridge? Any other integration ideas from the Common Lisp folks?
  - cultus 8y ago
    Replying to your other comment >It's surprising to me that there isn't a good way to call Maxima from either Python/R/Julia or even a non-CL lisp. There's Sage, which integrates Scipy/Numpy, Maxima, and some other things. However, I personally find Maxima to be pretty awful to use, and even it is nowhere close to Mathematica in functionality at this point. Mathematica itself is horrible to use (although extemely capable), so I'd love an open source, good alternative.
    - ssivark 8y ago
      A general comment about Mathematica, not just a response to the comment above: Trying to use Mathematica like Python/Matlab is setting yourself up for failure. It took me a few months of getting frustrated with Mathematica till I suddenly "got" it [1]. But once I understood that it's programming model revolves around pattern matching and string replacement, things just clicked and it was very easy to write Mathematica code after that. [1] Coincidentally, I was reading a little bit about functional programming when Mathematica finally started making sense. In particular, IIRC, I was looking at "Learn You a Haskell" for fun, unrelated to Mathematica, which was for work.
leephillips 8y ago
If, for some incomprehensible reason, you are not an LWN subscriber, here is a free link: https://lwn.net/SubscriberLink/763626/f2990348ebd06167/ https://lwn.net/SubscriberLink/763626/f2990348ebd06167/ Please consider subscribing, so they can pay me and other authors for more articles.
- sctb 8y ago
  Thanks! We've updated the link so users can get there right away.
m3mpp 8y ago
What is the language lineage? Is it functional or imperative? Dynamic or statically typed? Garbage collected or not? I wish every introduction to a new language would contain that kind of info, it'd be so much more informative. Edit: missed one, natively compiled or VM based.
- FridgeSeal 8y ago
  I’ll have a stab at answering this, someone who knows better can correct me. Dynamic with optional typing. General consensus is that it has a pretty great type system. Union types, language level implementation of high performance missing types. Garbage collected. JIT compiled using LLVM backend; entire language is written in itself, so the compiler doesn’t have any “black boxes”, I.e. it can optimise everything. Great multiple dispatch system. Imperative technically I guess? But feels like it’s lifted a bunch of good ideas from various functional and domain specific languages. Anything I missed?
  - tejtm 8y ago
    User defined types are __no__ different than "native" types. You are talking to the llvm compiler, sure you are talking in Julia most of the time, but if you happen to want to talk in C or FORTRAN or ASM in the middle of your high level script, just do. No marshalling of I/O or data structure impedance it all compiles the same.
    - nextos 8y ago
      Exactly. Typing choices are a key part of what makes Julia interesting. And user types are first-class.
    - earenndil 8y ago
      So can I define custom operations in llvm ir?
      - tejtm 8y ago
        You can write custom code in llvm IR.[0] I must leave it to you to tell me if you can use it as an operator. My guess is yes, but I will not be getting to play any time soon. [0]https://docs.julialang.org/en/stable/base/c/#Core.Intrinsics.llvmcall https://docs.julialang.org/en/stable/base/c/#Core.Intrinsics...
ArtWomb 8y ago
Congrats to JuliaLang team on shipping 1.0 release! Adoption is widespread across disciplines, from genomics to cosmology. Am currently evaluating JuliaImages, and it's almost feature complete with matlab. Predict next-gen computer vision framework will be designed in JuliaLang ;) http://news.mit.edu/2018/mit-developed-julia-programming-language-debuts-juliacon-0827 http://news.mit.edu/2018/mit-developed-julia-programming-lan...
gerdesj 8y ago
Julia can handle rational numbers, with a special syntax: 3//5 + 2//3 returns 19//15, while 3/5 + 2/3 gets you the floating-point answer 1.2666666666666666. Why not allow a/b + c/d = (a.d + c.b)/(b.d)? It's what happens when I get my pen and paper out in the real world (along with my tongue sticking out and much perspiration and perhaps a few doodles.) Why does Julia use / to cause one of those weird repeated decimal expansion things to appear but // is needed to keep things pure? Yes I am partially taking the piss but I'm also interested into the design decision into why // is needed to avoid a type conversion. I note that 1/2 = 0.5 was also a design decision.
- chrispeel 8y ago
  A rational number is represented in a fundamentally different way than a floating-point type, or a fixed-point number, or a BigFloat. Julia allows you to define your own types, so if you want to define a type that behaves in the special way you want, you're welcome to do so.
  - gerdesj 8y ago
    If I write 2/3 on a piece of paper it does not suddenly become 0.666666666666666666666 why does Julia (int al) insist on this? (tongue still in cheek)
    - KenoFischer 8y ago
      Well, yeah, that's because it's a piece of paper, not a calculator. More seriously, you have to be very careful "spooky action at a distance" in programming language design. For example, you might say something like "Well, if the compiler can prove that every constant involved is provable in rational arithmetic, do that, otherwise use floating point arithmetic", which seems like a perfectly sensible thing to do, until you realize that it means that very small changes to the code will have drastic effects on the correctness of your program (if suddenly the compiler is no longer able to prove rational arithmetic is the same answer). We do have that sort of thing happen with performance in Julia (and in many other languages too, of course) - if the compiler can prove it can do something faster, it'll try to, but we try, very, very hard to give everything predictable, sequential execution semantics (i.e. you should be able to take code, run it step by step and generally get the same answer as if you had executed it all at once). One particular example of this is comprehensions. In early versions of julia, the type of `[f(x) for x in a]` would depend on what the compiler was able to prove about `f` (and some other things), e.g. if the compiler knew that `f` always returned an `Int`, you'd get a `Vector{Int}`, otherwise a `Vector{Any}` (or whatever the compiler could prove). This turned out to be hugely annoying to people because the semantics of the computation changed (slightly, only an eltype which you generally don't look at), depending on what the compiler was or was not able to prove about your program. These days, we have runtime semantics for non-empty comprehensions and only use static information to improve performance (which as I said before, we do anyway, so users are used to it). That's not to say that you couldn't do a language that changes semantics based on whole program analysis (you could probably do it in Haskell for example), but you'd lose part of the gradual transition from REPL to full program, which is very important to a system like Julia.
jgmjgm 8y ago
I recall reading that the biggest problem with Julia was that the language was not sufficiently well tested in an automated way. The result of which was that complex bugs kept creeping back in making the language less reliable in a production environment. Is my memory faulty? If not, have these concerns been addressed?
- KenoFischer 8y ago
  You're probably thinking of complaints that our CI was flakey at times (failing despite changes being ok), which is true (though it's been pretty stable lately). However, people sometimes equate that with bad test coverage, which is absolutely not the same thing. Easiest way to get a test to pass is to delete it. From my perspective at least, the julia test suite is quite comprehensive (if something passes the test suite, I'm generally happy to put it on master and have people use it). We're also lucky that there is so much open source julia code available, which we can use as essentially one giant regression test. We used that heavily in the lead up to 1.0 to make sure that there weren't any surprise bugs waiting for people.
PinkMilkshake 8y ago
The Wikipedia page on Julia says one of the original goals was that Julia could be used as a specification language. Does Julia already provide tools for formal verification?
- StefanKarpinski 8y ago
  That seems likely to be an unfounded claim since that wasn't one of the original goals.
sgillen 8y ago
I’ve been really excited about Julia for awhile now. Does anyone here happen to know why the decision was made to not allow “normal” classes in Julia? I know you can define your own types to get some of the functionality but sometimes in numerical computing with matlab or python it really makes sense for me to pack functionality into a class. I know in matlab using classes causes a performance hit but I always thought that wasn’t because the feature was bolted on afterwards. Is there something I’m missing here?
- FridgeSeal 8y ago
  As someone who’s written a lot of Python I understand where you’re coming from, but I generally think of it the other way around: why does Python lack the ability to create lightweight structs that a lot of other languages have and I often find exceedingly useful (I mean I know why, dynamics and duck typing and all that). Creating a whole class just to neatly carry around a few related values always felt like a very “heavy” solution.
- mac01021 8y ago
  > it really makes sense for me to pack functionality into a class. Why? Where you would ordinarily write class Foo { int x; int my_method() { ... } } can't you just equivalently write type Foo x::Int end my_method(foo::Foo) = begin ... end ?
  - sgillen 8y ago
    Sure, I guess my question is WHY do it this way, which (to me) coming from C++/python seems pretty foreign. I was hoping one of the contributors to the language might be lurking here and offer some insight. I found a nice discussion about it here [1] though, if you are interested. (https://groups.google.com/forum/#!searchin/julia-users/object$20oriented/julia-users/VY4XDx9aHdU/qnFZU3kfBQAJ https://groups.google.com/forum/#!searchin/julia-users/objec...)
    - ChrisRackauckas 8y ago
      Julia's atom of compilation is the function, so having as few unique function definitions as possible makes it easier to optimize and reduces compilation time.

78 points by leephillips 8y ago

> Subscription required. The page you have tried to view (An introduction to the Julia language, part 1) is currently available to LWN subscribers only. Reader subscriptions are a necessary way to fund the continued existence of LWN and the quality of its content. If you are already an LWN.net subscriber, please log in with the form below to read this content. I'm pretty sure I can find a tutorial somewhere else.

kryptiskt 8y ago
Articles on LWN are publicly available the week after publishing.

adamnemecek 8y ago

I’ve been using Julia for a couple of months now and it’s unparalleled for scientific computation. More expressive than say python, super fast, easy ffi with python, cpp, MATLAB, Mathematica and others. But above all it’s the community that takes the cake. So many hard algorithms that I would have a hard time doing from scratch are implemented in Julia.

ummonk 8y ago
>So many hard algorithms that I would have a hard time doing from scratch are implemented in Julia. Well it helps that it's a lot easier to implement stuff from scratch in Julia than pretty much any other language.
- adamnemecek 8y ago
  You are correct.
vasili111 8y ago
How good is symbolic math in comparison to Mathematica?
- leephillips 8y ago
  It's not a symbolic math program, although there is a symbolic math package for it, that I haven't tried.
- cultus 8y ago
  As leephilips said, there's a package, but it's nowhere near what Mathematica can do symbolically. Julia was more intended as a numerical Matlab replacement. However, it could have a lot of potential here. It is a fully homoiconic language with real macros, so it could be a very good fit for symbolic math, just as Maxima is built in Common Lisp. It would be great if there was a real effort towards that. Mathematica is still really the only game in town.
  - waldrews 8y ago
    It's surprising to me that there isn't a good way to call Maxima from either Python/R/Julia or even a non-CL lisp. Since Maxima runs on top of Common Lisp, could it run on a JVM-based Common Lisp, and exchange objects with Clojure? Could the CLASP project, CL on the LLVM, lead to a Julia-Maxima bridge? Any other integration ideas from the Common Lisp folks?
- cultus 8y ago
  Replying to your other comment >It's surprising to me that there isn't a good way to call Maxima from either Python/R/Julia or even a non-CL lisp. There's Sage, which integrates Scipy/Numpy, Maxima, and some other things. However, I personally find Maxima to be pretty awful to use, and even it is nowhere close to Mathematica in functionality at this point. Mathematica itself is horrible to use (although extemely capable), so I'd love an open source, good alternative.
  - ssivark 8y ago
    A general comment about Mathematica, not just a response to the comment above: Trying to use Mathematica like Python/Matlab is setting yourself up for failure. It took me a few months of getting frustrated with Mathematica till I suddenly "got" it [1]. But once I understood that it's programming model revolves around pattern matching and string replacement, things just clicked and it was very easy to write Mathematica code after that. [1] Coincidentally, I was reading a little bit about functional programming when Mathematica finally started making sense. In particular, IIRC, I was looking at "Learn You a Haskell" for fun, unrelated to Mathematica, which was for work.

leephillips 8y ago

If, for some incomprehensible reason, you are not an LWN subscriber, here is a free link: https://lwn.net/SubscriberLink/763626/f2990348ebd06167/ https://lwn.net/SubscriberLink/763626/f2990348ebd06167/ Please consider subscribing, so they can pay me and other authors for more articles.

sctb 8y ago
Thanks! We've updated the link so users can get there right away.

m3mpp 8y ago

What is the language lineage? Is it functional or imperative? Dynamic or statically typed? Garbage collected or not? I wish every introduction to a new language would contain that kind of info, it'd be so much more informative. Edit: missed one, natively compiled or VM based.

FridgeSeal 8y ago
I’ll have a stab at answering this, someone who knows better can correct me. Dynamic with optional typing. General consensus is that it has a pretty great type system. Union types, language level implementation of high performance missing types. Garbage collected. JIT compiled using LLVM backend; entire language is written in itself, so the compiler doesn’t have any “black boxes”, I.e. it can optimise everything. Great multiple dispatch system. Imperative technically I guess? But feels like it’s lifted a bunch of good ideas from various functional and domain specific languages. Anything I missed?
- tejtm 8y ago
  User defined types are __no__ different than "native" types. You are talking to the llvm compiler, sure you are talking in Julia most of the time, but if you happen to want to talk in C or FORTRAN or ASM in the middle of your high level script, just do. No marshalling of I/O or data structure impedance it all compiles the same.
  - nextos 8y ago
    Exactly. Typing choices are a key part of what makes Julia interesting. And user types are first-class.
  - earenndil 8y ago
    So can I define custom operations in llvm ir?
    - tejtm 8y ago
      You can write custom code in llvm IR.[0] I must leave it to you to tell me if you can use it as an operator. My guess is yes, but I will not be getting to play any time soon. [0]https://docs.julialang.org/en/stable/base/c/#Core.Intrinsics.llvmcall https://docs.julialang.org/en/stable/base/c/#Core.Intrinsics...

ArtWomb 8y ago

Congrats to JuliaLang team on shipping 1.0 release! Adoption is widespread across disciplines, from genomics to cosmology. Am currently evaluating JuliaImages, and it's almost feature complete with matlab. Predict next-gen computer vision framework will be designed in JuliaLang ;) http://news.mit.edu/2018/mit-developed-julia-programming-language-debuts-juliacon-0827 http://news.mit.edu/2018/mit-developed-julia-programming-lan...

gerdesj 8y ago

Julia can handle rational numbers, with a special syntax: 3//5 + 2//3 returns 19//15, while 3/5 + 2/3 gets you the floating-point answer 1.2666666666666666. Why not allow a/b + c/d = (a.d + c.b)/(b.d)? It's what happens when I get my pen and paper out in the real world (along with my tongue sticking out and much perspiration and perhaps a few doodles.) Why does Julia use / to cause one of those weird repeated decimal expansion things to appear but // is needed to keep things pure? Yes I am partially taking the piss but I'm also interested into the design decision into why // is needed to avoid a type conversion. I note that 1/2 = 0.5 was also a design decision.

chrispeel 8y ago
A rational number is represented in a fundamentally different way than a floating-point type, or a fixed-point number, or a BigFloat. Julia allows you to define your own types, so if you want to define a type that behaves in the special way you want, you're welcome to do so.
- gerdesj 8y ago
  If I write 2/3 on a piece of paper it does not suddenly become 0.666666666666666666666 why does Julia (int al) insist on this? (tongue still in cheek)
  - KenoFischer 8y ago
    Well, yeah, that's because it's a piece of paper, not a calculator. More seriously, you have to be very careful "spooky action at a distance" in programming language design. For example, you might say something like "Well, if the compiler can prove that every constant involved is provable in rational arithmetic, do that, otherwise use floating point arithmetic", which seems like a perfectly sensible thing to do, until you realize that it means that very small changes to the code will have drastic effects on the correctness of your program (if suddenly the compiler is no longer able to prove rational arithmetic is the same answer). We do have that sort of thing happen with performance in Julia (and in many other languages too, of course) - if the compiler can prove it can do something faster, it'll try to, but we try, very, very hard to give everything predictable, sequential execution semantics (i.e. you should be able to take code, run it step by step and generally get the same answer as if you had executed it all at once). One particular example of this is comprehensions. In early versions of julia, the type of `[f(x) for x in a]` would depend on what the compiler was able to prove about `f` (and some other things), e.g. if the compiler knew that `f` always returned an `Int`, you'd get a `Vector{Int}`, otherwise a `Vector{Any}` (or whatever the compiler could prove). This turned out to be hugely annoying to people because the semantics of the computation changed (slightly, only an eltype which you generally don't look at), depending on what the compiler was or was not able to prove about your program. These days, we have runtime semantics for non-empty comprehensions and only use static information to improve performance (which as I said before, we do anyway, so users are used to it). That's not to say that you couldn't do a language that changes semantics based on whole program analysis (you could probably do it in Haskell for example), but you'd lose part of the gradual transition from REPL to full program, which is very important to a system like Julia.

jgmjgm 8y ago

I recall reading that the biggest problem with Julia was that the language was not sufficiently well tested in an automated way. The result of which was that complex bugs kept creeping back in making the language less reliable in a production environment. Is my memory faulty? If not, have these concerns been addressed?

KenoFischer 8y ago
You're probably thinking of complaints that our CI was flakey at times (failing despite changes being ok), which is true (though it's been pretty stable lately). However, people sometimes equate that with bad test coverage, which is absolutely not the same thing. Easiest way to get a test to pass is to delete it. From my perspective at least, the julia test suite is quite comprehensive (if something passes the test suite, I'm generally happy to put it on master and have people use it). We're also lucky that there is so much open source julia code available, which we can use as essentially one giant regression test. We used that heavily in the lead up to 1.0 to make sure that there weren't any surprise bugs waiting for people.

PinkMilkshake 8y ago

The Wikipedia page on Julia says one of the original goals was that Julia could be used as a specification language. Does Julia already provide tools for formal verification?

StefanKarpinski 8y ago
That seems likely to be an unfounded claim since that wasn't one of the original goals.

sgillen 8y ago

I’ve been really excited about Julia for awhile now. Does anyone here happen to know why the decision was made to not allow “normal” classes in Julia? I know you can define your own types to get some of the functionality but sometimes in numerical computing with matlab or python it really makes sense for me to pack functionality into a class. I know in matlab using classes causes a performance hit but I always thought that wasn’t because the feature was bolted on afterwards. Is there something I’m missing here?

FridgeSeal 8y ago
As someone who’s written a lot of Python I understand where you’re coming from, but I generally think of it the other way around: why does Python lack the ability to create lightweight structs that a lot of other languages have and I often find exceedingly useful (I mean I know why, dynamics and duck typing and all that). Creating a whole class just to neatly carry around a few related values always felt like a very “heavy” solution.
mac01021 8y ago
> it really makes sense for me to pack functionality into a class. Why? Where you would ordinarily write class Foo { int x; int my_method() { ... } } can't you just equivalently write type Foo x::Int end my_method(foo::Foo) = begin ... end ?
- sgillen 8y ago
  Sure, I guess my question is WHY do it this way, which (to me) coming from C++/python seems pretty foreign. I was hoping one of the contributors to the language might be lurking here and offer some insight. I found a nice discussion about it here [1] though, if you are interested. (https://groups.google.com/forum/#!searchin/julia-users/object$20oriented/julia-users/VY4XDx9aHdU/qnFZU3kfBQAJ https://groups.google.com/forum/#!searchin/julia-users/objec...)
  - ChrisRackauckas 8y ago
    Julia's atom of compilation is the function, so having as few unique function definitions as possible makes it easier to optimize and reduces compilation time.