8 ms·

Julia is great. It’s significantly simpler than Python while also being much more expressive. It’s too bad the typing is only for dispatch, but hopefully someon

by smabie 7y ago

Julia is great. It’s significantly simpler than Python while also being much more expressive. It’s too bad the typing is only for dispatch, but hopefully someone will write a typechecker someday. I’ve found it surprisingly refreshing to not have to think about classes and just add functions on objects wherever I want. Some languages solve this with monkey patching (which is bad), others like Scala with an extension class (reasonable, but you still don’t get access to private properties), but the Julia approach is cleaner. I wouldn’t use Julia for a non-scientific computing app as I don’t think it’s suitable, but for anything data science related, it’s great! And with the Python interop, I don’t really think there’s any reason not to use Julia for your next data science project. I suspect that over the next 5 years Python will no longer be used for these applications at all.

sgt101 7y ago
could you expand on the "too bad typing is only for dispatch"? I've enjoyed the way that the Julia type system can be used...
- smabie 7y ago
  It can’t be used for compile time correctness checking? i.e Julia isn’t a statically typed language.
  - sgt101 7y ago
    Yes - I understand why some people are keen on static types, but type inference is a powerful way to create funcetionality and I believe that the separation of concerns between the dispatcher and the programming code (removing the logic of decisioning from the code and using type inference instead) leads to clearer programs. I can't think how you can have that and have static checking as well (although I know that there are some compilers for some Julia code as mentioned in this thread)
    - Symmetry 7y ago
      Lots of languages with static typing use type inference. Haskell, OCaml, Go, Rust, Nim, etc. Even C++ has auto these days which I think when combined with templates gives you what you're talking about though it is a bit clunky. EDIT. For instance in Haskell I can just declar a function to sum the things in a container by saying summer a = folder (+) 0 a And the compiler will figure out that the function accepts a Foldable container of some sort full of things that are Nums without ever having to tell the compiler that (though it's a good idea for maintainability. And then I can just pass in a list of floats and the compiler will Do The Right Thing. And if I pass in a list of booleans instead the compiler will yell at me at compile time because there's no (+) operation for booleans.
      - socialdemocrat 7y ago
        I think he possibly phrased it wrong. Dynamic languages don't do type inference, they carry type with each value at runtime. Dynamic and statically typed languages each have their own wonderful advantages. There is just no way of replicating all the advantages of dynamic typing in a statically typed language. It is hard to sum up why in a short sentence but I have tried to articulate it in a longer article here: https://medium.com/@Jernfrost/the-many-advantages-of-dynamic-languages-267d08f4c7 https://medium.com/@Jernfrost/the-many-advantages-of-dynamic... Keep in mind this focuses only on advantages of dynamic languages. It does not mean I don't consider that static languages don't have their own unique advantages, but that was not the topic of my article. The primary problem with static typing as I see it is the complexity and mental overhead it adds. It is not just the complexity in the type system and the language itself, but also in the tooling surrounding it: build system, dependency management, binary interface definitions, debuggers, REPL etc. The complexity makes meta programming very hard to do for a software developer of average intelligence. Where I see a use for advance static languages is for very specialized systems which need a high level of correctness and where a company is able to do the job with a small team of very bright people.
        oxinabox 7y ago
        Dynamic languages do do type inference. Or at least they can. Julia does type inference at compile time. The difference is that it doesn't have to succeed. If you write type unstable code then it won't.
  - throwaway894345 7y ago
    Presumably this means that no one has implemented a suitable static type checker or its type annotations don't provide sufficient information for a static type checker to work in principle?
    - eigenspace 7y ago
      The problem is more the former (nobody has gotten around to it yet), but also a bit of the opposite of the latter, i.e. the type system is so rich, type-checking may be undecidable in too many contexts. Julia's parametric types are incredibly rich. You can look up 'dependant types' which are similar in the static world and see the sorts of hoops static language designers jump through to have dependant types and still be able to reliably prove theorems.
    - socialdemocrat 7y ago
      What you can do with Julia types is far more advance than I think you can do with any statically typed language. I can add a type annotation in Julia which is a function call which returns a type e.g. This function can be of any complexity. I think a better approach is simply a Linter. You report type problems you can find and ignore the rest. Not point trying to figure out the more advance Julia cases. I would assume 90% of Julia code will use types in a pretty straightforward manner and thus would be able to be analyzed by some sort of Linter. Lint.jl e.g.
      - throwaway894345 7y ago
        You're describing optional type systems; in this regard, Julia is on par with Python or TypeScript. The linter you're describing is also a type checker (like Python's Mypy). That said, it would be really interesting if someone would make a type checker that would evaluate/resolve those more complex types, at least those that can be guaranteed to terminate.
        socialdemocrat 7y ago
        Optional typing and what Julia does is quite different. Julia type system is nothing like Python or TypeScript. In Julia type annotations are not just a sprinkle of sugar. It has a very concrete use beyond asserting type consistency. Julia types are used to decide what code gets run and memory layout of data.
  - socialdemocrat 7y ago
    You can do some form of type checking with stuff like Lint.jl. I mean since Julia uses a bunch of type information and Julia parses code into data structures which are easy to process, you can to a lot of your own type verification before running a program. However I see Julia not being a statically typed language as a feature. If Julia was statically typed you would pretty much have lost everything that makes Julia a great language. You cannot do multiple dispatch with a statically typed language efficiently.
FranzFerdiNaN 7y ago
Python will still be used 20 years from now. The clear advantage of Python is the enormous ecosystem that is available, the millions of questions on SO giving solutions to every problem you can run into, the books and learning materials etc, programmers and corporations having invested loads of time and effort in building, maintaining and battle-testing libraries. Don't get me wrong, i think Julia is an amazing language, but being an amazing language is neither necessary or sufficient to succeed. R shows how you can succeed just fine with a kinda weird language.
- logicchains 7y ago
  >The clear advantage of Python is the enormous ecosystem that is available, the millions of questions on SO giving solutions to every problem you can run into All of this is also available in Julia via its near-seamless Python interop. Its package manager even does a better job of managing Python packages than Python does.
- ScottPJones 7y ago
  Python's ecosystem is great - but Julia's is growing incredibly fast, and in some cases Julia has already surpassed what is available in other languages (for example, take a look at the whole differential equations ecosystem: https://github.com/JuliaDiffEq https://github.com/JuliaDiffEq). Also, Python's ecosystem is only a 'pyimport(name)' away (using the PyCall.jl package). Same thing is true for R and a number of other languages (RCall.jl, JavaCall.jl, etc.) I've been using SymPy, QisKit, matplotlib and other Python packages with no problem in Julia.
- mumblemumble 7y ago
  I would be careful treating this too much like a popularity contest. Python will still be used 20 years from now, but that doesn't necessarily mean that it's going to be dominant 20 years from now. Source: I used to do a lot of programming in various dialects of Basic, which, a bit over 20 years ago, was popular largely because of its own ubiquity and popularity. And, while I still maintain some Basic code, I was surprised how quickly it died. One day everything was being written in it. The next day, we were were writing new things in a fancy new language that everyone agreed was more productive, and talking to the existing stuff through an FFI. And, a day later, we were replacing modules in order to get them off of the "legacy" platform.
mark_l_watson 7y ago
I have a collection of short Julia code snippets that do non numeric things like text processing, semantic web clients, etc. But in general I agree. Fir me, Lisp languages, Haskell, and Python are more general purpose.
- vasili111 7y ago
  >Fir me, Lisp languages, Haskell, and Python are more general purpose. You mean Julia as language is not enough good for general purpose or it lacks ecosystem (frameworks, libraries) for general purpose use?
  - smabie 7y ago
    A little of both, I guess. The biggest issue is that I, personally, get uncomfortable with dynamic typing when the project starts becoming over 1-5 Kloc. So I tend to stick with stuff like Scala, Rust, or OCaml for non data science tasks.
  - mark_l_watson 7y ago
    I didn’t express that correctly. What I meant to say was that I investigated Julia as a possible “use it for everything language.” For me, general purpose has to include deep learning tools. I really prefer Lisp and Haskell, but although Haskell’s TensorFlow support is pretty good, I found it easier to just use Python. I have had problems with the SBCL Common Lisp TensorFlow C bindings, but perhaps that was my fault.
    - iLemming 7y ago
      AFAIK Clojurians doing some interesting stuff in that space. They've figured out Python Interop, writing books on Deep learning, Linear Algebra, etc. all that in Clojure. https://aiprobook.com https://aiprobook.com https://nextjournal.com/gigasquid https://nextjournal.com/gigasquid
- e12e 7y ago
  What part of Julia isn't lispy enough that you don't consider it a lisp? I'm not saying you're wrong, just because there a scheme in there, just curious.
ScottPJones 7y ago
I've been using Julia for non-scientific computing programs for almost 5 years now, and (especially now that it is stable since the 1.0 release) have found it well suited for general programming as well. Having a language that is easy to write (like Python), runs fast (like C/C++), and incredibly flexible & expressive (like Lisp) makes programming fun again!
- skybrian 7y ago
  Interesting, how does deployment work for you? What sort of executables do you ship?
  - nnq 7y ago
    Don't people mostly ship containers nowadays? Or deploy code but standardize on language version. Why t f would you want to ship / deploy a naked executable in 2019?!
    - skybrian 7y ago
      Containers are common for server-side code but not for mobile, desktop, or client-side web code.
      - MiroF 7y ago
        e: ignore
    - unlinked_dll 7y ago
      Not all of us are web developers.
    - throwaway894345 7y ago
      * Container images are generally huge. Layer caching helps to the extent that layers are reusable. If you're lucky, you can use something like alpine for smaller huge images. If you write in a language that supports statically linked executables (e.g., Go) then you can get by with a scratch in many cases, but then you could more easily ship a naked executable * Container images take a long time to build and build caching only works well if your dependency tree is linear (it's not). (Caveat: there are niche alternatives to docker build for which this may not hold, but then you're subject to the other issues with niche tools--support, docs, compat, etc) * Your deployment target has to have a container runtime compatible with your image format. This is a PITA for end users in many cases. * Most popular container runtimes make container processes children of the daemon process, not children of the client process. This means killing the parent process will not kill the child process, which means CI environments need to handle all sorts of edge cases. For example, a CI job is waiting on a stuck container to terminate, but eventually times out or the job is killed or similar; each time this happens a container is leaked--this means that CI tools have to support the container runtime explicitly and handle all of these edge cases because it's prohibitively complex for users to manage. IMO the only advantage (and this is a huge advantage where applicable) for containers is that they can be orchestrated by things like Kubernetes or AWS Fargate.
real_eng_ 7y ago
Why every Julia user can't help but trash Python at every occasion? It's getting really tiring
- ScottPJones 7y ago
  I haven't seen that at all - many Julia programmers are (or were) also Python programmers. I think there is a lot of respect in the Julia community for Python & the Python ecosystem. There have even been a number of Julia talks at various PyCons over the past few years.
  - oxinabox 7y ago
    There was a python talk at the 2019 juliacon.
- smabie 7y ago
  Trashing is a little harsh, but Julia devs probably were mostly Python devs back in the day and are intimately familiar with the inadequacies of the language: the work required when you had to drop into C, the bad syntax for math, the constant conversions between ndarray, array, and lists, etc etc. At my old bioinformatics lab, Python literally wasted thousands and thousands of collective man-hours which would have been saved by Julia if it had existed at the time. Since a lot of these researchers couldn’t really program that well, they would write code that would literally take weeks to run. And then they would call me and I would rewrite it in C and it would then take an hour or two. Julia solves this problem. The amount of unnecessary supercomputer time (and electricity) that our lab (and others) were wasting with Python was, honestly, disgusting.
  - elcritch 7y ago
    Exactly... well I didn’t do supercomputing but still wasted a lot of time dealing with Python 2/3 issues among others. Then there’s the ability to do things like fast bootstrapping, Monte-Carlo and other handy statistical techniques without writing kernel functions in C. Python was great for it’s time and enabled a lot and still let’s lots of people do great things, it’s just not for me anymore.
vanderZwan 7y ago
> It’s too bad the typing is only for dispatch, but hopefully someone will write a typechecker someday I was under the impression that if you write a function that takes very specific sub-types as its parameters, and then try to call it with invalid parameters (that is, values that cannot be converted to the right type via the promotion rules) that Julia will complain about this?
- deleted 7y ago
  [deleted]
- tgflynn 7y ago
  It will complain, but typically only at run time.
  - vanderZwan 7y ago
    Ah, right! We're talking about AOT typechecking. I had forgotten Julia doesn't really do that (yet). Yes, fair is fair, since that is when it is the most useful its absence is a bit painful.

by smabie 7y ago

sgt101 7y ago

could you expand on the "too bad typing is only for dispatch"? I've enjoyed the way that the Julia type system can be used...

smabie 7y ago
It can’t be used for compile time correctness checking? i.e Julia isn’t a statically typed language.
- sgt101 7y ago
  Yes - I understand why some people are keen on static types, but type inference is a powerful way to create funcetionality and I believe that the separation of concerns between the dispatcher and the programming code (removing the logic of decisioning from the code and using type inference instead) leads to clearer programs. I can't think how you can have that and have static checking as well (although I know that there are some compilers for some Julia code as mentioned in this thread)
  - Symmetry 7y ago
    Lots of languages with static typing use type inference. Haskell, OCaml, Go, Rust, Nim, etc. Even C++ has auto these days which I think when combined with templates gives you what you're talking about though it is a bit clunky. EDIT. For instance in Haskell I can just declar a function to sum the things in a container by saying summer a = folder (+) 0 a And the compiler will figure out that the function accepts a Foldable container of some sort full of things that are Nums without ever having to tell the compiler that (though it's a good idea for maintainability. And then I can just pass in a list of floats and the compiler will Do The Right Thing. And if I pass in a list of booleans instead the compiler will yell at me at compile time because there's no (+) operation for booleans.
    - socialdemocrat 7y ago
      I think he possibly phrased it wrong. Dynamic languages don't do type inference, they carry type with each value at runtime. Dynamic and statically typed languages each have their own wonderful advantages. There is just no way of replicating all the advantages of dynamic typing in a statically typed language. It is hard to sum up why in a short sentence but I have tried to articulate it in a longer article here: https://medium.com/@Jernfrost/the-many-advantages-of-dynamic-languages-267d08f4c7 https://medium.com/@Jernfrost/the-many-advantages-of-dynamic... Keep in mind this focuses only on advantages of dynamic languages. It does not mean I don't consider that static languages don't have their own unique advantages, but that was not the topic of my article. The primary problem with static typing as I see it is the complexity and mental overhead it adds. It is not just the complexity in the type system and the language itself, but also in the tooling surrounding it: build system, dependency management, binary interface definitions, debuggers, REPL etc. The complexity makes meta programming very hard to do for a software developer of average intelligence. Where I see a use for advance static languages is for very specialized systems which need a high level of correctness and where a company is able to do the job with a small team of very bright people.
      - oxinabox 7y ago
        Dynamic languages do do type inference. Or at least they can. Julia does type inference at compile time. The difference is that it doesn't have to succeed. If you write type unstable code then it won't.
- throwaway894345 7y ago
  Presumably this means that no one has implemented a suitable static type checker or its type annotations don't provide sufficient information for a static type checker to work in principle?
  - eigenspace 7y ago
    The problem is more the former (nobody has gotten around to it yet), but also a bit of the opposite of the latter, i.e. the type system is so rich, type-checking may be undecidable in too many contexts. Julia's parametric types are incredibly rich. You can look up 'dependant types' which are similar in the static world and see the sorts of hoops static language designers jump through to have dependant types and still be able to reliably prove theorems.
  - socialdemocrat 7y ago
    What you can do with Julia types is far more advance than I think you can do with any statically typed language. I can add a type annotation in Julia which is a function call which returns a type e.g. This function can be of any complexity. I think a better approach is simply a Linter. You report type problems you can find and ignore the rest. Not point trying to figure out the more advance Julia cases. I would assume 90% of Julia code will use types in a pretty straightforward manner and thus would be able to be analyzed by some sort of Linter. Lint.jl e.g.
    - throwaway894345 7y ago
      You're describing optional type systems; in this regard, Julia is on par with Python or TypeScript. The linter you're describing is also a type checker (like Python's Mypy). That said, it would be really interesting if someone would make a type checker that would evaluate/resolve those more complex types, at least those that can be guaranteed to terminate.
      - socialdemocrat 7y ago
        Optional typing and what Julia does is quite different. Julia type system is nothing like Python or TypeScript. In Julia type annotations are not just a sprinkle of sugar. It has a very concrete use beyond asserting type consistency. Julia types are used to decide what code gets run and memory layout of data.
- socialdemocrat 7y ago
  You can do some form of type checking with stuff like Lint.jl. I mean since Julia uses a bunch of type information and Julia parses code into data structures which are easy to process, you can to a lot of your own type verification before running a program. However I see Julia not being a statically typed language as a feature. If Julia was statically typed you would pretty much have lost everything that makes Julia a great language. You cannot do multiple dispatch with a statically typed language efficiently.

FranzFerdiNaN 7y ago

Python will still be used 20 years from now. The clear advantage of Python is the enormous ecosystem that is available, the millions of questions on SO giving solutions to every problem you can run into, the books and learning materials etc, programmers and corporations having invested loads of time and effort in building, maintaining and battle-testing libraries. Don't get me wrong, i think Julia is an amazing language, but being an amazing language is neither necessary or sufficient to succeed. R shows how you can succeed just fine with a kinda weird language.

logicchains 7y ago
>The clear advantage of Python is the enormous ecosystem that is available, the millions of questions on SO giving solutions to every problem you can run into All of this is also available in Julia via its near-seamless Python interop. Its package manager even does a better job of managing Python packages than Python does.
ScottPJones 7y ago
Python's ecosystem is great - but Julia's is growing incredibly fast, and in some cases Julia has already surpassed what is available in other languages (for example, take a look at the whole differential equations ecosystem: https://github.com/JuliaDiffEq https://github.com/JuliaDiffEq). Also, Python's ecosystem is only a 'pyimport(name)' away (using the PyCall.jl package). Same thing is true for R and a number of other languages (RCall.jl, JavaCall.jl, etc.) I've been using SymPy, QisKit, matplotlib and other Python packages with no problem in Julia.
mumblemumble 7y ago
I would be careful treating this too much like a popularity contest. Python will still be used 20 years from now, but that doesn't necessarily mean that it's going to be dominant 20 years from now. Source: I used to do a lot of programming in various dialects of Basic, which, a bit over 20 years ago, was popular largely because of its own ubiquity and popularity. And, while I still maintain some Basic code, I was surprised how quickly it died. One day everything was being written in it. The next day, we were were writing new things in a fancy new language that everyone agreed was more productive, and talking to the existing stuff through an FFI. And, a day later, we were replacing modules in order to get them off of the "legacy" platform.

mark_l_watson 7y ago

I have a collection of short Julia code snippets that do non numeric things like text processing, semantic web clients, etc. But in general I agree. Fir me, Lisp languages, Haskell, and Python are more general purpose.

vasili111 7y ago
>Fir me, Lisp languages, Haskell, and Python are more general purpose. You mean Julia as language is not enough good for general purpose or it lacks ecosystem (frameworks, libraries) for general purpose use?
- smabie 7y ago
  A little of both, I guess. The biggest issue is that I, personally, get uncomfortable with dynamic typing when the project starts becoming over 1-5 Kloc. So I tend to stick with stuff like Scala, Rust, or OCaml for non data science tasks.
- mark_l_watson 7y ago
  I didn’t express that correctly. What I meant to say was that I investigated Julia as a possible “use it for everything language.” For me, general purpose has to include deep learning tools. I really prefer Lisp and Haskell, but although Haskell’s TensorFlow support is pretty good, I found it easier to just use Python. I have had problems with the SBCL Common Lisp TensorFlow C bindings, but perhaps that was my fault.
  - iLemming 7y ago
    AFAIK Clojurians doing some interesting stuff in that space. They've figured out Python Interop, writing books on Deep learning, Linear Algebra, etc. all that in Clojure. https://aiprobook.com https://aiprobook.com https://nextjournal.com/gigasquid https://nextjournal.com/gigasquid
e12e 7y ago
What part of Julia isn't lispy enough that you don't consider it a lisp? I'm not saying you're wrong, just because there a scheme in there, just curious.

ScottPJones 7y ago

I've been using Julia for non-scientific computing programs for almost 5 years now, and (especially now that it is stable since the 1.0 release) have found it well suited for general programming as well. Having a language that is easy to write (like Python), runs fast (like C/C++), and incredibly flexible & expressive (like Lisp) makes programming fun again!

skybrian 7y ago
Interesting, how does deployment work for you? What sort of executables do you ship?
- nnq 7y ago
  Don't people mostly ship containers nowadays? Or deploy code but standardize on language version. Why t f would you want to ship / deploy a naked executable in 2019?!
  - skybrian 7y ago
    Containers are common for server-side code but not for mobile, desktop, or client-side web code.
    - MiroF 7y ago
      e: ignore
  - unlinked_dll 7y ago
    Not all of us are web developers.
  - throwaway894345 7y ago
    * Container images are generally huge. Layer caching helps to the extent that layers are reusable. If you're lucky, you can use something like alpine for smaller huge images. If you write in a language that supports statically linked executables (e.g., Go) then you can get by with a scratch in many cases, but then you could more easily ship a naked executable * Container images take a long time to build and build caching only works well if your dependency tree is linear (it's not). (Caveat: there are niche alternatives to docker build for which this may not hold, but then you're subject to the other issues with niche tools--support, docs, compat, etc) * Your deployment target has to have a container runtime compatible with your image format. This is a PITA for end users in many cases. * Most popular container runtimes make container processes children of the daemon process, not children of the client process. This means killing the parent process will not kill the child process, which means CI environments need to handle all sorts of edge cases. For example, a CI job is waiting on a stuck container to terminate, but eventually times out or the job is killed or similar; each time this happens a container is leaked--this means that CI tools have to support the container runtime explicitly and handle all of these edge cases because it's prohibitively complex for users to manage. IMO the only advantage (and this is a huge advantage where applicable) for containers is that they can be orchestrated by things like Kubernetes or AWS Fargate.

real_eng_ 7y ago

Why every Julia user can't help but trash Python at every occasion? It's getting really tiring

ScottPJones 7y ago
I haven't seen that at all - many Julia programmers are (or were) also Python programmers. I think there is a lot of respect in the Julia community for Python & the Python ecosystem. There have even been a number of Julia talks at various PyCons over the past few years.
- oxinabox 7y ago
  There was a python talk at the 2019 juliacon.
smabie 7y ago
Trashing is a little harsh, but Julia devs probably were mostly Python devs back in the day and are intimately familiar with the inadequacies of the language: the work required when you had to drop into C, the bad syntax for math, the constant conversions between ndarray, array, and lists, etc etc. At my old bioinformatics lab, Python literally wasted thousands and thousands of collective man-hours which would have been saved by Julia if it had existed at the time. Since a lot of these researchers couldn’t really program that well, they would write code that would literally take weeks to run. And then they would call me and I would rewrite it in C and it would then take an hour or two. Julia solves this problem. The amount of unnecessary supercomputer time (and electricity) that our lab (and others) were wasting with Python was, honestly, disgusting.
- elcritch 7y ago
  Exactly... well I didn’t do supercomputing but still wasted a lot of time dealing with Python 2/3 issues among others. Then there’s the ability to do things like fast bootstrapping, Monte-Carlo and other handy statistical techniques without writing kernel functions in C. Python was great for it’s time and enabled a lot and still let’s lots of people do great things, it’s just not for me anymore.

vanderZwan 7y ago

> It’s too bad the typing is only for dispatch, but hopefully someone will write a typechecker someday I was under the impression that if you write a function that takes very specific sub-types as its parameters, and then try to call it with invalid parameters (that is, values that cannot be converted to the right type via the promotion rules) that Julia will complain about this?

deleted 7y ago
[deleted]
tgflynn 7y ago
It will complain, but typically only at run time.
- vanderZwan 7y ago
  Ah, right! We're talking about AOT typechecking. I had forgotten Julia doesn't really do that (yet). Yes, fair is fair, since that is when it is the most useful its absence is a bit painful.