3 ms·

Guys, What's you're opinion on Julia? I am thinking of using it for data science work. Any draw backs? or advantages I should know about?

by max_ 11mo ago

tomtom1337 11mo ago
If you’re young, new to data science and hoping to get a job in it after some time, then I absolutely recommend learning Python instead of Julia. But if you are just interested in learning a new language and trying it in data science OR are not currently looking to enter the data science job market, then by all means: Julia is great and in many ways superior to Python for data science. It’s just that «everyone» is doing data science in Python, and if you’re new to DS, then you should also know Python (but by all means learn Julia too!).
Darmani 11mo ago
I've worked in about 40 languages and have a Ph. D. in the subject. Every language has problems, some I like, some I'm not fond of There is only one language that I have an active hatred for, and that is Julia. Imagine you try to move a definition from one file to another. Sounds like a trivial piece of organization, right? In Julia, this is a hard problem, and you can wind up getting crashes deep in someone else's code. The reason is that this causes modules that don't import the new file to have different implementations of the same generic function in scope. Julia features the ability to run libraries on data types they were never designed for. But unlike civilized languages such as C++, this is done by randomly overriding a bunch of functions to do things they were not designed to do, and then hoping the library uses them in a way that produces the result you want. There is no way to guarantee this without reading the library in detail. Also no kind of semantic versioning that can tell you when the library has a breaking change or not, as almost any kind of change becomes a potentially-breaking change when you code like this. This is a problem unique to Julia. I brought up to the Julia creators that methods of the same interface should share common properties. This is a very basic principle of generic programming. One of them responded with personal insults. I'm not the only one with such experiences. Dan Luu wrote this piece 10 years ago, but the appendix shows the concerns have not been addressed: https://danluu.com/julialang/ https://danluu.com/julialang/
- oconnore 11mo ago
  I guess you had a bad experience, but this hasn’t been an issue for me using it for many years now.
  - almostgotcaught 11mo ago
    Lol is this meant to be an ironic comment?
- dandanua 11mo ago
  I don't quite understand what your issues were, I don't have any with the current module system https://docs.julialang.org/en/v1/manual/modules/ https://docs.julialang.org/en/v1/manual/modules/ Can you provide some concrete examples of that issues existing today?
  - Darmani 11mo ago
    I had a quick skim of the link you gave -- it looks to be the same as it was 6 years ago when I experienced this issue. So basically: Generic function F is defined in file A, and has a method definition DEF in file B. File C imports files A and B, and then calls function G that internally uses generic function F. Somewhere internally, it winds up running F with method definition DEF. You move DEF to file D, but don't update the imports of file B. When file B calls G, it calls F with some default implementation instead of DEF, and then you get an error from function G. Some details here might be wrong over how exactly to imports need to be set up, as I haven't used Julia being traumatized by it 6 years ago, but that's basically it.
    - dandanua 11mo ago
      Well, with generic programming constructs you have more freedom, and with more freedom you have more ways to shoot yourself in the foot. I don't think Julia has some illogical or inconsistent design decisions of its module system. In fact, it's better than many other because the order in which you import modules doesn't matter.
- 11mo ago
tombert 11mo ago
I don’t use it for much data science, and I only have used it for one project (to speed up my dad’s Octave code), but I actually really liked it. Generally speaking things were extremely fast, even without any specific optimization on my end. Like, without hyperbole, a direct port of my dad’s code was between 50-100x faster in most cases, and when I started optimizing it I ended up getting it going about 150x faster. There was a bit of weirdness with the type system with its dynamic dispatch making things slow, but specifying a type in function headers would resolve those issues. I also thought that the macro system was pretty nice; for the most part I found creating custom syntax and using their own helpers was pretty nice and easy to grok. Since I don’t do much data work I haven’t had much of an excuse to use it again, but since it does offer channels and queues for thread synchronization I might be able to find a project to use it for.
dandanua 11mo ago
As a language, Julia is strictly superior to Python in many ways (e.g., it's easy to write multithreading code). Currently Python still can be preferable due to its huge ecosystem and super optimised libraries for data science. But Julia is catching fast.
pjmlp 11mo ago
Drawback, it isn't Python in community scale, remember it was born in 1996, so it had enough time to grow. Advantages, it is yet another Lisp like language in a Algol like syntax, like Dylan and Wolfram Alpha, also another one with multi-methods support a la Common Lisp, Dylan, Clojure, and whoever else implements a subset of CLOS. It was designed from the ground up to be compiled with a JIT, not as an afterthought. These are the kinds of places making use of it, https://juliahub.com/case-studies https://juliahub.com/case-studies
wolvesechoes 11mo ago
I would love to love it, but for now I hate to hate it. Dynamic yet performant language with LISPy features and focus on numerical applications? Count me in. But then I found out that execution of some ideas is rather bad, and some ideas are great on paper, but not in practice. For example, debugging experience is a joke even compared to SBCL debugger (and you of course need to download package Debugger.jl, because who needs a good debugger in base language implementation?) And multiple dispatch is a very powerful feature... I sometimes think it is too powerful. There is no proper IDE, and VSC extension was slow and unstable when I tried it (last time few months ago). But my biggest gripe is with people developing Julia. Throughout the years every time people complained about something ("time to first plot", static compilation etc.) the initial responses were always "you are holding it wrong", "Julia is not intended to be used this way", "just keep you REPL open", "just use this 3rd party package", only to few releases later try to address the problem, sometimes in suboptimal way. It is nice that in the end they try to deliver solutions, but it seems to me it always require constant push from the bottom. Moreover, I am quite allergic to marketing strategies and hype generation: Julia doesn't run like C when you write it like Python. It can be very fast, but then it requires quite detailed tuning. You don't need to think about memory management, until you need to, because otherwise allocations kill your performance. You can omit types, until you can't. Those things are quite obvious, but then why produce so much hype and bullshit people through curated and carefully tuned microbenchmarks? It maybe solves two-language problem, but in return it gives you million packages issue. You need a package to have a tolerable debugger (Debugger.jl), you need a package to have static and performant arrays (StaticArrays.jl), you need a package to have enums worth using, you need a package to hot-reload your code without restarting REPL (Revise.jl), you need a package to compile you code to an executable (PackageCompiler.jl/StaticCompiler.jl, they started to address that in the last release) etc. And then you need to precompile them on your machine to have reasonable startup time. TLDR: Julia is wasted potential.
- hatmatrix 11mo ago
  As for the tooling, julia-snail on emacs is supposed to be like SLIME for Lisp. But sounds like that isn't your main gripe. Having to load so many packages is a indeed a pain, but it does suggest the core language is rather minimal...

Guys, What's you're opinion on Julia? I am thinking of using it for data science work. Any draw backs? or advantages I should know about?

by max_ 11mo ago

Guys, What's you're opinion on Julia? I am thinking of using it for data science work. Any draw backs? or advantages I should know about?

tomtom1337 11mo ago

If you’re young, new to data science and hoping to get a job in it after some time, then I absolutely recommend learning Python instead of Julia. But if you are just interested in learning a new language and trying it in data science OR are not currently looking to enter the data science job market, then by all means: Julia is great and in many ways superior to Python for data science. It’s just that «everyone» is doing data science in Python, and if you’re new to DS, then you should also know Python (but by all means learn Julia too!).

Darmani 11mo ago

I've worked in about 40 languages and have a Ph. D. in the subject. Every language has problems, some I like, some I'm not fond of There is only one language that I have an active hatred for, and that is Julia. Imagine you try to move a definition from one file to another. Sounds like a trivial piece of organization, right? In Julia, this is a hard problem, and you can wind up getting crashes deep in someone else's code. The reason is that this causes modules that don't import the new file to have different implementations of the same generic function in scope. Julia features the ability to run libraries on data types they were never designed for. But unlike civilized languages such as C++, this is done by randomly overriding a bunch of functions to do things they were not designed to do, and then hoping the library uses them in a way that produces the result you want. There is no way to guarantee this without reading the library in detail. Also no kind of semantic versioning that can tell you when the library has a breaking change or not, as almost any kind of change becomes a potentially-breaking change when you code like this. This is a problem unique to Julia. I brought up to the Julia creators that methods of the same interface should share common properties. This is a very basic principle of generic programming. One of them responded with personal insults. I'm not the only one with such experiences. Dan Luu wrote this piece 10 years ago, but the appendix shows the concerns have not been addressed: https://danluu.com/julialang/ https://danluu.com/julialang/

oconnore 11mo ago
I guess you had a bad experience, but this hasn’t been an issue for me using it for many years now.
- almostgotcaught 11mo ago
  Lol is this meant to be an ironic comment?
dandanua 11mo ago
I don't quite understand what your issues were, I don't have any with the current module system https://docs.julialang.org/en/v1/manual/modules/ https://docs.julialang.org/en/v1/manual/modules/ Can you provide some concrete examples of that issues existing today?
- Darmani 11mo ago
  I had a quick skim of the link you gave -- it looks to be the same as it was 6 years ago when I experienced this issue. So basically: Generic function F is defined in file A, and has a method definition DEF in file B. File C imports files A and B, and then calls function G that internally uses generic function F. Somewhere internally, it winds up running F with method definition DEF. You move DEF to file D, but don't update the imports of file B. When file B calls G, it calls F with some default implementation instead of DEF, and then you get an error from function G. Some details here might be wrong over how exactly to imports need to be set up, as I haven't used Julia being traumatized by it 6 years ago, but that's basically it.
  - dandanua 11mo ago
    Well, with generic programming constructs you have more freedom, and with more freedom you have more ways to shoot yourself in the foot. I don't think Julia has some illogical or inconsistent design decisions of its module system. In fact, it's better than many other because the order in which you import modules doesn't matter.
11mo ago

tombert 11mo ago

I don’t use it for much data science, and I only have used it for one project (to speed up my dad’s Octave code), but I actually really liked it. Generally speaking things were extremely fast, even without any specific optimization on my end. Like, without hyperbole, a direct port of my dad’s code was between 50-100x faster in most cases, and when I started optimizing it I ended up getting it going about 150x faster. There was a bit of weirdness with the type system with its dynamic dispatch making things slow, but specifying a type in function headers would resolve those issues. I also thought that the macro system was pretty nice; for the most part I found creating custom syntax and using their own helpers was pretty nice and easy to grok. Since I don’t do much data work I haven’t had much of an excuse to use it again, but since it does offer channels and queues for thread synchronization I might be able to find a project to use it for.

dandanua 11mo ago

As a language, Julia is strictly superior to Python in many ways (e.g., it's easy to write multithreading code). Currently Python still can be preferable due to its huge ecosystem and super optimised libraries for data science. But Julia is catching fast.

pjmlp 11mo ago

Drawback, it isn't Python in community scale, remember it was born in 1996, so it had enough time to grow. Advantages, it is yet another Lisp like language in a Algol like syntax, like Dylan and Wolfram Alpha, also another one with multi-methods support a la Common Lisp, Dylan, Clojure, and whoever else implements a subset of CLOS. It was designed from the ground up to be compiled with a JIT, not as an afterthought. These are the kinds of places making use of it, https://juliahub.com/case-studies https://juliahub.com/case-studies

wolvesechoes 11mo ago

I would love to love it, but for now I hate to hate it. Dynamic yet performant language with LISPy features and focus on numerical applications? Count me in. But then I found out that execution of some ideas is rather bad, and some ideas are great on paper, but not in practice. For example, debugging experience is a joke even compared to SBCL debugger (and you of course need to download package Debugger.jl, because who needs a good debugger in base language implementation?) And multiple dispatch is a very powerful feature... I sometimes think it is too powerful. There is no proper IDE, and VSC extension was slow and unstable when I tried it (last time few months ago). But my biggest gripe is with people developing Julia. Throughout the years every time people complained about something ("time to first plot", static compilation etc.) the initial responses were always "you are holding it wrong", "Julia is not intended to be used this way", "just keep you REPL open", "just use this 3rd party package", only to few releases later try to address the problem, sometimes in suboptimal way. It is nice that in the end they try to deliver solutions, but it seems to me it always require constant push from the bottom. Moreover, I am quite allergic to marketing strategies and hype generation: Julia doesn't run like C when you write it like Python. It can be very fast, but then it requires quite detailed tuning. You don't need to think about memory management, until you need to, because otherwise allocations kill your performance. You can omit types, until you can't. Those things are quite obvious, but then why produce so much hype and bullshit people through curated and carefully tuned microbenchmarks? It maybe solves two-language problem, but in return it gives you million packages issue. You need a package to have a tolerable debugger (Debugger.jl), you need a package to have static and performant arrays (StaticArrays.jl), you need a package to have enums worth using, you need a package to hot-reload your code without restarting REPL (Revise.jl), you need a package to compile you code to an executable (PackageCompiler.jl/StaticCompiler.jl, they started to address that in the last release) etc. And then you need to precompile them on your machine to have reasonable startup time. TLDR: Julia is wasted potential.

hatmatrix 11mo ago
As for the tooling, julia-snail on emacs is supposed to be like SLIME for Lisp. But sounds like that isn't your main gripe. Having to load so many packages is a indeed a pain, but it does suggest the core language is rather minimal...