21 ms·

Why I still recommend Julia

219 points by amkkma 4y ago

skybrian 4y ago
Maybe Julia should have an easier way to know if you're using a combination that's been tried before and is supported?
- abhimanyuaryan 4y ago
  this is actually interesting but sounds very complex. How about writing tests?
  - throwaway290 4y ago
    > How about writing more code? FTFY. Tests are code. This code needs to be maintained, since it's coupled to your architecture. This code has bugs of its own. Etc. Writing tests is a fine choice if you have the resources, and also isn't a valid answer to a criticism of the language.
dontbenebby 4y ago
- melissalobos 4y ago
  > niche programming language I mean `niche` is a little weird here since Julia is very specifically a general purpose programming language. It just happens to be really really good at math. [1] https://julialang.org/#tab-general https://julialang.org/#tab-general
  - sk1pper 4y ago
    > Julia is very specifically a general purpose programming language I’ve been trying to figure this out recently - because I love Julia’s features. Readable like Python, but with more ability to optimize performance, and also lispy with macros and generic functions. I’m personally interested in it as a general purpose language. But when I search around about it, most folks to seem to relegate it to the data science realm only. Everyone seems to be saying: well it’s certainly general purpose capable, but its designers are focused on data science, and that will continue to be the primary goal. As such, don’t expect to see it widely adopted as outside of data science anytime soon. I don’t want that to be the case, but it seems harder to build broader excitement about the language if it’s going to continue to be perceived as niche.
    - krastanov 4y ago
      Bringing up data science as its niche is somewhat funny, because the most mature libraries in the language (where it is light years ahead of other ecosystems) are in the general sciences (e.g. differential equations, math optimization, etc). It is true that the Julia ecosystem as a whole is mature only in a few niches, but focusing specifically on narrow data science claims would make me doubt the knowledge of the person making that claim.
melissalobos 4y ago
I do think that Julia in general isn't the best for interacting with C(based off of some experimenting 2 years ago). I have tried using it with a decent number of odd C libraries used in Scientific Computing(with actual capital S and C. Which always involved binary blob libraries.). I think if you control more of the stack it is a better language. But when using it with vendor provided binaries I ran into many many issues. > "with actual capital S and C." This part was referencing the industry and not me being egotistic.
- celrod 4y ago
  I find `ccall` and especially `@ccall` easy to use, but thankfully haven't spent much time wrapping C libraries so I'd consider myself far from an expert in the matter. Creating mutable structs and `GC.@preserve`ing them is an effective means of getting stack allocated memory, so long as the structs do not escape. E.g., I occasionally follow this approach: mem = Ref{NTuple{32,Float64}}() GC.@preserve mem begin p = Base.unsafe_convert(Ptr{Float64}, mem) # do things with p, e.g. pass it to a C library end
  - deleted 4y ago
    [deleted]
- abhimanyuaryan 4y ago
  but the whole point of using Julia is that you don't need C(relatively same speed). Why would you interface it a Scientific Computing library written in C. Also C doesn't really sound like a good language for Scientific Computing. People do SC in Python and which is why Julia i.e. to solve two language problems and mind boggling speeds like C with syntax like Python
  - uecker 4y ago
    C is an excellent language for scientific computing. A lot of low-level and high-level libraries for scientific computing are written in C. Toolchain support is excellent. It is maybe less good a putting some scripts together quickly.
ssivark 4y ago
To phrase succinctly the (direct) counter to the previous blog post: Julia makes possible the kind of combinatorial composition (and therefore, radical modularity & reuse) that is simply not possible in most languages. This will lead to some friction as the community figures out the right design patterns in these uncharted waters, but on the flip side one already gets many superpowers, provided one is careful about testing the composition works as expected, rather than just blindly assuming it does.
- SnowHill9902 4y ago
  Can you give some examples?
  - adgjlsfhk1 4y ago
    the simplest I know of is Complex{Rational{Int}}. works in Julia, breaks in python.
    - ChrisRackauckas 4y ago
      And that works and optimizes in the differential equation solvers.
blindseer 4y ago
The example using Measurements.jl only works as a good example for composibility if you use `Number` as the type for the argument. If you happen to use Int then it doesn't work: using Measurements plus_one(x::Int) = x + 1 plus_one(measurement(1)) ERROR: MethodError: no method matching plus_one(::Measurement{Float64}) Closest candidates are: plus_one(::Int64) at REPL[2]:1 Stacktrace: [1] top-level scope @ REPL[3]:1 This is perhaps my biggest annoyance with Julia. Concrete types cannot be subtyped, and hence cannot be extended by other packages. And explaining to researchers and scientists that they need to use abstract types to make their code more composable is exercise in frustration. I think computer scientists with experience programming in C++ may have good intuition for when to use concrete or abstract types in the function signatures, but research scientists in ML and optimization (in my experience) just don't do a good job of that. And just end up having awful Julia code to work with, that isn't really extendable. In my opinion, readability suffers greatly when you have to understand the type hierarchy in order to find out if your code will MethodError or not. For example, if the author use `plus_one(x::Rational) = x + 1` instead of `plus_one(x::::Number) = x + 1`, would the code that uses Measurements.jl have worked? Who knows just by looking at the code. It turns out it doesn't work and there's a MethodError. The built in type hierarchy is great, but third party packages are hit and miss. Honestly a simple solution to this would be to allow concrete types to be supertypes of other types. My understanding is that there's no compiler related reason for this, and this is just for "good practices" but for the life of me I don't understand why this limitation was made. I've seen Julia code that creates a supertype abstract type for every concrete type they create, and it is just awful to deal with. Fortunately, it is somewhat easy to write a macro to make this happen, but still very annoying that one has to do this at all. Combine that with the lack of a file system package modules (like in Python or Rust), the lack of any kind of doctests, the built in testing being so lackluster, you end up with really poorly organized code. TSCoding has a 1 min video on why he doesn't program in Haskell anymore [1] and I can't help but feel his reasons directly apply to Julia too. For example, in order to run that example for this comment, I had to run `add Measurements` and it took 7 minutes to download the package. Why is the "registry" for packages in Julia a giant git repository? It takes SO long to update every time. This is an example of a poor software engineering decision in an otherwise elegant language. [1] https://www.youtube.com/watch?v=SPwnfSmyAGI https://www.youtube.com/watch?v=SPwnfSmyAGI
anigbrowl 4y ago
I really like Julia, and I think this article does a great job of defending its design decisions. But I'm a bit disappointed that it doesn't address Yuri's main criticism, which is that there's some worrying correctness bugs in Julia that aren't addressed as well as they should be. I'm no language expert, so I can't tell if Yuri's criticisms are just a function of the smaller number of people working on Julia vs a very popular language like Python, or whether the failings he discusses stand out because so many things Just Work in Julia, or whether it indicates some much deeper problem, so I've gone back to look at the original thread - https://news.ycombinator.com/item?id=31396861 https://news.ycombinator.com/item?id=31396861 I'm interested to hear what others think of Julia's longevity prospects. Its popularity in climate modeling and other scientific contexts suggest it will be around for the long haul, and that Julia projects needn't end up as abandonware for lack of footing. Is that accurate?
- oivey 4y ago
  I think the language has amassed a nearly unassailable lead in differential equations, scientific machine learning, and (to a lesser degree) optimization. That lead hinges on advantages of the language, so I think others are unlikely to catch up. The story for working with GPUs is also really good compared to e.g. Python. I think those things alone are enough to keep the language around for a while.
  - andi999 4y ago
    What is your evidence? Especially machine learning is done a lot in industry and I think nobody has even heard of Julia outside academic circles.
    - semi-extrinsic 4y ago
      I guess when GP says "scientific machine learning" he means "research into machine learning algorithms", which is typically done in academic circles, no? Then when someone's discovered something new and useful, the focus turns towards making fast and readily available implementations in commonly used frameworks. Like Torch, which you can use both from PyTorch and from Julia.
elchief 4y ago
Julia was always gonna be the RC Cola of data science languages. Python and R were too far ahead and had too much mind share
- shadowofneptune 4y ago
  Am I the only one who actually likes the taste of RC the most? That aside, Julia is relatively young for a programming language, 10 years old. It's still possible for it to find a niche, even if it is not the one it aimed for. It took until the late 00s for Python to enter the data science niche in the first place.
  - anothernewdude 4y ago
    Sure, RC is fine, but if you 1-base your arrays then you chose to be the pariah. I have no patience for technology that just chooses to be special for the sake of it.
    - throwaway290 4y ago
      This argument could be made about any language, though. Python's indentation significance? Rust's match (vs switch)? R's arrow assignment?
    - stellalo 4y ago
      Let’s be serious: I yet have to see a convincing argument for 0-based being better than 1-based, or the other way around. And designing a language based on “what everybody else does” is definitely not the right approach.
      - bzxcvbn 4y ago
        If there are no convincing arguments one way or the other, then "what everybody else does" becomes the convincing argument. Why change established conventions for no good reason? There's no reason for curly braces to indicate control blocks and square braces to indicate indexing, but if a language swapped the two, what would you say?
systems 4y ago
I like Julia a lot I think it needs two things (for me) 1. better database support, also need to support BI Cubes likes MS SSAS 2. fix scoping rules I created a small feature request on github, where I suggest a fix for their scoping (inspired a lot by perl), and I would like to take this opportunity to promote my suggestion, I think its a good one, that will make Julia nicer https://github.com/JuliaLang/julia/issues/37187 https://github.com/JuliaLang/julia/issues/37187
the__alchemist 4y ago
> Finally, it is also likely that CPUs will become faster over time and that tooling around reducing TTFX will improve. I don't agree with this as a mitigation. This line of reasoning is why despite steady hardware improvements over the past few decades, responsiveness of (PC) programs and websites have stagnated or regressed. To the main TTFX issue - I won't consider Julia until this is taken seriously.
- celrod 4y ago
  Many of us in the Julia community (myself included) take it very seriously and spend a substantial amount of our time working to mitigate it.
- phonebucket 4y ago
  The quote provided explicitly suggests that tooling around reducing TTFX should also improve. It’s not suggesting improved CPU power as the sole mitigation.
- skohan 4y ago
  I totally agree, leaning on Moore's Law to solve performance issue doesn't work. When you have performance issues on the language itself, you're essentially paying a performance tax on every piece of code that executes, in a compounding way. So if Julia became a widespread programming language for general applications, you have those small delays leaking into all sorts of operations, and before you know it your whole system feels slow and bogged down. I have been testing some performance oriented software recently, and it's amazing how much more productive I feel when every keypress is immediately reflected on screen with no delay. We can adapt to poor performance, but it adds a constant cognitive load to deal with it. In my mind Julia is suited for limited applications, like doing work in a jupyter notebook, but is not suited for general applications unless the TTFX issue can be fixed.
  - jakobnissen 4y ago
    In my opinion, there is no such thing as a "general application". What about generating static websites? Is that a "general application"? Is TTFX a big issue there? Currently, the Julia package Franklin.jl takes around 15 seconds to create the website the first time (changes in the source code update the page in less than one second). I'd argue no. What about Julia to produce generative art? Is TTFX an issue there? Etc. If "general applications" means _all possible applications_, then I'm not sure any language is suitable for general applications. Julia has weaknesses which limits what it's good at. Right now I think it has more weaknesses than other languages. Some of these limits will hopefully be removed or reduced in the future. But fundamentally, I don't see Julia as some kind of domain specific or niche language. Just like any language, it just has its own set of tradeoffs.
nxpnsv 4y ago
I wonder who needs the introduction to objects to parse the rest of the post… it seems challenging
bzxcvbn 4y ago
How can the author make the argument that OOP makes things too tightly coupled, then claim that it's up to the developer to check if every single coupling in their code is correct? The coupling is still tight, it's just not enforced by the compiler. There's a reason for tight coupling in OOP: it's better to let the compiler check for correctness rather than every single developer out there.
- phonebucket 4y ago
  > to check if every single coupling in their code is correct? The author is not addressing the couplings in their code itself, but the code couplings between their code and all libraries in the Julia ecosystem, which is an altogether more audacious goal.
practal 4y ago
I just came across Yuri's criticism for the first time, but it makes sense to me. I am not a user of Julia, but have followed it with interest since they published their first paper about it. With hindsight, it is clear that they would run into correctness issues due to their powerful composability features. The solution is obvious and hard at the same time: there must be a way to PROVE correctness. Of course, to incorporate a prover into Julia will be pretty hard, it is probably much easier to incorporate (some of) the ideas of Julia into a new shiny prover.
- mgaunard 4y ago
  any language with generic programming has the same issues. This is in no way specific to Julia.
  - practal 4y ago
    I think it is specific, because a) Julia is a dynamic language, and b) it uses dynamic multiple dispatch. I think these features are great, but on their own they lead to exactly the situation as described.
    - mgaunard 4y ago
      so you get type errors at runtime rather than compile-time. That's a known problem with dynamic typing.
      - practal 4y ago
        As far as I understand the situation the problem is that you do NOT always get type errors during runtime. Instead, you just get a wrong result, because the combination is legally allowed (that means, the types are accepted), but has not been foreseen and is handled wrongly. For example with Python, there is also dynamic typing, but not dynamic multiple dispatch. There do not seem to be correctness problems of the same sort as in Julia in Python. So dynamic multiple dispatch, or maybe just Julia's version of it, seems to be the culprit. But an in-depth analysis of this is needed before making a final verdict. But as I said before, it seems inevitable: dynamic multiple dispatch leads to an explosion of possible combinations, how do you make sure that all of these combinations work as expected? And what is "expected" in the first place?
tpoacher 4y ago
Btw, you can totally have classes in Julia. And methods will be compiled like all other functions. It's just not considered particularly idiomatic.
rmbyrro 4y ago
> One such operation can be to sell. The sell function can do things like marking the book object as sold Maybe this isn’t a good example of OOP? Does it suggest the author perhaps doesn't fully understand principles behind loosely coupled OO code? If is_sold is an attribute of book, the class shouldn't have a sell method, but a mark_as_sold. The action of selling is not the responsibility of a Book. Baking it into Book will create tight coupling between areas of the code (product and commerce) that should be loosely coupled. The "commerce" side may change for reasons unrelated to the "product" side. Keep'em separated. And if we think for a minute, is_sold should not be a Book attribute in the first place. It seems to me this piece of information should be handled somewhere else related to inventory, not by the Book itself. There should be a ProductEntry class, for instance. It could have a product_type and product_sku attributes, for example, pointing to a Book. Even here, is_sold shouldn't be an attribute, but a method. It would have an availability_status attribute. func is_sold(self) { return self.status == ProductStatus.SOLD } Ultimately, the inventory code should not care that it's a Book or whatever. The logic of whether it's available or not doesn't belong to the product itself. What if someone bought, but returned it? Maybe it will be available for shipping tomorrow and the store wants to start reselling it already? Is it the Book responsibility to track that? No way.
- vnorilo 4y ago
  Seems to me that you are describing something that should be a relational database, and where object orientation doesn't seem to bring anything of value.
  - thecompilr 4y ago
    That OP just pointed how the provided example would actually work in an OOP project, not their fault to have to work with an example that is indeed an inventory management problem.
- croes 4y ago
  Isn't the point that in OOP you need to think about the design of your objects but in Julia this isn't necessary? Your critique seems to prove the author's claim.
anonymoushn 4y ago
> Based on the amount and the ferocity of the comments, it is natural to conclude that Julia as a whole must produce incorrect results and therefore cannot be a productive environment. However, the scope of the blog post and the discussions are narrow. In general, I still recommend Julia because it is fundamentally productive and, with care, correct. If TFA really means "Most of the time if statements take the correct branch, and it's unusual for them to take the incorrect branch, and you should use tests to detect whether the if statements in your program are working" then I would appreciate it if TFA would come out and say that. I am sort of used to programming in environments in which if statements work 100% of the time.
- Sukera 4y ago
  Aside from me disagreeing with this article here in that I don't think "all of julia is incorrect" is a valid take from Yuri's original article, I'm not sure how you arrive at "if branches don't do what I think they do" from that sentence. Are you referring to one (already fixed) bug[0]? [0]: https://github.com/JuliaLang/julia/issues/41096 https://github.com/JuliaLang/julia/issues/41096
  - anonymoushn 4y ago
    I am also thinking about an issue where catch randomly does not catch exceptions. I have not found the right issue in the issue tracker, so I don't know whether this is fixed.
    - ModernMech 4y ago
      Can you provide some more detail on this issue? That sounds… not good.
baazaa 4y ago
R has multiple dispatch right with s4, I don't recall running into huge library compatibility issues with R? Is the only difference between R and Julia maturity where R has ironed out the sorts of bugs Yuri was running into?
- krastanov 4y ago
  I think the difference is that in Julia multiple dispatch is the main paradigm to structure code (together with very aggressive devirtualization/specialization/compilation). That enables quite amazing things. Other languages have multiple dispatch as well, but it is not foundational to the ecosystem in them. They lack the "magic" but they also have lower propensity for the interface mismatch bugs described through these two threads.
einpoklum 4y ago
> The idea is that when you are, for example, writing business logic with books, then you want to neatly put the operations on the book near the definition of the book. That's not the idea. You put the code for the "internals" of a book near to its definition. In other words: The code that needs to see the private/protected members of a class. It is not good OOP - if I may be so bold as to use that phrase - to stick as much functionality as you can involving some class, into that class.
smarx007 4y ago
I don't see how it addresses the original complaint. Vishnevsky basically stated that if you are trying to run a scientific experiment on a supercomputer, maybe it's a risky idea to use a new programming language with a new stdlib and a bunch of OSS libraries vs using an old language like C with very stable set of existing code because new things tend to have unknown bugs? Vishnevsky has a point, but unless you are running some critical computations on supercomputers, maybe it doesn't apply to you? To be clear, in supercomputing environments people still use old versions of CentOS just to make sure that library version updates do not change their computation results. I don't think many people here would say "I am sticking to Ubuntu 16.04 because I am afraid that the updates to some library like gmplib will slightly change my computation results in a way that is hard for me to detect". Also, just staying with the old doesn't mean it's correct. You can also introduce bugs to your libs. I think NASA thought this through long time ago and solved it by making sure critical parts of the code are implemented twice using different stacks with different programmers. If you are NASA, CERN, LLNL, or a bank, maybe it's a good idea to implement your computations once in Python and once in Julia (by at least two different programmers) and compare the outputs. And I don't think in this situation Julia is any different from other languages (other than you may put too much trust into it and skip this dual implementation). Case in point: https://github.com/scipy/scipy/issues?q=is%3Aissue+is%3Aclosed+label%3Adefect+wrong https://github.com/scipy/scipy/issues?q=is%3Aissue+is%3Aclos...
- ModernMech 4y ago
  > If you are NASA, CERN, LLNL, or a bank, maybe it's a good idea to implement your computations once in Python and once in Julia Doesn’t this negate one of Julia’s main selling points? That it has “solved the two-language problem”. Ironic for them to solve that in the performance domain only to then need a second language to prove correctness.
  - deleted 4y ago
    [deleted]
  - davisoneee 4y ago
    I interpreted it more that, in domains where correctness is vital (rather than "good enough"), you want more than one implementation no matter what languages you use. Maybe that's not what parent was going for, but I think it's like the reproducible/replicable difference in research... can you use the author's code and data, getting the same result... can you use the author's algorithm/pseudocode and data, and get the same result... can you use the author's algorithm/code and different data, and get an _equivalent_ result?
t_mann 4y ago
It seems like the author isn't really defending against the bulk of the criticisms, which revolve around correctness and reliability, and instead highlights 'cool' language features. If you can't trust gradients or probability distributions / sampling to be correct, that's a pretty damning verdict for a language aimed at numerical calculations. The fact that it has some traction in academia (where the link from incorrect computation results to real-world consequences is extremely long-winded and hard to attribute) is worrying from that perspective. Imagine Boeing were to use Julia for its computations (not unlikely given that it has a great repuation for differential equation solvers).
- dekhn 4y ago
  I'm sure Boeing uses Julia somewhere for its computations... and they have extensive testing to support that. I recall when the PR folks at SAS, a statistical tool, said that nobody would trust a Boeing plane that had been developed using R (an open source alternative to SAS), and Boeing replied that they used R to design airplanes.
  - t_mann 4y ago
    Well, it's also not like Boeing didn't have some major engineering f*ups in recent years, but we're going on a tangent here.
  - notagoodidea 4y ago
    Besides that the SAS vs R is something still ongoing. I heard the aame point made in favour of SAS in Public Health, Clinical trials analysis and set up and different public administration. They trust the algorithms provided by a closed source certified vendors and not an open source alternative. At the same time, correct implementation of statistics is hard and testing it to be correct is harder.
    - dekhn 4y ago
      the story you heard about health is exactly backward. I had also heard that FDA approvals depended on SAS, but I spoke more recently with somebody who does this and it turns out both the submittors and the FDA use R, not SAS. There have definitely been communities of stats people who "trust" only certain codes or routines, but generally, the trend has been towards open implementations of the code, as it allows more experts to inspect the workings when they detect problems.
rightbyte 4y ago
My major problem with Julia is that writing performant code is a bit like writing Haskell. You need deep understanding of the compiler to know what constructs will mess up performance. Especially how typeinfo is propagated.
- moelf 4y ago
  at least in Julia you have `@code_warntype`, JET.jl, `@benchmark`, and the language being dynamic also helps a lot.
- adgjlsfhk1 4y ago
  that is a major issue, and I'm not sure if there's a good solution. the one mitigating factor is that the high level languages Julia competes with (Matlab, R and python) have a similar ease of writing much slower code.
  - leephillips 4y ago
    In practice it’s not so major. Following a few simple guidelines from the performance section of the official manual is enough for most people most of the time. And the most important of these guidelines are practices that an experienced programmer would probably already be following, such as not using global variables.
- nsajko 4y ago
  > You need deep understanding of the compiler to know what constructs will mess up performance. Especially how typeinfo is propagated. Could you expand on this? It's not clear whether you're just talking about issues that would trip up beginner who hasn't read the relevant sections in the manual yet.
tharne 4y ago
I've noticed that any time you need to use the word "still" to argue for something, you've usually already lost the debate or the larger point. I can't imagine anyone writing a post along the lines of, "Why I still recommend SQL for querying databases", even though SQL was conceived back in the 70's. "Still", in this context more often than not suggests that a grasping-at-straws is taking place. Julia has a lot of neat features, but at the end of the day, it's a research language rather than a getting things done language.
- DNF2 4y ago
  This seems like some sort of magical thinking. Some random person uses a particular word in a personal blog post (there's nothing official about this), and somehow this influences the technical merits of the subject of the blog post. Can you explain the causal connection here?
amkkma 4y ago
Various people have expressed skepticism of needing language level traits, but it's in vogue to be writing interface packages with no top level abstract types (see Tables.jl) because we don't have some way to extensibly express being part of multiple type trees. It's challenging enough that Abstract Types have no interface specs (it's claimed that this is a feature, allows interfaces to develop over time), but it's even tougher when things get even looser with this new trend towards have untyped functions...even various interface package maintainers themselves have said the current state of affairs is difficult. And if we do want to opt into abstract typing, you're going to be inheriting lots of functionality that could be better broken into smaller typeclass like components (see AbstractArray). I think Julia is screaming for some capability to break up Abstract types and being able to inherit from multiple kinds of things, along with specified interfaces with these things. The problem is that this is very difficult, given the complex subtyping system and multiple dispatch. Adding more complexity exacerbates already present method and typing decidability issues. People are working on this on the periphery, but I hope it can become a priority of the core team and that it's solvable.

219 points by amkkma 4y ago

skybrian 4y ago

Maybe Julia should have an easier way to know if you're using a combination that's been tried before and is supported?

abhimanyuaryan 4y ago
this is actually interesting but sounds very complex. How about writing tests?
- throwaway290 4y ago
  > How about writing more code? FTFY. Tests are code. This code needs to be maintained, since it's coupled to your architecture. This code has bugs of its own. Etc. Writing tests is a fine choice if you have the resources, and also isn't a valid answer to a criticism of the language.

dontbenebby 4y ago

melissalobos 4y ago
> niche programming language I mean `niche` is a little weird here since Julia is very specifically a general purpose programming language. It just happens to be really really good at math. [1] https://julialang.org/#tab-general https://julialang.org/#tab-general
- sk1pper 4y ago
  > Julia is very specifically a general purpose programming language I’ve been trying to figure this out recently - because I love Julia’s features. Readable like Python, but with more ability to optimize performance, and also lispy with macros and generic functions. I’m personally interested in it as a general purpose language. But when I search around about it, most folks to seem to relegate it to the data science realm only. Everyone seems to be saying: well it’s certainly general purpose capable, but its designers are focused on data science, and that will continue to be the primary goal. As such, don’t expect to see it widely adopted as outside of data science anytime soon. I don’t want that to be the case, but it seems harder to build broader excitement about the language if it’s going to continue to be perceived as niche.
  - krastanov 4y ago
    Bringing up data science as its niche is somewhat funny, because the most mature libraries in the language (where it is light years ahead of other ecosystems) are in the general sciences (e.g. differential equations, math optimization, etc). It is true that the Julia ecosystem as a whole is mature only in a few niches, but focusing specifically on narrow data science claims would make me doubt the knowledge of the person making that claim.

melissalobos 4y ago

I do think that Julia in general isn't the best for interacting with C(based off of some experimenting 2 years ago). I have tried using it with a decent number of odd C libraries used in Scientific Computing(with actual capital S and C. Which always involved binary blob libraries.). I think if you control more of the stack it is a better language. But when using it with vendor provided binaries I ran into many many issues. > "with actual capital S and C." This part was referencing the industry and not me being egotistic.

celrod 4y ago
I find `ccall` and especially `@ccall` easy to use, but thankfully haven't spent much time wrapping C libraries so I'd consider myself far from an expert in the matter. Creating mutable structs and `GC.@preserve`ing them is an effective means of getting stack allocated memory, so long as the structs do not escape. E.g., I occasionally follow this approach: mem = Ref{NTuple{32,Float64}}() GC.@preserve mem begin p = Base.unsafe_convert(Ptr{Float64}, mem) # do things with p, e.g. pass it to a C library end
- deleted 4y ago
  [deleted]
abhimanyuaryan 4y ago
but the whole point of using Julia is that you don't need C(relatively same speed). Why would you interface it a Scientific Computing library written in C. Also C doesn't really sound like a good language for Scientific Computing. People do SC in Python and which is why Julia i.e. to solve two language problems and mind boggling speeds like C with syntax like Python
- uecker 4y ago
  C is an excellent language for scientific computing. A lot of low-level and high-level libraries for scientific computing are written in C. Toolchain support is excellent. It is maybe less good a putting some scripts together quickly.

ssivark 4y ago

To phrase succinctly the (direct) counter to the previous blog post: Julia makes possible the kind of combinatorial composition (and therefore, radical modularity & reuse) that is simply not possible in most languages. This will lead to some friction as the community figures out the right design patterns in these uncharted waters, but on the flip side one already gets many superpowers, provided one is careful about testing the composition works as expected, rather than just blindly assuming it does.

SnowHill9902 4y ago
Can you give some examples?
- adgjlsfhk1 4y ago
  the simplest I know of is Complex{Rational{Int}}. works in Julia, breaks in python.
  - ChrisRackauckas 4y ago
    And that works and optimizes in the differential equation solvers.

blindseer 4y ago

The example using Measurements.jl only works as a good example for composibility if you use `Number` as the type for the argument. If you happen to use Int then it doesn't work: using Measurements plus_one(x::Int) = x + 1 plus_one(measurement(1)) ERROR: MethodError: no method matching plus_one(::Measurement{Float64}) Closest candidates are: plus_one(::Int64) at REPL[2]:1 Stacktrace: [1] top-level scope @ REPL[3]:1 This is perhaps my biggest annoyance with Julia. Concrete types cannot be subtyped, and hence cannot be extended by other packages. And explaining to researchers and scientists that they need to use abstract types to make their code more composable is exercise in frustration. I think computer scientists with experience programming in C++ may have good intuition for when to use concrete or abstract types in the function signatures, but research scientists in ML and optimization (in my experience) just don't do a good job of that. And just end up having awful Julia code to work with, that isn't really extendable. In my opinion, readability suffers greatly when you have to understand the type hierarchy in order to find out if your code will MethodError or not. For example, if the author use `plus_one(x::Rational) = x + 1` instead of `plus_one(x::::Number) = x + 1`, would the code that uses Measurements.jl have worked? Who knows just by looking at the code. It turns out it doesn't work and there's a MethodError. The built in type hierarchy is great, but third party packages are hit and miss. Honestly a simple solution to this would be to allow concrete types to be supertypes of other types. My understanding is that there's no compiler related reason for this, and this is just for "good practices" but for the life of me I don't understand why this limitation was made. I've seen Julia code that creates a supertype abstract type for every concrete type they create, and it is just awful to deal with. Fortunately, it is somewhat easy to write a macro to make this happen, but still very annoying that one has to do this at all. Combine that with the lack of a file system package modules (like in Python or Rust), the lack of any kind of doctests, the built in testing being so lackluster, you end up with really poorly organized code. TSCoding has a 1 min video on why he doesn't program in Haskell anymore [1] and I can't help but feel his reasons directly apply to Julia too. For example, in order to run that example for this comment, I had to run `add Measurements` and it took 7 minutes to download the package. Why is the "registry" for packages in Julia a giant git repository? It takes SO long to update every time. This is an example of a poor software engineering decision in an otherwise elegant language. [1] https://www.youtube.com/watch?v=SPwnfSmyAGI https://www.youtube.com/watch?v=SPwnfSmyAGI

anigbrowl 4y ago

I really like Julia, and I think this article does a great job of defending its design decisions. But I'm a bit disappointed that it doesn't address Yuri's main criticism, which is that there's some worrying correctness bugs in Julia that aren't addressed as well as they should be. I'm no language expert, so I can't tell if Yuri's criticisms are just a function of the smaller number of people working on Julia vs a very popular language like Python, or whether the failings he discusses stand out because so many things Just Work in Julia, or whether it indicates some much deeper problem, so I've gone back to look at the original thread - https://news.ycombinator.com/item?id=31396861 https://news.ycombinator.com/item?id=31396861 I'm interested to hear what others think of Julia's longevity prospects. Its popularity in climate modeling and other scientific contexts suggest it will be around for the long haul, and that Julia projects needn't end up as abandonware for lack of footing. Is that accurate?

oivey 4y ago
I think the language has amassed a nearly unassailable lead in differential equations, scientific machine learning, and (to a lesser degree) optimization. That lead hinges on advantages of the language, so I think others are unlikely to catch up. The story for working with GPUs is also really good compared to e.g. Python. I think those things alone are enough to keep the language around for a while.
- andi999 4y ago
  What is your evidence? Especially machine learning is done a lot in industry and I think nobody has even heard of Julia outside academic circles.
  - semi-extrinsic 4y ago
    I guess when GP says "scientific machine learning" he means "research into machine learning algorithms", which is typically done in academic circles, no? Then when someone's discovered something new and useful, the focus turns towards making fast and readily available implementations in commonly used frameworks. Like Torch, which you can use both from PyTorch and from Julia.

elchief 4y ago

Julia was always gonna be the RC Cola of data science languages. Python and R were too far ahead and had too much mind share

shadowofneptune 4y ago
Am I the only one who actually likes the taste of RC the most? That aside, Julia is relatively young for a programming language, 10 years old. It's still possible for it to find a niche, even if it is not the one it aimed for. It took until the late 00s for Python to enter the data science niche in the first place.
- anothernewdude 4y ago
  Sure, RC is fine, but if you 1-base your arrays then you chose to be the pariah. I have no patience for technology that just chooses to be special for the sake of it.
  - throwaway290 4y ago
    This argument could be made about any language, though. Python's indentation significance? Rust's match (vs switch)? R's arrow assignment?
  - stellalo 4y ago
    Let’s be serious: I yet have to see a convincing argument for 0-based being better than 1-based, or the other way around. And designing a language based on “what everybody else does” is definitely not the right approach.
    - bzxcvbn 4y ago
      If there are no convincing arguments one way or the other, then "what everybody else does" becomes the convincing argument. Why change established conventions for no good reason? There's no reason for curly braces to indicate control blocks and square braces to indicate indexing, but if a language swapped the two, what would you say?

systems 4y ago

I like Julia a lot I think it needs two things (for me) 1. better database support, also need to support BI Cubes likes MS SSAS 2. fix scoping rules I created a small feature request on github, where I suggest a fix for their scoping (inspired a lot by perl), and I would like to take this opportunity to promote my suggestion, I think its a good one, that will make Julia nicer https://github.com/JuliaLang/julia/issues/37187 https://github.com/JuliaLang/julia/issues/37187

the__alchemist 4y ago

> Finally, it is also likely that CPUs will become faster over time and that tooling around reducing TTFX will improve. I don't agree with this as a mitigation. This line of reasoning is why despite steady hardware improvements over the past few decades, responsiveness of (PC) programs and websites have stagnated or regressed. To the main TTFX issue - I won't consider Julia until this is taken seriously.

celrod 4y ago
Many of us in the Julia community (myself included) take it very seriously and spend a substantial amount of our time working to mitigate it.
phonebucket 4y ago
The quote provided explicitly suggests that tooling around reducing TTFX should also improve. It’s not suggesting improved CPU power as the sole mitigation.
skohan 4y ago
I totally agree, leaning on Moore's Law to solve performance issue doesn't work. When you have performance issues on the language itself, you're essentially paying a performance tax on every piece of code that executes, in a compounding way. So if Julia became a widespread programming language for general applications, you have those small delays leaking into all sorts of operations, and before you know it your whole system feels slow and bogged down. I have been testing some performance oriented software recently, and it's amazing how much more productive I feel when every keypress is immediately reflected on screen with no delay. We can adapt to poor performance, but it adds a constant cognitive load to deal with it. In my mind Julia is suited for limited applications, like doing work in a jupyter notebook, but is not suited for general applications unless the TTFX issue can be fixed.
- jakobnissen 4y ago
  In my opinion, there is no such thing as a "general application". What about generating static websites? Is that a "general application"? Is TTFX a big issue there? Currently, the Julia package Franklin.jl takes around 15 seconds to create the website the first time (changes in the source code update the page in less than one second). I'd argue no. What about Julia to produce generative art? Is TTFX an issue there? Etc. If "general applications" means _all possible applications_, then I'm not sure any language is suitable for general applications. Julia has weaknesses which limits what it's good at. Right now I think it has more weaknesses than other languages. Some of these limits will hopefully be removed or reduced in the future. But fundamentally, I don't see Julia as some kind of domain specific or niche language. Just like any language, it just has its own set of tradeoffs.

nxpnsv 4y ago

I wonder who needs the introduction to objects to parse the rest of the post… it seems challenging

bzxcvbn 4y ago

How can the author make the argument that OOP makes things too tightly coupled, then claim that it's up to the developer to check if every single coupling in their code is correct? The coupling is still tight, it's just not enforced by the compiler. There's a reason for tight coupling in OOP: it's better to let the compiler check for correctness rather than every single developer out there.

phonebucket 4y ago
> to check if every single coupling in their code is correct? The author is not addressing the couplings in their code itself, but the code couplings between their code and all libraries in the Julia ecosystem, which is an altogether more audacious goal.

practal 4y ago

I just came across Yuri's criticism for the first time, but it makes sense to me. I am not a user of Julia, but have followed it with interest since they published their first paper about it. With hindsight, it is clear that they would run into correctness issues due to their powerful composability features. The solution is obvious and hard at the same time: there must be a way to PROVE correctness. Of course, to incorporate a prover into Julia will be pretty hard, it is probably much easier to incorporate (some of) the ideas of Julia into a new shiny prover.

mgaunard 4y ago
any language with generic programming has the same issues. This is in no way specific to Julia.
- practal 4y ago
  I think it is specific, because a) Julia is a dynamic language, and b) it uses dynamic multiple dispatch. I think these features are great, but on their own they lead to exactly the situation as described.
  - mgaunard 4y ago
    so you get type errors at runtime rather than compile-time. That's a known problem with dynamic typing.
    - practal 4y ago
      As far as I understand the situation the problem is that you do NOT always get type errors during runtime. Instead, you just get a wrong result, because the combination is legally allowed (that means, the types are accepted), but has not been foreseen and is handled wrongly. For example with Python, there is also dynamic typing, but not dynamic multiple dispatch. There do not seem to be correctness problems of the same sort as in Julia in Python. So dynamic multiple dispatch, or maybe just Julia's version of it, seems to be the culprit. But an in-depth analysis of this is needed before making a final verdict. But as I said before, it seems inevitable: dynamic multiple dispatch leads to an explosion of possible combinations, how do you make sure that all of these combinations work as expected? And what is "expected" in the first place?

tpoacher 4y ago

Btw, you can totally have classes in Julia. And methods will be compiled like all other functions. It's just not considered particularly idiomatic.

rmbyrro 4y ago

> One such operation can be to sell. The sell function can do things like marking the book object as sold Maybe this isn’t a good example of OOP? Does it suggest the author perhaps doesn't fully understand principles behind loosely coupled OO code? If is_sold is an attribute of book, the class shouldn't have a sell method, but a mark_as_sold. The action of selling is not the responsibility of a Book. Baking it into Book will create tight coupling between areas of the code (product and commerce) that should be loosely coupled. The "commerce" side may change for reasons unrelated to the "product" side. Keep'em separated. And if we think for a minute, is_sold should not be a Book attribute in the first place. It seems to me this piece of information should be handled somewhere else related to inventory, not by the Book itself. There should be a ProductEntry class, for instance. It could have a product_type and product_sku attributes, for example, pointing to a Book. Even here, is_sold shouldn't be an attribute, but a method. It would have an availability_status attribute. func is_sold(self) { return self.status == ProductStatus.SOLD } Ultimately, the inventory code should not care that it's a Book or whatever. The logic of whether it's available or not doesn't belong to the product itself. What if someone bought, but returned it? Maybe it will be available for shipping tomorrow and the store wants to start reselling it already? Is it the Book responsibility to track that? No way.

vnorilo 4y ago
Seems to me that you are describing something that should be a relational database, and where object orientation doesn't seem to bring anything of value.
- thecompilr 4y ago
  That OP just pointed how the provided example would actually work in an OOP project, not their fault to have to work with an example that is indeed an inventory management problem.
croes 4y ago
Isn't the point that in OOP you need to think about the design of your objects but in Julia this isn't necessary? Your critique seems to prove the author's claim.

anonymoushn 4y ago

> Based on the amount and the ferocity of the comments, it is natural to conclude that Julia as a whole must produce incorrect results and therefore cannot be a productive environment. However, the scope of the blog post and the discussions are narrow. In general, I still recommend Julia because it is fundamentally productive and, with care, correct. If TFA really means "Most of the time if statements take the correct branch, and it's unusual for them to take the incorrect branch, and you should use tests to detect whether the if statements in your program are working" then I would appreciate it if TFA would come out and say that. I am sort of used to programming in environments in which if statements work 100% of the time.

Sukera 4y ago
Aside from me disagreeing with this article here in that I don't think "all of julia is incorrect" is a valid take from Yuri's original article, I'm not sure how you arrive at "if branches don't do what I think they do" from that sentence. Are you referring to one (already fixed) bug[0]? [0]: https://github.com/JuliaLang/julia/issues/41096 https://github.com/JuliaLang/julia/issues/41096
- anonymoushn 4y ago
  I am also thinking about an issue where catch randomly does not catch exceptions. I have not found the right issue in the issue tracker, so I don't know whether this is fixed.
  - ModernMech 4y ago
    Can you provide some more detail on this issue? That sounds… not good.

baazaa 4y ago

R has multiple dispatch right with s4, I don't recall running into huge library compatibility issues with R? Is the only difference between R and Julia maturity where R has ironed out the sorts of bugs Yuri was running into?

krastanov 4y ago
I think the difference is that in Julia multiple dispatch is the main paradigm to structure code (together with very aggressive devirtualization/specialization/compilation). That enables quite amazing things. Other languages have multiple dispatch as well, but it is not foundational to the ecosystem in them. They lack the "magic" but they also have lower propensity for the interface mismatch bugs described through these two threads.

einpoklum 4y ago

> The idea is that when you are, for example, writing business logic with books, then you want to neatly put the operations on the book near the definition of the book. That's not the idea. You put the code for the "internals" of a book near to its definition. In other words: The code that needs to see the private/protected members of a class. It is not good OOP - if I may be so bold as to use that phrase - to stick as much functionality as you can involving some class, into that class.

smarx007 4y ago

I don't see how it addresses the original complaint. Vishnevsky basically stated that if you are trying to run a scientific experiment on a supercomputer, maybe it's a risky idea to use a new programming language with a new stdlib and a bunch of OSS libraries vs using an old language like C with very stable set of existing code because new things tend to have unknown bugs? Vishnevsky has a point, but unless you are running some critical computations on supercomputers, maybe it doesn't apply to you? To be clear, in supercomputing environments people still use old versions of CentOS just to make sure that library version updates do not change their computation results. I don't think many people here would say "I am sticking to Ubuntu 16.04 because I am afraid that the updates to some library like gmplib will slightly change my computation results in a way that is hard for me to detect". Also, just staying with the old doesn't mean it's correct. You can also introduce bugs to your libs. I think NASA thought this through long time ago and solved it by making sure critical parts of the code are implemented twice using different stacks with different programmers. If you are NASA, CERN, LLNL, or a bank, maybe it's a good idea to implement your computations once in Python and once in Julia (by at least two different programmers) and compare the outputs. And I don't think in this situation Julia is any different from other languages (other than you may put too much trust into it and skip this dual implementation). Case in point: https://github.com/scipy/scipy/issues?q=is%3Aissue+is%3Aclosed+label%3Adefect+wrong https://github.com/scipy/scipy/issues?q=is%3Aissue+is%3Aclos...

ModernMech 4y ago
> If you are NASA, CERN, LLNL, or a bank, maybe it's a good idea to implement your computations once in Python and once in Julia Doesn’t this negate one of Julia’s main selling points? That it has “solved the two-language problem”. Ironic for them to solve that in the performance domain only to then need a second language to prove correctness.
- deleted 4y ago
  [deleted]
- davisoneee 4y ago
  I interpreted it more that, in domains where correctness is vital (rather than "good enough"), you want more than one implementation no matter what languages you use. Maybe that's not what parent was going for, but I think it's like the reproducible/replicable difference in research... can you use the author's code and data, getting the same result... can you use the author's algorithm/pseudocode and data, and get the same result... can you use the author's algorithm/code and different data, and get an _equivalent_ result?

t_mann 4y ago

It seems like the author isn't really defending against the bulk of the criticisms, which revolve around correctness and reliability, and instead highlights 'cool' language features. If you can't trust gradients or probability distributions / sampling to be correct, that's a pretty damning verdict for a language aimed at numerical calculations. The fact that it has some traction in academia (where the link from incorrect computation results to real-world consequences is extremely long-winded and hard to attribute) is worrying from that perspective. Imagine Boeing were to use Julia for its computations (not unlikely given that it has a great repuation for differential equation solvers).

dekhn 4y ago
I'm sure Boeing uses Julia somewhere for its computations... and they have extensive testing to support that. I recall when the PR folks at SAS, a statistical tool, said that nobody would trust a Boeing plane that had been developed using R (an open source alternative to SAS), and Boeing replied that they used R to design airplanes.
- t_mann 4y ago
  Well, it's also not like Boeing didn't have some major engineering f*ups in recent years, but we're going on a tangent here.
- notagoodidea 4y ago
  Besides that the SAS vs R is something still ongoing. I heard the aame point made in favour of SAS in Public Health, Clinical trials analysis and set up and different public administration. They trust the algorithms provided by a closed source certified vendors and not an open source alternative. At the same time, correct implementation of statistics is hard and testing it to be correct is harder.
  - dekhn 4y ago
    the story you heard about health is exactly backward. I had also heard that FDA approvals depended on SAS, but I spoke more recently with somebody who does this and it turns out both the submittors and the FDA use R, not SAS. There have definitely been communities of stats people who "trust" only certain codes or routines, but generally, the trend has been towards open implementations of the code, as it allows more experts to inspect the workings when they detect problems.

rightbyte 4y ago

My major problem with Julia is that writing performant code is a bit like writing Haskell. You need deep understanding of the compiler to know what constructs will mess up performance. Especially how typeinfo is propagated.

moelf 4y ago
at least in Julia you have `@code_warntype`, JET.jl, `@benchmark`, and the language being dynamic also helps a lot.
adgjlsfhk1 4y ago
that is a major issue, and I'm not sure if there's a good solution. the one mitigating factor is that the high level languages Julia competes with (Matlab, R and python) have a similar ease of writing much slower code.
- leephillips 4y ago
  In practice it’s not so major. Following a few simple guidelines from the performance section of the official manual is enough for most people most of the time. And the most important of these guidelines are practices that an experienced programmer would probably already be following, such as not using global variables.
nsajko 4y ago
> You need deep understanding of the compiler to know what constructs will mess up performance. Especially how typeinfo is propagated. Could you expand on this? It's not clear whether you're just talking about issues that would trip up beginner who hasn't read the relevant sections in the manual yet.

tharne 4y ago

I've noticed that any time you need to use the word "still" to argue for something, you've usually already lost the debate or the larger point. I can't imagine anyone writing a post along the lines of, "Why I still recommend SQL for querying databases", even though SQL was conceived back in the 70's. "Still", in this context more often than not suggests that a grasping-at-straws is taking place. Julia has a lot of neat features, but at the end of the day, it's a research language rather than a getting things done language.

DNF2 4y ago
This seems like some sort of magical thinking. Some random person uses a particular word in a personal blog post (there's nothing official about this), and somehow this influences the technical merits of the subject of the blog post. Can you explain the causal connection here?

amkkma 4y ago

Various people have expressed skepticism of needing language level traits, but it's in vogue to be writing interface packages with no top level abstract types (see Tables.jl) because we don't have some way to extensibly express being part of multiple type trees. It's challenging enough that Abstract Types have no interface specs (it's claimed that this is a feature, allows interfaces to develop over time), but it's even tougher when things get even looser with this new trend towards have untyped functions...even various interface package maintainers themselves have said the current state of affairs is difficult. And if we do want to opt into abstract typing, you're going to be inheriting lots of functionality that could be better broken into smaller typeclass like components (see AbstractArray). I think Julia is screaming for some capability to break up Abstract types and being able to inherit from multiple kinds of things, along with specified interfaces with these things. The problem is that this is very difficult, given the complex subtyping system and multiple dispatch. Adding more complexity exacerbates already present method and typing decidability issues. People are working on this on the periphery, but I hope it can become a priority of the core team and that it's solvable.