5 ms·

I was initially quite keen on Julia but this critique rather put me off: https://yuri.is/not-julia/ https://yuri.is/not-julia/ and I've not seen a rebuttal.

kdwikzncba 3y ago
Maybe sometime from the realm of psychology would like to theorize why this article became so popular? I never found it compelling. I think it's because evaluating a language is horribly complicated, and people crave certainty and the article sounds very definive. Maybe people see it as a sort of the article is right/Julia bas vs the article is wrong/Julia good.
phkahler 3y ago
Regarding the @inbounds problem. If removing @inbounds leads to out of bounds memory access, doesn't that mean it was masking a bug? Shouldn't it crash with a message like an assert?
- balencpp 3y ago
  I think the implication is that @inbounds turns out of bound memory accesses into undefined, silent behavior, which can stay hidden and cause weird bugs later on. Out of bounds without using @inbounds leads to an exception indeed.
- jakobnissen 3y ago
  What do you mean precisely? Removing @inbounds should never leads to OOB access. The @inbounds macro disables the boundscheck. If what you mean was that: If the act of removing an @inbounds leads to a bounds exception, then the code was buggy even without the macro. That's true. IIUC, the main issue that Yuri talks about is that Julia is a highly generic language where people share abstractions quite a lot, but there is little agreement on what those abstractions are, and the language provide zero tooling or help with that
  - phkahler 3y ago
    >> What do you mean precisely? Removing @inbounds should never leads to OOB access. The @inbounds macro disables the boundscheck. Oh, that explains it. I thought it enabled a bounds check that somehow allowed code to run rather than fault. So it is a promise to the compiler that the code is correct and doesn't need checking. Composition with arbitrary bounds silently breaks that promise. Go it.
- eigenspace 3y ago
  If @inbounds wasn't used, then yes, it would throw an error, but @inbounds is specifically for removing those error paths. What happened here is that julia has a package called OffsetArrays.jl which can be used to create arrays with arbitrary index offsets. Julia arrays start at index 1 (1-based indexing), but with OffsetArrays you could make 0-based arrays, or 10-based arrays or whatever you want. There happened to be a bunch of code laying around which was older than OffsetArrays.jl and did things like function f(v::AbstractVector) for i in 1:length(v) @inbounds do_something(v[i]) end end which implictly assumed that all abstract vectors could be indexed according to normal 1-based indexing rules. This created a lot of silent problems when people then went and fed offset arrays into functions that were using such incorrect indexing schemes. Instead, modern julia code that works on AbstractArray is supposed to be written using functions like eachindex, or axes, i.e. function f(v::AbstractVector) for i in eachindex(v) @inbounds do_something(v[i]) end end though Julia's compiler is now smart enough that using @inbounds like this is typically unnecessary, since it can see that eachindex(v) can't produce an out-of-bounds index.
  - renox 3y ago
    Couldn't this kind of things be caught by type checking? 1-indexed vectors should be different type than arbitrary offset vector (implicitly convertible in one way only).
    - eigenspace 3y ago
      They are different types, but they share a common supertype (AbstractArray). AbstractArray includes thing like regular Array, and also a whole zoo of other array types (including OffsetArray). The problematic code was incorrectly saying it worked on any subtype of AbstractArray, but was actually written assuming that the arrays were 1-based which is an invalid assumption.
rightbyte 3y ago
I think those points are just the symptoms. The real problem is having an out of tree dependency on LLVM. It is too much work to keep up with. The LLVM project does not care about breaking their external users' code.
- ribs 3y ago
  That would be true for many, many projects, wouldn’t it? Every project that isn’t in the LLVM tree?
  - rightbyte 3y ago
    Yes. Rust surely has way more resources and can handle it better, but dealing with LLVM versions is probably still a big annoyance for them. Imagine trying the build something around gcc's internals, just because the gcc project exposed those as a public API and marketed it as such.
- eigenspace 3y ago
  No, that has nothing to do with any of the problems julia faces. Julia's LLVM is just regular LLVM with some patches that are usually upstreamed. The Julia team is pretty good at keeping up with upstream LLVM.
  - rightbyte 3y ago
    It is the social problem of having people show up and grind maintenance work to just make it compile. Which means other questions are left untouched. It is approximately zero sum. I mean, fundamentally, keeping a C++ abi backwards compatible is a horror story. It like, just doesn't work, for non-trivial projects. So LLVM breaks all the time. https://docs.julialang.org/en/v1/devdocs/llvm/ https://docs.julialang.org/en/v1/devdocs/llvm/ Julia got like 12 custom LLVM passes. They are deep into the LLVM. And they seems to include C++ headers from the LLVM project, not using the C api. In practice you'd want insider maintainers in the LLVM project to guard your dependency like it would have been an in tree front-end.
KenoFischer 3y ago
I don't really know what kind of rebuttal you're looking for, but I will link my HN comments from when this was first posted for some thoughts: https://news.ycombinator.com/item?id=31396861#31398796 https://news.ycombinator.com/item?id=31396861#31398796. As I said, in the linked post, I'm quite skeptical of the business of trying to assess relative buginess of programming in different systems, because that has strong dependencies on what you consider core vs packages and what exactly you're trying to do. However, bugs in general suck and we've been thinking a fair bit about what additional tooling the language could provide to help people avoid the classes of bugs that Yuri encountered in the post. The biggest class of problems in the blog post, is that it's pretty clear that `@inbounds` (and I will extend this to `@assume_effects`, even though that wasn't around when Yuri wrote his post) is problematic, because it's too hard (arguably impossible) to write correctly. My proposal for what to do instead is at https://github.com/JuliaLang/julia/pull/50641 https://github.com/JuliaLang/julia/pull/50641. Another common theme is that while Julia is great at composition, it's not clear what's expected to work and what isn't, because the interfaces are informal and not checked. This is a hard design problem, because it's quite close to the reasons why Julia works well. My current thoughts on that are here: https://github.com/Keno/InterfaceSpecs.jl https://github.com/Keno/InterfaceSpecs.jl but there's other proposals also.
- lheck 3y ago
  > Another common theme is that while Julia is great at composition, it's not clear what's expected to work and what isn't, because the interfaces are informal and not checked. This is THE huge issue when combined with the other variables: - you often have hundreds of depedencies that interlock - you often have dependencies that work with multiple packages, and they will often pull them all into your tree (there are sometimes "bridge packages" written, but that is O(n^2) in the number of packages) - many maintainers do not care about good versioning practices, because they are not enforced, and it is not their problem when they break a bajillion other packages - many people writing Julia code are not software developers. this is great! but they usually don't look at what their testing coverage looks like. often the CI always fails on a package and it's ignored. Would Julia only allow multiple dispatch as a link between those packages, the situation would look a little better. But often the packages also talk through simple accesses of values like `X.someproperty`. I've seen situations where maintainers would add and remove these properties and break their own other libraries. Better "enforcement" of these types of things - however that would look like - would be a huge improvement for the time sink that is maintaining a Julia package.
j_maffe 3y ago
I remember reading this article and being curious I looked at all the correctness issues. I think I found almost all of the issues have been resolved. Of course, they might've gotten priority after the publication of this article and might not mean that the fundamental issues are not their. But I think it is a sign that the language is in continuous improvement.
balencpp 3y ago
Besides the 'correctness' issues, all of the issues that blogpost mentioned were fixed back then ASAP. Most of it arise from the usage of @inbounds, which to be fair, the documentation clearly tells you its usage is unsafe. Is it bad there were unrealized bugs in the official example? Yes. Did it get fixed immediately once reported? Yes. Mistakes happen everywhere else. I don't like the stance the blogpost took at all, and I find it quite misleading as it falsely implies Julia is bug-ridden. Personally, the number of bugs I've faced in Python is far more than the ones I've run into in Julia.
- tastyminerals2 3y ago
  It's not the fact that the bugs were fixed. It would be really strange to revisit this article and find out that they weren't, right? It's the fact that the author has spent years working with the language and published several libraries on the way. His article contains other links to similar experiences other ppl had with the language. There is another informative JAX vs Julia article there (by Patrick Kidger) as well. It's just enough material for all kinds of doubt and this really sucks for the language. I simply cannot image a situation where Julia will be seriously considered in a commercial company after this, not in prod. And what's worse, anyone who will fancy the idea of investing into a new language will have a hard time justifying its worth. After all, when you read about several months of hunting for bugs, you definitely don't want to be that person.
  - davidwritesbugs 3y ago
    >And what's worse, anyone who will fancy the idea of investing into a new language will have a hard time justifying its worth. After all, when you read about several months of hunting for bugs, you definitely don't want to be that person. Yea, that would be me. Julia seemed like a good fit but I don't want to spend months bug hunting someone else's language
adammarples 3y ago
This article is a litany github issues for correctness bugs in library code, every one of which appears to have been fixed long ago now so it's not quite true to say there's been no rebuttal. Whether there remain a lot of correctness bugs in julia library code I can't say, but not these ones any more.
eigenspace 3y ago
It really sucks to see what an inordinately wide audience that article has found, and how little context people reading it have.
Sukera 3y ago
There is no rebuttal because nothing much has really changed culture wise. Sure, the various @inbounds issues and concrete bugs that are mentioned in Yuris post have mostly been addressed, but the larger point (that is, "what can I actually expect/get guaranteed when calling a given function?") definitely hasn't been, at least not culturally (there are parts of the ecosystem that are better at this, of course). Documentation of pre- and postconditions are still lackluster, PRs trying to establish that for functions in Base stall for unclear reasons/don't get followups and when you try to talk about that on Slack retorts boil down to "we're tired of hearing you complain about this" instead of trying to find a systemic solution to that problem. Until that changes, I have large doubts about Yuris post losing relevance. My own efforts (shameless plug, https://github.com/Seelengrab/PropCheck.jl https://github.com/Seelengrab/PropCheck.jl for property based testing inspired by Hedgehog and https://github.com/Seelengrab/RequiredInterfaces.jl https://github.com/Seelengrab/RequiredInterfaces.jl for somewhat formalizing "what methods are needed to subtype an abstract type") are unused in the wider community as far as I can tell, in spite of people speaking highly of them when coming across them. I also don't think Kenos InterfaceSpecs.jl is the way forward either - I think there's quite a lot of design space left in the typesystem the language could do without reaching for z3 and other SAT/SMT solvers. I personally attribute the lack of progress on that front to the lack of coherent direction of the project at large (and specifically not to the failings of individuals - folks are always very busy with their lives outside of Julia development/other priorities). In spite of the fact that making this single area better could be a big boon with more traditional software engineers, which are very underrepresented in the community.
- ChrisRackauckas 3y ago
  There is a culture of well-defined interfaces which are checked at compile time. This is something that was emphasized in recent posts and changes such as: * New trait systems define the allowed inputs into given solvers in order to enforce guarantees https://sciml.ai/news/2022/10/08/error_messages/ https://sciml.ai/news/2022/10/08/error_messages/ * The SciMLOperators.jl interface clarified operators (https://docs.sciml.ai/SciMLOperators/stable/interface/ https://docs.sciml.ai/SciMLOperators/stable/interface/) * The highest level SciMLBase interface got codified (https://docs.sciml.ai/SciMLBase/stable/ https://docs.sciml.ai/SciMLBase/stable/) along with the trait systems it requires. * ArrayInterface.jl (https://docs.sciml.ai/ArrayInterface/stable/ https://docs.sciml.ai/ArrayInterface/stable/) and StaticArrayInterface.jl (https://docs.sciml.ai/StaticArrayInterface/stable/ https://docs.sciml.ai/StaticArrayInterface/stable/) have dropped fallback definitions and require definition of traits for downstream packages in order for any array types to be allowed into package functions * SciMLStyle (https://github.com/SciML/SciMLStyle https://github.com/SciML/SciMLStyle) came into existence and defines a style which avoids any of the behaviors seen in the blog post. * Julia came out with package extensions in v1.9 (https://www.youtube.com/watch?v=TiIZlQhFzyk https://www.youtube.com/watch?v=TiIZlQhFzyk) and with the interface checking, implicit interface support was turned mostly into explicit interface support where packages and types opt-in via defining traits (this is still continuing to evolve but is mostly there). Given all of these changes, most of the things in the blog post would now error in many standard packages without someone explicitly defining interface trait functions to allow an object in that doesn't satisfy the interface which it's claiming to. Of course, not every person or every package has changed, but what I described here are major interface, style, and cultural changes to some of the most widely used packages since 2020.

kdwikzncba 3y ago

Maybe sometime from the realm of psychology would like to theorize why this article became so popular? I never found it compelling. I think it's because evaluating a language is horribly complicated, and people crave certainty and the article sounds very definive. Maybe people see it as a sort of the article is right/Julia bas vs the article is wrong/Julia good.

phkahler 3y ago

Regarding the @inbounds problem. If removing @inbounds leads to out of bounds memory access, doesn't that mean it was masking a bug? Shouldn't it crash with a message like an assert?

balencpp 3y ago
I think the implication is that @inbounds turns out of bound memory accesses into undefined, silent behavior, which can stay hidden and cause weird bugs later on. Out of bounds without using @inbounds leads to an exception indeed.
jakobnissen 3y ago
What do you mean precisely? Removing @inbounds should never leads to OOB access. The @inbounds macro disables the boundscheck. If what you mean was that: If the act of removing an @inbounds leads to a bounds exception, then the code was buggy even without the macro. That's true. IIUC, the main issue that Yuri talks about is that Julia is a highly generic language where people share abstractions quite a lot, but there is little agreement on what those abstractions are, and the language provide zero tooling or help with that
- phkahler 3y ago
  >> What do you mean precisely? Removing @inbounds should never leads to OOB access. The @inbounds macro disables the boundscheck. Oh, that explains it. I thought it enabled a bounds check that somehow allowed code to run rather than fault. So it is a promise to the compiler that the code is correct and doesn't need checking. Composition with arbitrary bounds silently breaks that promise. Go it.
eigenspace 3y ago
If @inbounds wasn't used, then yes, it would throw an error, but @inbounds is specifically for removing those error paths. What happened here is that julia has a package called OffsetArrays.jl which can be used to create arrays with arbitrary index offsets. Julia arrays start at index 1 (1-based indexing), but with OffsetArrays you could make 0-based arrays, or 10-based arrays or whatever you want. There happened to be a bunch of code laying around which was older than OffsetArrays.jl and did things like function f(v::AbstractVector) for i in 1:length(v) @inbounds do_something(v[i]) end end which implictly assumed that all abstract vectors could be indexed according to normal 1-based indexing rules. This created a lot of silent problems when people then went and fed offset arrays into functions that were using such incorrect indexing schemes. Instead, modern julia code that works on AbstractArray is supposed to be written using functions like eachindex, or axes, i.e. function f(v::AbstractVector) for i in eachindex(v) @inbounds do_something(v[i]) end end though Julia's compiler is now smart enough that using @inbounds like this is typically unnecessary, since it can see that eachindex(v) can't produce an out-of-bounds index.
- renox 3y ago
  Couldn't this kind of things be caught by type checking? 1-indexed vectors should be different type than arbitrary offset vector (implicitly convertible in one way only).
  - eigenspace 3y ago
    They are different types, but they share a common supertype (AbstractArray). AbstractArray includes thing like regular Array, and also a whole zoo of other array types (including OffsetArray). The problematic code was incorrectly saying it worked on any subtype of AbstractArray, but was actually written assuming that the arrays were 1-based which is an invalid assumption.

rightbyte 3y ago

I think those points are just the symptoms. The real problem is having an out of tree dependency on LLVM. It is too much work to keep up with. The LLVM project does not care about breaking their external users' code.

ribs 3y ago
That would be true for many, many projects, wouldn’t it? Every project that isn’t in the LLVM tree?
- rightbyte 3y ago
  Yes. Rust surely has way more resources and can handle it better, but dealing with LLVM versions is probably still a big annoyance for them. Imagine trying the build something around gcc's internals, just because the gcc project exposed those as a public API and marketed it as such.
eigenspace 3y ago
No, that has nothing to do with any of the problems julia faces. Julia's LLVM is just regular LLVM with some patches that are usually upstreamed. The Julia team is pretty good at keeping up with upstream LLVM.
- rightbyte 3y ago
  It is the social problem of having people show up and grind maintenance work to just make it compile. Which means other questions are left untouched. It is approximately zero sum. I mean, fundamentally, keeping a C++ abi backwards compatible is a horror story. It like, just doesn't work, for non-trivial projects. So LLVM breaks all the time. https://docs.julialang.org/en/v1/devdocs/llvm/ https://docs.julialang.org/en/v1/devdocs/llvm/ Julia got like 12 custom LLVM passes. They are deep into the LLVM. And they seems to include C++ headers from the LLVM project, not using the C api. In practice you'd want insider maintainers in the LLVM project to guard your dependency like it would have been an in tree front-end.

KenoFischer 3y ago

I don't really know what kind of rebuttal you're looking for, but I will link my HN comments from when this was first posted for some thoughts: https://news.ycombinator.com/item?id=31396861#31398796 https://news.ycombinator.com/item?id=31396861#31398796. As I said, in the linked post, I'm quite skeptical of the business of trying to assess relative buginess of programming in different systems, because that has strong dependencies on what you consider core vs packages and what exactly you're trying to do. However, bugs in general suck and we've been thinking a fair bit about what additional tooling the language could provide to help people avoid the classes of bugs that Yuri encountered in the post. The biggest class of problems in the blog post, is that it's pretty clear that `@inbounds` (and I will extend this to `@assume_effects`, even though that wasn't around when Yuri wrote his post) is problematic, because it's too hard (arguably impossible) to write correctly. My proposal for what to do instead is at https://github.com/JuliaLang/julia/pull/50641 https://github.com/JuliaLang/julia/pull/50641. Another common theme is that while Julia is great at composition, it's not clear what's expected to work and what isn't, because the interfaces are informal and not checked. This is a hard design problem, because it's quite close to the reasons why Julia works well. My current thoughts on that are here: https://github.com/Keno/InterfaceSpecs.jl https://github.com/Keno/InterfaceSpecs.jl but there's other proposals also.

lheck 3y ago
> Another common theme is that while Julia is great at composition, it's not clear what's expected to work and what isn't, because the interfaces are informal and not checked. This is THE huge issue when combined with the other variables: - you often have hundreds of depedencies that interlock - you often have dependencies that work with multiple packages, and they will often pull them all into your tree (there are sometimes "bridge packages" written, but that is O(n^2) in the number of packages) - many maintainers do not care about good versioning practices, because they are not enforced, and it is not their problem when they break a bajillion other packages - many people writing Julia code are not software developers. this is great! but they usually don't look at what their testing coverage looks like. often the CI always fails on a package and it's ignored. Would Julia only allow multiple dispatch as a link between those packages, the situation would look a little better. But often the packages also talk through simple accesses of values like `X.someproperty`. I've seen situations where maintainers would add and remove these properties and break their own other libraries. Better "enforcement" of these types of things - however that would look like - would be a huge improvement for the time sink that is maintaining a Julia package.

j_maffe 3y ago

I remember reading this article and being curious I looked at all the correctness issues. I think I found almost all of the issues have been resolved. Of course, they might've gotten priority after the publication of this article and might not mean that the fundamental issues are not their. But I think it is a sign that the language is in continuous improvement.

balencpp 3y ago

Besides the 'correctness' issues, all of the issues that blogpost mentioned were fixed back then ASAP. Most of it arise from the usage of @inbounds, which to be fair, the documentation clearly tells you its usage is unsafe. Is it bad there were unrealized bugs in the official example? Yes. Did it get fixed immediately once reported? Yes. Mistakes happen everywhere else. I don't like the stance the blogpost took at all, and I find it quite misleading as it falsely implies Julia is bug-ridden. Personally, the number of bugs I've faced in Python is far more than the ones I've run into in Julia.

tastyminerals2 3y ago
It's not the fact that the bugs were fixed. It would be really strange to revisit this article and find out that they weren't, right? It's the fact that the author has spent years working with the language and published several libraries on the way. His article contains other links to similar experiences other ppl had with the language. There is another informative JAX vs Julia article there (by Patrick Kidger) as well. It's just enough material for all kinds of doubt and this really sucks for the language. I simply cannot image a situation where Julia will be seriously considered in a commercial company after this, not in prod. And what's worse, anyone who will fancy the idea of investing into a new language will have a hard time justifying its worth. After all, when you read about several months of hunting for bugs, you definitely don't want to be that person.
- davidwritesbugs 3y ago
  >And what's worse, anyone who will fancy the idea of investing into a new language will have a hard time justifying its worth. After all, when you read about several months of hunting for bugs, you definitely don't want to be that person. Yea, that would be me. Julia seemed like a good fit but I don't want to spend months bug hunting someone else's language

adammarples 3y ago

This article is a litany github issues for correctness bugs in library code, every one of which appears to have been fixed long ago now so it's not quite true to say there's been no rebuttal. Whether there remain a lot of correctness bugs in julia library code I can't say, but not these ones any more.

eigenspace 3y ago

It really sucks to see what an inordinately wide audience that article has found, and how little context people reading it have.

Sukera 3y ago

There is no rebuttal because nothing much has really changed culture wise. Sure, the various @inbounds issues and concrete bugs that are mentioned in Yuris post have mostly been addressed, but the larger point (that is, "what can I actually expect/get guaranteed when calling a given function?") definitely hasn't been, at least not culturally (there are parts of the ecosystem that are better at this, of course). Documentation of pre- and postconditions are still lackluster, PRs trying to establish that for functions in Base stall for unclear reasons/don't get followups and when you try to talk about that on Slack retorts boil down to "we're tired of hearing you complain about this" instead of trying to find a systemic solution to that problem. Until that changes, I have large doubts about Yuris post losing relevance. My own efforts (shameless plug, https://github.com/Seelengrab/PropCheck.jl https://github.com/Seelengrab/PropCheck.jl for property based testing inspired by Hedgehog and https://github.com/Seelengrab/RequiredInterfaces.jl https://github.com/Seelengrab/RequiredInterfaces.jl for somewhat formalizing "what methods are needed to subtype an abstract type") are unused in the wider community as far as I can tell, in spite of people speaking highly of them when coming across them. I also don't think Kenos InterfaceSpecs.jl is the way forward either - I think there's quite a lot of design space left in the typesystem the language could do without reaching for z3 and other SAT/SMT solvers. I personally attribute the lack of progress on that front to the lack of coherent direction of the project at large (and specifically not to the failings of individuals - folks are always very busy with their lives outside of Julia development/other priorities). In spite of the fact that making this single area better could be a big boon with more traditional software engineers, which are very underrepresented in the community.

ChrisRackauckas 3y ago
There is a culture of well-defined interfaces which are checked at compile time. This is something that was emphasized in recent posts and changes such as: * New trait systems define the allowed inputs into given solvers in order to enforce guarantees https://sciml.ai/news/2022/10/08/error_messages/ https://sciml.ai/news/2022/10/08/error_messages/ * The SciMLOperators.jl interface clarified operators (https://docs.sciml.ai/SciMLOperators/stable/interface/ https://docs.sciml.ai/SciMLOperators/stable/interface/) * The highest level SciMLBase interface got codified (https://docs.sciml.ai/SciMLBase/stable/ https://docs.sciml.ai/SciMLBase/stable/) along with the trait systems it requires. * ArrayInterface.jl (https://docs.sciml.ai/ArrayInterface/stable/ https://docs.sciml.ai/ArrayInterface/stable/) and StaticArrayInterface.jl (https://docs.sciml.ai/StaticArrayInterface/stable/ https://docs.sciml.ai/StaticArrayInterface/stable/) have dropped fallback definitions and require definition of traits for downstream packages in order for any array types to be allowed into package functions * SciMLStyle (https://github.com/SciML/SciMLStyle https://github.com/SciML/SciMLStyle) came into existence and defines a style which avoids any of the behaviors seen in the blog post. * Julia came out with package extensions in v1.9 (https://www.youtube.com/watch?v=TiIZlQhFzyk https://www.youtube.com/watch?v=TiIZlQhFzyk) and with the interface checking, implicit interface support was turned mostly into explicit interface support where packages and types opt-in via defining traits (this is still continuing to evolve but is mostly there). Given all of these changes, most of the things in the blog post would now error in many standard packages without someone explicitly defining interface trait functions to allow an object in that doesn't satisfy the interface which it's claiming to. Of course, not every person or every package has changed, but what I described here are major interface, style, and cultural changes to some of the most widely used packages since 2020.