4 ms·

I just came across Yuri's criticism for the first time, but it makes sense to me. I am not a user of Julia, but have followed it with interest since they publis

I just came across Yuri's criticism for the first time, but it makes sense to me. I am not a user of Julia, but have followed it with interest since they published their first paper about it. With hindsight, it is clear that they would run into correctness issues due to their powerful composability features. The solution is obvious and hard at the same time: there must be a way to PROVE correctness. Of course, to incorporate a prover into Julia will be pretty hard, it is probably much easier to incorporate (some of) the ideas of Julia into a new shiny prover.

mgaunard 4y ago
any language with generic programming has the same issues. This is in no way specific to Julia.
- practal 4y ago
  I think it is specific, because a) Julia is a dynamic language, and b) it uses dynamic multiple dispatch. I think these features are great, but on their own they lead to exactly the situation as described.
  - mgaunard 4y ago
    so you get type errors at runtime rather than compile-time. That's a known problem with dynamic typing.
    - practal 4y ago
      As far as I understand the situation the problem is that you do NOT always get type errors during runtime. Instead, you just get a wrong result, because the combination is legally allowed (that means, the types are accepted), but has not been foreseen and is handled wrongly. For example with Python, there is also dynamic typing, but not dynamic multiple dispatch. There do not seem to be correctness problems of the same sort as in Julia in Python. So dynamic multiple dispatch, or maybe just Julia's version of it, seems to be the culprit. But an in-depth analysis of this is needed before making a final verdict. But as I said before, it seems inevitable: dynamic multiple dispatch leads to an explosion of possible combinations, how do you make sure that all of these combinations work as expected? And what is "expected" in the first place?
      - ChrisRackauckas 4y ago
        No, you do get type errors during runtime. The most common one is a MethodNotFound error, which corresponds to a dispatch not being found. This is the one that people then complain about for long stacktraces and as being hard to read (and that's a valid criticism). The reason for it is because if you do xy with a type combination that does not have a corresponding dispatch, i.e. (x::T1,y::T2) not defined anywhere, then it looks through the method table of the function, does not find one, and throws this MethodNotFound error. You will only get no error if a method is found. Now what can happen is that you can have a method to an abstract type, *(x::T1,y::AbstractArray), but `y` does not "actually" act like an AbstractArray in some way. If the way that it's "not an AbstractArray" is that it's missing some method overloads of the AbstractArray interface (https://docs.julialang.org/en/v1/manual/interfaces/#man-interface-array https://docs.julialang.org/en/v1/manual/interfaces/#man-inte...), you will get a MethodNotFound error thrown on that interface function. Thus you will only not get an error if someone has declared `typeof(y) <: AbstractArray` and implemented the AbstractArray interface. However, what Yuri pointed out is that there are some packages (specifically in the statistics area) which implemented functions like `f(A::AbstractArray)` but used `for i in 1:length(A)` to iterate through x's values. Notice that the AbstractArray interface has interface functions for "non-traditional indices", including `axes(A)` which is a function to call to get "the a tuple of AbstractUnitRange{<:Integer} of valid indices". Thus these codes are incorrect, because by the definition of the interface you should be doing `for i in axes(A)` if you want to support an AbstractArray because there is no guarantee that its indices go from `1:length(A)`. Note that this was added to the `AbstractArray` interface in the v1.0 change, which is notably after the codes he referenced were written, and thus it's more that they were not updated to handle this expanded interface when the v1.0 transition occurred. This is important to understand because the criticisms and proposed "solutions" don't actually match the case... at all. This is not a case of Julia just letting anything through: someone had to purposefully define these functions for them to exist. And interfaces are not a solution here because there is an interface here, its rules were just not followed. I don't know of an interface system which would actually throw an error if someone does a loop `for i in 1:length(A)` in a code where `A` is then indexed by the element. That analysis is rather difficult at the compiler level because it's non-local: `length(A)` is valid since querying for the length is part of the AbstractArray interface (for good reasons), so then `1:length(A)` is valid since that's just range construction on integers, so the for loop construction itself is valid, and it's only invalid because of some other knowledge about how `A[i]` should work (this look structure could be correct if it's not used to `A[i]` but rather do something like `sum(i)` without indexing). If you want this to throw an error, the only real thing you could do is remove indexing from the AbstractArray interface and solely rely on iteration, which I'm not opposed to (given the relationship to GPUs of course), but etc. you can see the question to solving this is "what is the right interface?" not "are there even interfaces?" (of which the answer is, yes but the errors are thrown at runtime MethodNotFound instead of compile time MethodNotImplemented for undefined things, the latter would be cool for better debugging and stacktraces but isn't a solution). This is why the real discussions are not about interfaces as a solution, they don't solve this issue, and even further languages with interfaces also have this issue. It's about tools for helping code style. You probably should just never do `for i in 1:length(A)`, probably you should always do `for i in eachindex(A)` or `for i in axes(A)` because those iteration styles work for `Array` but also work for any `AbstractArray` and thus it's just a safer way to code. That is why there are specific mentions to not do this in style guides (for example, https://github.com/SciML/SciMLStyle#generic-code-is-preferred-unless-code-is-known-to-be-specific https://github.com/SciML/SciMLStyle#generic-code-is-preferre...), and things like JuliaFormatter automatically flag it as a style break (which would cause CI failures in organizations like SciML which enforce SciML Style formatting as a CI run with Github Actions https://github.com/SciML/ModelingToolkit.jl/blob/v8.14.1/.github/workflows/FormatCheck.yml https://github.com/SciML/ModelingToolkit.jl/blob/v8.14.1/.gi...). There's a call to add linting support for this as well, flagging it any time someone writes this code. If everyone is told to not assume 1-based indexing, formatting CI fails if it is assumed, and the linter underlines every piece of code that does it as red, (along with many other measures, which includes extensive downstream testing, fuzzing against other array types, etc.) then we're at least pretty well guarded against it. And many Julia organizations, like SciML, have these practices in place to guard against it. Yuri's specific discussion is more that JuliaStats does not.

mgaunard 4y ago

any language with generic programming has the same issues. This is in no way specific to Julia.

practal 4y ago
I think it is specific, because a) Julia is a dynamic language, and b) it uses dynamic multiple dispatch. I think these features are great, but on their own they lead to exactly the situation as described.
- mgaunard 4y ago
  so you get type errors at runtime rather than compile-time. That's a known problem with dynamic typing.
  - practal 4y ago
    As far as I understand the situation the problem is that you do NOT always get type errors during runtime. Instead, you just get a wrong result, because the combination is legally allowed (that means, the types are accepted), but has not been foreseen and is handled wrongly. For example with Python, there is also dynamic typing, but not dynamic multiple dispatch. There do not seem to be correctness problems of the same sort as in Julia in Python. So dynamic multiple dispatch, or maybe just Julia's version of it, seems to be the culprit. But an in-depth analysis of this is needed before making a final verdict. But as I said before, it seems inevitable: dynamic multiple dispatch leads to an explosion of possible combinations, how do you make sure that all of these combinations work as expected? And what is "expected" in the first place?
    - ChrisRackauckas 4y ago
      No, you do get type errors during runtime. The most common one is a MethodNotFound error, which corresponds to a dispatch not being found. This is the one that people then complain about for long stacktraces and as being hard to read (and that's a valid criticism). The reason for it is because if you do xy with a type combination that does not have a corresponding dispatch, i.e. (x::T1,y::T2) not defined anywhere, then it looks through the method table of the function, does not find one, and throws this MethodNotFound error. You will only get no error if a method is found. Now what can happen is that you can have a method to an abstract type, *(x::T1,y::AbstractArray), but `y` does not "actually" act like an AbstractArray in some way. If the way that it's "not an AbstractArray" is that it's missing some method overloads of the AbstractArray interface (https://docs.julialang.org/en/v1/manual/interfaces/#man-interface-array https://docs.julialang.org/en/v1/manual/interfaces/#man-inte...), you will get a MethodNotFound error thrown on that interface function. Thus you will only not get an error if someone has declared `typeof(y) <: AbstractArray` and implemented the AbstractArray interface. However, what Yuri pointed out is that there are some packages (specifically in the statistics area) which implemented functions like `f(A::AbstractArray)` but used `for i in 1:length(A)` to iterate through x's values. Notice that the AbstractArray interface has interface functions for "non-traditional indices", including `axes(A)` which is a function to call to get "the a tuple of AbstractUnitRange{<:Integer} of valid indices". Thus these codes are incorrect, because by the definition of the interface you should be doing `for i in axes(A)` if you want to support an AbstractArray because there is no guarantee that its indices go from `1:length(A)`. Note that this was added to the `AbstractArray` interface in the v1.0 change, which is notably after the codes he referenced were written, and thus it's more that they were not updated to handle this expanded interface when the v1.0 transition occurred. This is important to understand because the criticisms and proposed "solutions" don't actually match the case... at all. This is not a case of Julia just letting anything through: someone had to purposefully define these functions for them to exist. And interfaces are not a solution here because there is an interface here, its rules were just not followed. I don't know of an interface system which would actually throw an error if someone does a loop `for i in 1:length(A)` in a code where `A` is then indexed by the element. That analysis is rather difficult at the compiler level because it's non-local: `length(A)` is valid since querying for the length is part of the AbstractArray interface (for good reasons), so then `1:length(A)` is valid since that's just range construction on integers, so the for loop construction itself is valid, and it's only invalid because of some other knowledge about how `A[i]` should work (this look structure could be correct if it's not used to `A[i]` but rather do something like `sum(i)` without indexing). If you want this to throw an error, the only real thing you could do is remove indexing from the AbstractArray interface and solely rely on iteration, which I'm not opposed to (given the relationship to GPUs of course), but etc. you can see the question to solving this is "what is the right interface?" not "are there even interfaces?" (of which the answer is, yes but the errors are thrown at runtime MethodNotFound instead of compile time MethodNotImplemented for undefined things, the latter would be cool for better debugging and stacktraces but isn't a solution). This is why the real discussions are not about interfaces as a solution, they don't solve this issue, and even further languages with interfaces also have this issue. It's about tools for helping code style. You probably should just never do `for i in 1:length(A)`, probably you should always do `for i in eachindex(A)` or `for i in axes(A)` because those iteration styles work for `Array` but also work for any `AbstractArray` and thus it's just a safer way to code. That is why there are specific mentions to not do this in style guides (for example, https://github.com/SciML/SciMLStyle#generic-code-is-preferred-unless-code-is-known-to-be-specific https://github.com/SciML/SciMLStyle#generic-code-is-preferre...), and things like JuliaFormatter automatically flag it as a style break (which would cause CI failures in organizations like SciML which enforce SciML Style formatting as a CI run with Github Actions https://github.com/SciML/ModelingToolkit.jl/blob/v8.14.1/.github/workflows/FormatCheck.yml https://github.com/SciML/ModelingToolkit.jl/blob/v8.14.1/.gi...). There's a call to add linting support for this as well, flagging it any time someone writes this code. If everyone is told to not assume 1-based indexing, formatting CI fails if it is assumed, and the linter underlines every piece of code that does it as red, (along with many other measures, which includes extensive downstream testing, fuzzing against other array types, etc.) then we're at least pretty well guarded against it. And many Julia organizations, like SciML, have these practices in place to guard against it. Yuri's specific discussion is more that JuliaStats does not.