5 ms·

Has anything changed since then? What are y'all's thoughts about correctness in Julia in 2025?

postflopclarity 1y ago
improving, still not perfect. it was true then, and is even more true now, that a large fraction of "correctness bugs" (maybe even the majority) arise from `OffsetArrays.jl`, so a simple solution besides "avoid Julia" is "avoid that package"
- hatmatrix 1y ago
  The greater issue that there is still no way to prevent those types of composability bugs.
  - postflopclarity 1y ago
    sure there are ways. they're just not employed as diligently as they should be. that's more of a social problem than a technical problem.
  - krull10 1y ago
    The simplest approach is to always read the interface of packages one wants to use, and if one isn't provided look at the code / open an issue to interact with the developers about their input assumptions. One should also make tests to ensure the interface behaves in the expected manner when working with your code. Using this approach since 2017 I've never really encountered the types of issues mentioned in Yuri's blog post. The biggest issue I've had is if some user-package makes a change that is effectively breaking but they don't flag it the associated release as breaking. But this isn't really a Julia issue so much as a user-space issue, and can happen in any language when relying on others' libraries.
    - hatmatrix 1y ago
      It doesn't happen so frequently in practice, which is maybe why it's not felt equally by everyone in the community. I've followed some discussions on this topic and the current solution is, as you say, to place the onus entirely on the user. There is another faction that wants to support the user through the IDE to warn them of errors they might be making, and others that want to implement formal specifications that catch the errors during specifications. It's not an easy problem to fix and therefore the "simplest approach" as you describe remains the leading solution.
- missinglugnut 1y ago
  It baffles me that they dug this hole in the first place. I have feelings on the zero-indexing vs one-indexing debate, but at the end of the day you can write correct code in either, as long as you know which one you're using. But Julia fucked it up to where it's not clear what you're using, and library writers don't know which one has been passed! It's insane. They chose style over consistency and correctness and it's caused years of suffering.
  - csvance 1y ago
    Technically you don't need to know what array indexing is being used if you iterate using firstindex(arr):lastindex(arr). AFAIK the issue was that this wasn't consistently done across the Julia ecosystem including parts of the standard library at the time. No clue as to whether this still holds true, but I don't worry about it because I don't use OffsetArrays.
leephillips 1y ago
This is a reasonable article, but way out of date now. Almost all issues raised were solved a while ago.
- hatmatrix 1y ago
  I say this as a huge Julia fan, but the point is not the specific bugs in the article, but the culture of not prioritizing correctness in computation. The initial response by many (not all) in the community was look, those specific bugs are fixed; all languages have bugs; more importantly - look at the benchmark speeds of these computations! Which only reinforced this negative perception. My understanding is that it's a difficult problem to solve, and there are people working on traits/interfaces - but these are still peripheral projects and not part of the core mission to my knowledge. In practice, composability problems arise seldomly, but there is no formal way to guard against it yet. I believe there was some work done at Northeastern U. [1] toward this goal but it's still up to the user to "be careful", essentially. [1] https://repository.library.northeastern.edu/files/neu:4f20cn18h https://repository.library.northeastern.edu/files/neu:4f20cn...
  - munificent 1y ago
    > the culture of not prioritizing correctness in computation. In a language with pervasive use of generic methods, I don't know what actually means. If I write a function like: function add3(x, y, z) x + y + z end Is it correct or not? What does "correct" even mean here? If you call it with values where `+` is defined on them and does what you expect, then my function probably does what you expect too. But if you pass values where `+` does something weird, then `add3()` does something weird too. Is that correct? What expectations should someone have about a function whose behavior is defined in terms of calls to other open-ended generic functions?
    - hatmatrix 1y ago
      I should have been more precise in my language - it's not numerical correctness but composability correctness. They won't appear in a simple example like the one you provided, but more complicated ones are provided in the original post - in the example it partially centers around how getindex should be used with a particular struct and so on.
jakobnissen 1y ago
I've been using Julia for seven years, four years as my main language at work. It's my view that all the major points in the blog post are true, and the problem persists. It's slightly better now, because Julia has more usage in industry and less usage by hobby hackers. I'm convinced it's caused by two factors: The first is the duck-typed, dynamic nature of the language, which, like Python, gives the developer no tools to check or enforce correctness. More fundamentally, the culture of Julia is a cowboy hacking culture where we just start writing, and then we can always kick the code around once bugs appears. There seem to be an almost complete disinterest in careful documentation of behaviour and edge cases, or even actual descriptions of what some abstraction is supposed to do. The natural result is that people interpret all kinds of meaning to any abstraction and use them in slightly different ways. It's madness. As an example, consider [the definition of Base.seek](https://docs.julialang.org/en/v1/base/io-network/#Base.seek https://docs.julialang.org/en/v1/base/io-network/#Base.seek). There is no description of what the position is, what type is can be or operations it's supposed to support. Nor that the seek position is typically zero-indexed. There is no description of what should happen for out of bounds seeks, or how it differs for files open in reading and writing mode. Nor any description of the errors it can throw. I must emphasize that this kind of documentation is the norm, not the exception. This kind of indifference towards actually specifying behaviour is not a foundation you can build a language on. And it's very hard to change in retrospect, because by now, seek means a bunch of different things in Base Julia and the ecosystem, and it would be breaking to change. I've several times seen a core dev change some behaviour of some code because they clearly thought the behaviour was always meant to be X, even though it actually did Y, arguing that Y was an implementation detail. No shit - everything is an implementation detail when nothing is documented. I think Julia needs to grow up and begin taking it's documentation and interfaces seriously.

improving, still not perfect. it was true then, and is even more true now, that a large fraction of "correctness bugs" (maybe even the majority) arise from `OffsetArrays.jl`, so a simple solution besides "avoid Julia" is "avoid that package"

hatmatrix 1y ago
The greater issue that there is still no way to prevent those types of composability bugs.
- postflopclarity 1y ago
  sure there are ways. they're just not employed as diligently as they should be. that's more of a social problem than a technical problem.
- krull10 1y ago
  The simplest approach is to always read the interface of packages one wants to use, and if one isn't provided look at the code / open an issue to interact with the developers about their input assumptions. One should also make tests to ensure the interface behaves in the expected manner when working with your code. Using this approach since 2017 I've never really encountered the types of issues mentioned in Yuri's blog post. The biggest issue I've had is if some user-package makes a change that is effectively breaking but they don't flag it the associated release as breaking. But this isn't really a Julia issue so much as a user-space issue, and can happen in any language when relying on others' libraries.
  - hatmatrix 1y ago
    It doesn't happen so frequently in practice, which is maybe why it's not felt equally by everyone in the community. I've followed some discussions on this topic and the current solution is, as you say, to place the onus entirely on the user. There is another faction that wants to support the user through the IDE to warn them of errors they might be making, and others that want to implement formal specifications that catch the errors during specifications. It's not an easy problem to fix and therefore the "simplest approach" as you describe remains the leading solution.
missinglugnut 1y ago
It baffles me that they dug this hole in the first place. I have feelings on the zero-indexing vs one-indexing debate, but at the end of the day you can write correct code in either, as long as you know which one you're using. But Julia fucked it up to where it's not clear what you're using, and library writers don't know which one has been passed! It's insane. They chose style over consistency and correctness and it's caused years of suffering.
- csvance 1y ago
  Technically you don't need to know what array indexing is being used if you iterate using firstindex(arr):lastindex(arr). AFAIK the issue was that this wasn't consistently done across the Julia ecosystem including parts of the standard library at the time. No clue as to whether this still holds true, but I don't worry about it because I don't use OffsetArrays.

leephillips 1y ago

This is a reasonable article, but way out of date now. Almost all issues raised were solved a while ago.

hatmatrix 1y ago
I say this as a huge Julia fan, but the point is not the specific bugs in the article, but the culture of not prioritizing correctness in computation. The initial response by many (not all) in the community was look, those specific bugs are fixed; all languages have bugs; more importantly - look at the benchmark speeds of these computations! Which only reinforced this negative perception. My understanding is that it's a difficult problem to solve, and there are people working on traits/interfaces - but these are still peripheral projects and not part of the core mission to my knowledge. In practice, composability problems arise seldomly, but there is no formal way to guard against it yet. I believe there was some work done at Northeastern U. [1] toward this goal but it's still up to the user to "be careful", essentially. [1] https://repository.library.northeastern.edu/files/neu:4f20cn18h https://repository.library.northeastern.edu/files/neu:4f20cn...
- munificent 1y ago
  > the culture of not prioritizing correctness in computation. In a language with pervasive use of generic methods, I don't know what actually means. If I write a function like: function add3(x, y, z) x + y + z end Is it correct or not? What does "correct" even mean here? If you call it with values where `+` is defined on them and does what you expect, then my function probably does what you expect too. But if you pass values where `+` does something weird, then `add3()` does something weird too. Is that correct? What expectations should someone have about a function whose behavior is defined in terms of calls to other open-ended generic functions?
  - hatmatrix 1y ago
    I should have been more precise in my language - it's not numerical correctness but composability correctness. They won't appear in a simple example like the one you provided, but more complicated ones are provided in the original post - in the example it partially centers around how getindex should be used with a particular struct and so on.

jakobnissen 1y ago

I've been using Julia for seven years, four years as my main language at work. It's my view that all the major points in the blog post are true, and the problem persists. It's slightly better now, because Julia has more usage in industry and less usage by hobby hackers. I'm convinced it's caused by two factors: The first is the duck-typed, dynamic nature of the language, which, like Python, gives the developer no tools to check or enforce correctness. More fundamentally, the culture of Julia is a cowboy hacking culture where we just start writing, and then we can always kick the code around once bugs appears. There seem to be an almost complete disinterest in careful documentation of behaviour and edge cases, or even actual descriptions of what some abstraction is supposed to do. The natural result is that people interpret all kinds of meaning to any abstraction and use them in slightly different ways. It's madness. As an example, consider [the definition of Base.seek](https://docs.julialang.org/en/v1/base/io-network/#Base.seek https://docs.julialang.org/en/v1/base/io-network/#Base.seek). There is no description of what the position is, what type is can be or operations it's supposed to support. Nor that the seek position is typically zero-indexed. There is no description of what should happen for out of bounds seeks, or how it differs for files open in reading and writing mode. Nor any description of the errors it can throw. I must emphasize that this kind of documentation is the norm, not the exception. This kind of indifference towards actually specifying behaviour is not a foundation you can build a language on. And it's very hard to change in retrospect, because by now, seek means a bunch of different things in Base Julia and the ecosystem, and it would be breaking to change. I've several times seen a core dev change some behaviour of some code because they clearly thought the behaviour was always meant to be X, even though it actually did Y, arguing that Y was an implementation detail. No shit - everything is an implementation detail when nothing is documented. I think Julia needs to grow up and begin taking it's documentation and interfaces seriously.