3 ms·

> If you can't trust gradients or probability distributions / sampling to be correct, that's a pretty damning verdict for a language aimed at numerical calculat

> If you can't trust gradients or probability distributions / sampling to be correct, that's a pretty damning verdict for a language aimed at numerical calculations. If you draw the distinction between Julia the language and Julia the ecosystem, then that's not a damning verdict for the language, even if we accept the premise that you can't trust the computations you mentioned. The way that Julia composes is unprecedented. Typical examples include computing with measurement uncertainty, or big floats through functions that were not explicitly designed to support those types. This is great, but the correctness of this composition cannot be assumed. One good proxy for how this composition occurs is in many type combinations. As Here is a concrete, made-up example. There is a function you've been using without any problem that takes an argument of type `DenseArray{Float64, 2}`. (Meaning, the function gets called with an argument of that type. The function definition is not constrained to work on that type.) If you've never focused on the correctness (e.g. reviewing the code, unit tests) of this function to see what it does on `SparseArray{BigFloat, 2}`, I would not take it for granted that it works. This is perhaps too big a burden on a user. Some might say they rather not have the compositionality unless it's guaranteed to be correct. But, if you're actually asking me to imagine using Julia in a situation where I really want to be sure about the correctness of a Julia program, I would at the very attempt to characterize all the expected types to go into any function. If at runtime a function is called with a new type, I would log that, and possibly error out. Note this is related to, but not equivalent to what's necessary to do "static" compilation in Julia.

ModernMech 4y ago
> If you draw the distinction between Julia the language and Julia the ecosystem Language and ecosystem are so intertwined that I almost never do this. Maybe when there are multiple well developed ecosystems, but when there’s only one, then it and the language are inseparable to me (for any language, not just Julia).
- gugagore 4y ago
  I wonder if you feel that way as equally about C and Lisp as you do about Julia.
  - t_mann 4y ago
    I remember Julia being pitched as a competitor especially for Matlab, but also R,... much more so do than for C or Lisp. So it seems like a fair comparison, and those are definitely to some extent seen as identical with their package ecosystem (in Matlab, most packages actually come from the same vendor, and base R utility is a tiny fraction of the utility of its packages).
t_mann 4y ago
Thanks for the explanation. Yeah, that is an interesting feature. Sounds like you'll have to do a lot of reading documentations and testing other packages yourself then. A typing system that allows defining accepted, rejected and 'other' data types for functions (and a compiler flag for either warning about the latter or not) would be interesting. But the original critique also mentioned several correctness issues in the core language itself, so the issue does seem to go deeper.
- gugagore 4y ago
  There are several correctness issues lurking about in virtually all software. That part of the critique is uninteresting to me. If I want to pull up a blog post about several correctness issues in any programming language, I bet I could. Also depends what you count as correct, see undefined behavior.
  - t_mann 4y ago
    > Also depends what you count as correct I feel that should be the job of the authors to think about what correctness means for their application, and under what circumstances they can give a correct answer. Ideally, also warn the user when the inputs don't fall within that domain or give them options to handle such cases. Let's talk about an example: say you have a neat algorithm for fast multiplication of sparse matrices, approximately correct up to a defined Frobenius error. Say we have some reasonable treatment of dimension mismatch and NaN, inf,... with option flags already. Now if I wanted to know whether your algorithm has the same numerical guarantee for a dense matrix, I'd have to read your paper, run some tests, ask around, and I might never be quite sure. It seems much more sane to put the onus on the authors to be explicit about the behaviour in that case. Your and others' comments seem to put the onus mostly on the user for thinking about the behaviour of others' code. That does not seem like a good practice, especially in the context of complex math and nested composition.
- oivey 4y ago
  You could do something like this: plus_one(x::Number) = x + 1 function plus_one(x) @warn "untested type usage of plus_one" x + 1 end That's cumbersome, but a macro could make it less annoying.
martinhath 4y ago
> This is perhaps too big a burden on a user. Some might say they rather not have the compositionality unless it's guaranteed to be correct. I think this is it. Or rather, I think to some people, myself included, compositionality should imply correctness. If you /can/ compose things, but the result is wrong, can you really say that you've got composition? How useful it is really to be able to plug in your own types in other peoples libraries if you have to trace through the execution of that library, figure out which libraries they transitively use, to ensure that all of your instantiations are sound? How do you even test this properly? It's a really hard problem, and from what I can tell, Julia gives you no tools to deal with this. [0] is probably relevant here, although I'm not sure I share the positive outlook. [0]: http://www.jerf.org/iri/post/2954 http://www.jerf.org/iri/post/2954
- tgflynn 4y ago
  I don't have experience with using Julia in really large code bases but my intuition has always been that the combination of these design characteristics in one language: - easy unrestricted composability - lack of well-defined interfaces - lack of an effective way to use the strong typing system to validate correctness is not a very good idea.
  - gugagore 4y ago
    I think Julia (mostly meaning pervasive multiple dispatch, which is unprecedented at that scale Julia has it) unlocks a new way to organize programs. Some formalized notion of "interface" is certainly important, but it seems no clear formulation has emerged. I think it's fairly consistent with the dynamic nature of Julia and fairly inconsistent with the static nature of Julia. I don't know what a good solution to the interface problem is. I think one can get pretty far with writing tests to check interfaces. If you are a library that expects user-defined types, you can expose an interface to these tests so that a user can check if they've implemented everything. This is a very generic approach, and aside from the key limitation of only giving results at runtime, is much more powerful.
    - tgflynn 4y ago

ModernMech 4y ago

> If you draw the distinction between Julia the language and Julia the ecosystem Language and ecosystem are so intertwined that I almost never do this. Maybe when there are multiple well developed ecosystems, but when there’s only one, then it and the language are inseparable to me (for any language, not just Julia).

gugagore 4y ago
I wonder if you feel that way as equally about C and Lisp as you do about Julia.
- t_mann 4y ago
  I remember Julia being pitched as a competitor especially for Matlab, but also R,... much more so do than for C or Lisp. So it seems like a fair comparison, and those are definitely to some extent seen as identical with their package ecosystem (in Matlab, most packages actually come from the same vendor, and base R utility is a tiny fraction of the utility of its packages).

t_mann 4y ago

Thanks for the explanation. Yeah, that is an interesting feature. Sounds like you'll have to do a lot of reading documentations and testing other packages yourself then. A typing system that allows defining accepted, rejected and 'other' data types for functions (and a compiler flag for either warning about the latter or not) would be interesting. But the original critique also mentioned several correctness issues in the core language itself, so the issue does seem to go deeper.

gugagore 4y ago
There are several correctness issues lurking about in virtually all software. That part of the critique is uninteresting to me. If I want to pull up a blog post about several correctness issues in any programming language, I bet I could. Also depends what you count as correct, see undefined behavior.
- t_mann 4y ago
  > Also depends what you count as correct I feel that should be the job of the authors to think about what correctness means for their application, and under what circumstances they can give a correct answer. Ideally, also warn the user when the inputs don't fall within that domain or give them options to handle such cases. Let's talk about an example: say you have a neat algorithm for fast multiplication of sparse matrices, approximately correct up to a defined Frobenius error. Say we have some reasonable treatment of dimension mismatch and NaN, inf,... with option flags already. Now if I wanted to know whether your algorithm has the same numerical guarantee for a dense matrix, I'd have to read your paper, run some tests, ask around, and I might never be quite sure. It seems much more sane to put the onus on the authors to be explicit about the behaviour in that case. Your and others' comments seem to put the onus mostly on the user for thinking about the behaviour of others' code. That does not seem like a good practice, especially in the context of complex math and nested composition.
oivey 4y ago
You could do something like this: plus_one(x::Number) = x + 1 function plus_one(x) @warn "untested type usage of plus_one" x + 1 end That's cumbersome, but a macro could make it less annoying.

martinhath 4y ago

> This is perhaps too big a burden on a user. Some might say they rather not have the compositionality unless it's guaranteed to be correct. I think this is it. Or rather, I think to some people, myself included, compositionality should imply correctness. If you /can/ compose things, but the result is wrong, can you really say that you've got composition? How useful it is really to be able to plug in your own types in other peoples libraries if you have to trace through the execution of that library, figure out which libraries they transitively use, to ensure that all of your instantiations are sound? How do you even test this properly? It's a really hard problem, and from what I can tell, Julia gives you no tools to deal with this. [0] is probably relevant here, although I'm not sure I share the positive outlook. [0]: http://www.jerf.org/iri/post/2954 http://www.jerf.org/iri/post/2954

tgflynn 4y ago
I don't have experience with using Julia in really large code bases but my intuition has always been that the combination of these design characteristics in one language: - easy unrestricted composability - lack of well-defined interfaces - lack of an effective way to use the strong typing system to validate correctness is not a very good idea.
- gugagore 4y ago
  I think Julia (mostly meaning pervasive multiple dispatch, which is unprecedented at that scale Julia has it) unlocks a new way to organize programs. Some formalized notion of "interface" is certainly important, but it seems no clear formulation has emerged. I think it's fairly consistent with the dynamic nature of Julia and fairly inconsistent with the static nature of Julia. I don't know what a good solution to the interface problem is. I think one can get pretty far with writing tests to check interfaces. If you are a library that expects user-defined types, you can expose an interface to these tests so that a user can check if they've implemented everything. This is a very generic approach, and aside from the key limitation of only giving results at runtime, is much more powerful.
  - tgflynn 4y ago