6 ms·

In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-interpolations-as-input-for-a-neural-ode/51224/6 https://discourse.julialang.

by tagrun 4y ago

In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-interpolations-as-input-for-a-neural-ode/51224/6 https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is citing is actually a problem or not. Item #2, again another Zygote bug. Item #3, which package? That sounds like an hyperbole, an extrapolation from a small sample, and "I'll avoid naming names" is a lazy excuse that would hide this. It is similarly easy to point to poorly written (or poorly documented) JAX code or Python code as well, so that doesn't prove that "Julia is lacklustre and JAX shines". Also, as an academician, I strongly disagree that learning_rate=... is better than η=..., but that's a matter of convention & taste, with no bearing on the correctness or performance of the package/language. It's bikeshedding. I agree that errors are usually not "instructable" in Julia ML packages (which needs to be improved), so monkey typing is less likely to succeed. Item #4 is such a nitpicker. Sure, Julia may not have a special syntax for that particular fringe array slicing like Numpy does and you'll instead need to make a function call, but matrix/tensor code in Numpy is usually filled with calls to zips and cats with explicit indices, whereas in Julia, no explicit indexing needs to be done usually. Also, one can nitpick similarly in the opposite direction: Julia has many language features lacking from JAX or Python, why not talk about those as well? I'm not a huge fan of Julia either (mainly because of it's garbage collected nature), but this is such a low-effort criticism of it.

blindseer 4y ago
I’ve been using Julia since 2017 and still do on a day to day basis, and I agree with the author in a lot of cases, even his subjective naming conventions gripes. The author’s biggest criticism is that Julia doesn’t have tooling to make the developer experience better. There’s Revise, JuliFormatter, LanguageServer and Jet, but the development experience in Python is enviable. There’s like 3 different REPLs, at least two competing linters and auto formatters. It’s okay to admit that these are places Julia is lacking. I think your kind of response to criticism about Julia is what gives the Julia community a bad name, in my opinion. What is wrong with saying these things suck and need improvements? Would you rather Julia not improve and stay the way it is right now forever? Surely I hope not.
- tagrun 4y ago
  What exactly do you think I said about Julia's linters and auto-formatters?
- dman 4y ago
  Just trying to use this thread for a market demand survey - Julia devs, would you pay $9.99 a month for better tooling? [Maybe responses to this will encourage devs to notice that there is a viable market here?]
  - manwe150 4y ago
    That question is rather cryptic. Your profile LinkedIn url also might be broken? (For added mystery maybe)
  - Buttons840 4y ago
    Yes. But not to pay for a software license. I'd pay $10 a month as a donation to a group improving Julia though.
  - goerz 4y ago
    I would, for linting and vim integration comparable to what’s available for Python.
- Tarrosion 4y ago
  There's something a little strange and subjective about saying "a problem with language X is that third party package Y has a bug." Y != X. If Y is some tiny package that hardly anyone uses or cares about, then bugs in Y don't imply much about the typical experience of using X. But if Y is a very common package, practically essential to everyday workflows, then bugs in Y do have implications for the typical X user. The subjectivity arises in deciding whether some third party package Y is totally common, essential, de facto part of the language itself, or esoteric, negligible, who cares. I perceive that a lot of the back and forth around "is Julia good" basically boils down to this. "I don't like Julia! I tried to use it and <package> had an annoying bug!" "Okay, but <package> isn't part of Julia itself or even written/maintained by core Julia contributors, so bugs therein are irrelevant for the awesome multiple dispatch speedy beauty of Julia!" "Hard disagree, <package> is the de facto standard for <common task Julia should be good at> and maintained by <important prominent figure in the Julia community. If even such a high visibility package is broken, the whole ecosystem must be rotten!" The parties are just talking past each other here; there's no ground truth. We'd all be better off if we were more explicit about whether comments applied to (a) the core language itself (b) core language and standard lib (c) the universal experience of working in Julia, i.e. packages that nearly everyone encounters (or lack thereof) (d) packages which are key for certain kind of work but irrelevant for others; auto differentiation would seem to go here (e) niche packages. Disclaimers?: I love Julia-the-language; it is my favorite programming language by a mile. My brain seems to work similarly to Julia-the-language, so programming in Julia feels fluid and effortless to me; no other language sparks joy the same way. But Julia-the-ecosystem certainly has its holes and weak spots. And the limited size of Julia-the-community means even relatively prominent packages can be less maintained than e.g. Python analogues. Example 1: it's unnerving the extent to which the Julia data ecosystem rests on the heroic efforts of a few people [1]. Example 2: even relatively mainstream packages can have issues and PRs unaddressed indefinitely, e.g. I like to plot with Gadfly.jl but am nonetheless frustrated that I've had an open issue there and an upstream PR for over a year. [1] https://github.com/orgs/JuliaData/people https://github.com/orgs/JuliaData/people
nullstyle 4y ago
> but that's a matter of convention & taste, with no bearing on the correctness or performance of the package/language. It's bikeshedding. I heartily disagree with that. Bikeshedding is about focussing on the trivial, and the symbols we choose in our codebases are hardly trivial, and indeed many of us regard naming things as one of the central problems in programming[1]. [1]:https://medium.com/hackernoon/naming-the-things-in-programming-230590016f00 https://medium.com/hackernoon/naming-the-things-in-programmi...
- tagrun 4y ago
  Unlike the example in the link you give, η isn't a generic random name like a,x that can mean anything. If you ever read a paper on stochastic gradient optimization, you'd know that η means learning rate in the context. It is bikeshedding because it is analogous to insisting that using "angle" instead of "θ", or "radius" instead of "r" in a 2D geometry library is superior and takes your code from being a lackluster to something that shines (in the words of the original author), while not having anything useful to say anything about the mathematical/technical aspects of the code itself. Here is the definition of bikeshedding: > The term was coined as a metaphor to illuminate Parkinson’s Law of Triviality. Parkinson observed that a committee whose job is to approve plans for a nuclear power plant may spend the majority of its time on relatively unimportant but easy-to-grasp issues, such as what materials to use for the staff bikeshed, while neglecting the design of the power plant itself, which is far more important but also far more difficult to criticize constructively. It was popularized in the Berkeley Software Distribution community by Poul-Henning Kamp[1] and has spread from there to the software industry at large. from https://en.wiktionary.org/wiki/bikeshedding https://en.wiktionary.org/wiki/bikeshedding
  - 00ajcr 4y ago
    My interpretation of the point in the blog post was that explicitly spelling out variable names makes APIs and the underlying code much more accessible to a wider audience. Sure, there'll be a subset of users of these libraries that have read ML/textbooks and are familiar with what η means in this context. Today, many (most?) users of ML libraries will probably not know what η means without looking it up. Adhering to mathematical notation puts up an unnecessary barrier to using the API/code and ultimately limits wider engagement/collaboration. To attract a bigger slice of the ML community, choosing names that the ML hobbyyist can read, understand and use without pause is the better path forward.
patrick451 4y ago
> In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-inter https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is citing is actually a problem or not. > Item #2, again another Zygote bug. If flux chose a buggy package as a dependency, that's on them, and users are well justified in steering clear of Flux if it's authors are not in habit of auditing the depencies they pull in. As of today, the Project.toml for both Flux and DiffEqFlux still lists Zygote as a dependency. Neither list Enzyme. https://github.com/FluxML/Flux.jl/blob/master/Project.toml https://github.com/FluxML/Flux.jl/blob/master/Project.toml https://github.com/SciML/DiffEqFlux.jl/blob/master/Project.toml https://github.com/SciML/DiffEqFlux.jl/blob/master/Project.t...
- ChrisRackauckas 4y ago
  > If flux chose a buggy package as a dependency, that's on them, and users are well justified in steering clear of Flux if it's authors are not in habit of auditing the dependencies they pull in. As of today, the Project.toml for both Flux and DiffEqFlux still lists Zygote as a dependency. Neither list Enzyme. For DiffEqFlux, it's just for backwards compatibility to Flux. DiffEqFlux is a weird library because it's been "eradicated" over time. There was a point in time where it defined some necessary things to do the things in its documentation. At this point, for most tutorials it's not even required that you have the DiffEqFlux library to do it. That makes it a rather odd library haha. The transition is: - GalacticOptim.jl has become a fully-fledged optimization package formalizing some of the heuristics and multi-package support it was using internally. https://galacticoptim.sciml.ai/dev/ https://galacticoptim.sciml.ai/dev/ - The adjoint overloads are handled all by DiffEqSensitivity.jl at this point. If you try to use them without having that library then you get an error asking you to install it. DiffEqFlux.jl just reexports it. This is why you don't see the Enzyme.jl dependency. - Enzyme.jl has gotten better and better over time, and has become the go-to for DiffEq. In fact, we use a polyalgorithm under the hood that tends to prefer Enzyme.jl and ReverseDiff.jl for the autodiff over Zygote.jl, so it's weird that in 2022 any comparisons still feature Zygote.jl given that, internally, it's almost certainly not using Zygote. Zygote is what the user sees and interacts with, but it's not the core. (Even then, that will be changing soon with the coming Enzyme.jl overloads) - FastChain was a setup to get around issues with Flux, but that has become a full-fledged library of its own, Lux.jl, which isn't completely done yet but will be how those pieces get deleted. So yes, the state of DiffEqFlux.jl is that it has been a fast moving library to the point of its own destruction, basically just holding what would become "hacks" to make the high end work that were then slowly absorbed to become things that "just work" when using Julia. What the library is pivoting towards now is just being a high-level interface for common machine learning use cases of differential equations, like for defining FFJORD or continuous normalizing flows architectures, which you could do from scratch but it's nice to have somewhere that these are all defined. What to do with those tutorials, who knows, move those to DiffEqSensitivity.jl docs, and then we need some kind of inter-module documentation so that way people can easily see the 25+ docs of SciML in one website (or whatever the number is). But honestly, while it becomes a documentation mess to eradicate a higher-level library like this, this has been our dream with Julia over the last few years. Having no place to describe the differentiability of solvers means differentiable programming is truly working. With Enzyme's improvements and such, we're supporting even things like mutation, which is progressively making almost any Julia code you write "just work" with automatic differentiation. No hacks are required to make it work with some underdocumented sublanguage (cough Jax). As everything becomes automated, it becomes harder to document it as a feature because it's instead just a property of being a Julia library.

by tagrun 4y ago

blindseer 4y ago

I’ve been using Julia since 2017 and still do on a day to day basis, and I agree with the author in a lot of cases, even his subjective naming conventions gripes. The author’s biggest criticism is that Julia doesn’t have tooling to make the developer experience better. There’s Revise, JuliFormatter, LanguageServer and Jet, but the development experience in Python is enviable. There’s like 3 different REPLs, at least two competing linters and auto formatters. It’s okay to admit that these are places Julia is lacking. I think your kind of response to criticism about Julia is what gives the Julia community a bad name, in my opinion. What is wrong with saying these things suck and need improvements? Would you rather Julia not improve and stay the way it is right now forever? Surely I hope not.

tagrun 4y ago
What exactly do you think I said about Julia's linters and auto-formatters?
dman 4y ago
Just trying to use this thread for a market demand survey - Julia devs, would you pay $9.99 a month for better tooling? [Maybe responses to this will encourage devs to notice that there is a viable market here?]
- manwe150 4y ago
  That question is rather cryptic. Your profile LinkedIn url also might be broken? (For added mystery maybe)
- Buttons840 4y ago
  Yes. But not to pay for a software license. I'd pay $10 a month as a donation to a group improving Julia though.
- goerz 4y ago
  I would, for linting and vim integration comparable to what’s available for Python.
Tarrosion 4y ago
There's something a little strange and subjective about saying "a problem with language X is that third party package Y has a bug." Y != X. If Y is some tiny package that hardly anyone uses or cares about, then bugs in Y don't imply much about the typical experience of using X. But if Y is a very common package, practically essential to everyday workflows, then bugs in Y do have implications for the typical X user. The subjectivity arises in deciding whether some third party package Y is totally common, essential, de facto part of the language itself, or esoteric, negligible, who cares. I perceive that a lot of the back and forth around "is Julia good" basically boils down to this. "I don't like Julia! I tried to use it and <package> had an annoying bug!" "Okay, but <package> isn't part of Julia itself or even written/maintained by core Julia contributors, so bugs therein are irrelevant for the awesome multiple dispatch speedy beauty of Julia!" "Hard disagree, <package> is the de facto standard for <common task Julia should be good at> and maintained by <important prominent figure in the Julia community. If even such a high visibility package is broken, the whole ecosystem must be rotten!" The parties are just talking past each other here; there's no ground truth. We'd all be better off if we were more explicit about whether comments applied to (a) the core language itself (b) core language and standard lib (c) the universal experience of working in Julia, i.e. packages that nearly everyone encounters (or lack thereof) (d) packages which are key for certain kind of work but irrelevant for others; auto differentiation would seem to go here (e) niche packages. Disclaimers?: I love Julia-the-language; it is my favorite programming language by a mile. My brain seems to work similarly to Julia-the-language, so programming in Julia feels fluid and effortless to me; no other language sparks joy the same way. But Julia-the-ecosystem certainly has its holes and weak spots. And the limited size of Julia-the-community means even relatively prominent packages can be less maintained than e.g. Python analogues. Example 1: it's unnerving the extent to which the Julia data ecosystem rests on the heroic efforts of a few people [1]. Example 2: even relatively mainstream packages can have issues and PRs unaddressed indefinitely, e.g. I like to plot with Gadfly.jl but am nonetheless frustrated that I've had an open issue there and an upstream PR for over a year. [1] https://github.com/orgs/JuliaData/people https://github.com/orgs/JuliaData/people

nullstyle 4y ago

> but that's a matter of convention & taste, with no bearing on the correctness or performance of the package/language. It's bikeshedding. I heartily disagree with that. Bikeshedding is about focussing on the trivial, and the symbols we choose in our codebases are hardly trivial, and indeed many of us regard naming things as one of the central problems in programming[1]. [1]:https://medium.com/hackernoon/naming-the-things-in-programming-230590016f00 https://medium.com/hackernoon/naming-the-things-in-programmi...

tagrun 4y ago
Unlike the example in the link you give, η isn't a generic random name like a,x that can mean anything. If you ever read a paper on stochastic gradient optimization, you'd know that η means learning rate in the context. It is bikeshedding because it is analogous to insisting that using "angle" instead of "θ", or "radius" instead of "r" in a 2D geometry library is superior and takes your code from being a lackluster to something that shines (in the words of the original author), while not having anything useful to say anything about the mathematical/technical aspects of the code itself. Here is the definition of bikeshedding: > The term was coined as a metaphor to illuminate Parkinson’s Law of Triviality. Parkinson observed that a committee whose job is to approve plans for a nuclear power plant may spend the majority of its time on relatively unimportant but easy-to-grasp issues, such as what materials to use for the staff bikeshed, while neglecting the design of the power plant itself, which is far more important but also far more difficult to criticize constructively. It was popularized in the Berkeley Software Distribution community by Poul-Henning Kamp[1] and has spread from there to the software industry at large. from https://en.wiktionary.org/wiki/bikeshedding https://en.wiktionary.org/wiki/bikeshedding
- 00ajcr 4y ago
  My interpretation of the point in the blog post was that explicitly spelling out variable names makes APIs and the underlying code much more accessible to a wider audience. Sure, there'll be a subset of users of these libraries that have read ML/textbooks and are familiar with what η means in this context. Today, many (most?) users of ML libraries will probably not know what η means without looking it up. Adhering to mathematical notation puts up an unnecessary barrier to using the API/code and ultimately limits wider engagement/collaboration. To attract a bigger slice of the ML community, choosing names that the ML hobbyyist can read, understand and use without pause is the better path forward.

patrick451 4y ago

> In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-inter https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is citing is actually a problem or not. > Item #2, again another Zygote bug. If flux chose a buggy package as a dependency, that's on them, and users are well justified in steering clear of Flux if it's authors are not in habit of auditing the depencies they pull in. As of today, the Project.toml for both Flux and DiffEqFlux still lists Zygote as a dependency. Neither list Enzyme. https://github.com/FluxML/Flux.jl/blob/master/Project.toml https://github.com/FluxML/Flux.jl/blob/master/Project.toml https://github.com/SciML/DiffEqFlux.jl/blob/master/Project.toml https://github.com/SciML/DiffEqFlux.jl/blob/master/Project.t...

ChrisRackauckas 4y ago
> If flux chose a buggy package as a dependency, that's on them, and users are well justified in steering clear of Flux if it's authors are not in habit of auditing the dependencies they pull in. As of today, the Project.toml for both Flux and DiffEqFlux still lists Zygote as a dependency. Neither list Enzyme. For DiffEqFlux, it's just for backwards compatibility to Flux. DiffEqFlux is a weird library because it's been "eradicated" over time. There was a point in time where it defined some necessary things to do the things in its documentation. At this point, for most tutorials it's not even required that you have the DiffEqFlux library to do it. That makes it a rather odd library haha. The transition is: - GalacticOptim.jl has become a fully-fledged optimization package formalizing some of the heuristics and multi-package support it was using internally. https://galacticoptim.sciml.ai/dev/ https://galacticoptim.sciml.ai/dev/ - The adjoint overloads are handled all by DiffEqSensitivity.jl at this point. If you try to use them without having that library then you get an error asking you to install it. DiffEqFlux.jl just reexports it. This is why you don't see the Enzyme.jl dependency. - Enzyme.jl has gotten better and better over time, and has become the go-to for DiffEq. In fact, we use a polyalgorithm under the hood that tends to prefer Enzyme.jl and ReverseDiff.jl for the autodiff over Zygote.jl, so it's weird that in 2022 any comparisons still feature Zygote.jl given that, internally, it's almost certainly not using Zygote. Zygote is what the user sees and interacts with, but it's not the core. (Even then, that will be changing soon with the coming Enzyme.jl overloads) - FastChain was a setup to get around issues with Flux, but that has become a full-fledged library of its own, Lux.jl, which isn't completely done yet but will be how those pieces get deleted. So yes, the state of DiffEqFlux.jl is that it has been a fast moving library to the point of its own destruction, basically just holding what would become "hacks" to make the high end work that were then slowly absorbed to become things that "just work" when using Julia. What the library is pivoting towards now is just being a high-level interface for common machine learning use cases of differential equations, like for defining FFJORD or continuous normalizing flows architectures, which you could do from scratch but it's nice to have somewhere that these are all defined. What to do with those tutorials, who knows, move those to DiffEqSensitivity.jl docs, and then we need some kind of inter-module documentation so that way people can easily see the 25+ docs of SciML in one website (or whatever the number is). But honestly, while it becomes a documentation mess to eradicate a higher-level library like this, this has been our dream with Julia over the last few years. Having no place to describe the differentiability of solvers means differentiable programming is truly working. With Enzyme's improvements and such, we're supporting even things like mutation, which is progressively making almost any Julia code you write "just work" with automatic differentiation. No hacks are required to make it work with some underdocumented sublanguage (cough Jax). As everything becomes automated, it becomes harder to document it as a feature because it's instead just a property of being a Julia library.