10 ms·

Jax vs. Julia (Vs PyTorch)

141 points by sebg 5y ago

adgjlsfhk1 5y ago
""" Even in the major well-known well-respected Julia packages – I’ll avoid naming names – the source code has very obvious cases of unused local variables, dead code branches that can never be reached, etc. """ Please name names. that way we can fix stuff. Other than that, great post!
- lern_too_spel 5y ago
  I think the author is pushing for better code quality tooling in Julia instead of having people manually fix these problems.
  - adgjlsfhk1 5y ago
    we absolutely should, but in the short term, fixing issues is good.
machinekob 5y ago
I strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means so its easier to read some omega then for example learning_rate or lr. But for us mortal who cross multiple domains its just getting extremely frustrating to read full math based notation without any extra info about notation in package/functions etc. so debugging multiple sub-packages is just getting too time consuming as you have to learn both person style of writing code, whole scientific notation and get domain knowledge before you can even touch the code.
- bobbylarrybobby 5y ago
  And if you ever do want to edit the code, you have to know the name of every non-ASCII symbol the codebase uses if you want to type out those same symbols without copying and pasting them. If you're not familiar with the material, entering a character like ξ can be a real challenge, and is actually more keystrokes than just typing "xi".
  - tagrun 5y ago
    Difference is 1 keystroke: you type \xi. Why is that a "real challenge"?
    - animal_spirits 5y ago
      The challenge comes from a person like me, who doesn't know off the top of my head which Greek letter ξ is. So for each of these symbols I'd have to google them and learn it, or have some notepad where I can copy paste the needed symbol
      - tagrun 5y ago
        So you don't know the Greek alphabet, but write high-performance computing code involving non-trivial math? (Julia's main use case is HPC)
        belval 5y ago
        The author example is using the greek letter "eta" instead of spelling out "learning_rate", this is a pretty damning example.
        ShamelessC 5y ago
        What? Did you not read the parent of this thread? > strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means In any case, I program HPC stuff myself with pytorch and no - I don't know the Greek alphabet and probably don't understand "non trivial math". The assumption that these people can't contribute is pretty off-putting honestly. More engineers would join such efforts if there wasn't so much gatekeeping.
        tagrun 5y ago
        So you think there's a cabal of "gatekeeper" scientists who are trying to hold the hordes of capable (yet somehow unfamiliar with the corresponding mathematical literature) engineers from making useful contributions to their libraries, because they use the conventional notation from the scientific literature? They should instead tread carefully and dispose of all Greek letters in their code to ensure you are not put off, because that's the real reason that stops you from making useful contributions?
- aaplok 5y ago
  > Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters Naming things by their English name is not more universal than using Greek letters. It's just serving amother group of people who live in a different bubble.
  - belval 5y ago
    Yes and no, the example that the author gives is actually a very good one: > Many Julia APIs look like Optimiser(η=...) rather than Optimiser(learning_rate=...). This is a pretty unreadable convention. The learning rate is a well known name that basically every one will understand, on the other hand, "η" or eta, is not even used everywhere in the literature with some papers using alpha instead. This just looks clever, it's a pretty bad parameter name.
    - melissalobos 5y ago
      > The learning rate is a well known name that basically every one will understand Absolutely! Because as we all know, everyone speaks English. The GP's point was that greek letters are used in lots and lots of papers even written in other languages. I have read quite a few papers in Japanese that used exactly the same conventions with respect to the greek letters and latin letters used.
      - belval 5y ago
        How many researchers in the ML/DL community don't speak English? I don't have hard numbers but I highly doubt that it's a significant proportion. What is the reach of your Japanese papers when almost no-one outside of Japan can read Japanese? Even China, despite their best effort to de-westernize their culture still uses English in their research papers. And if all the above wasn't enough, Julia's libraries are still all in English so if an hypothetical researcher's English is so poor that they don't know what "learning rate" is, I'd venture that they'll have trouble programming in Julia/JAX/PyTorch.
        5y ago
- agumonkey 5y ago
  I understand both camps but I believe, these are superficial problems. It's like worrying about the comfort of seat in the operating room of a nuclear plant.
  - nextos 5y ago
    Exactly. I actually find Julia's ecosystem (not the language) way more approachable than Python's. In Python, most libraries are big monoliths. Whereas in Julia, libraries are small and composable. Furthermore, it's the same language all the way down. Python's libraries are superb, but the learning curve to develop (not to use them) is really steep.
    - jstx1 5y ago
      I don't understand. What do you mean by "learning curve to develop" an existing Python library?
      - borodi 5y ago
        I imagine it has to do with the fact that lots of python libraries are mostly c/c++ or fortran, while Julia packages are usually just Julia.
      - adgjlsfhk1 5y ago
        Python's story for package development is a mess compared to Julia. If you know what you're doing you can make a new Julia package in the general registry with unit tests, CI, and code coverage in about 30 minutes. To contribute to an existing repository, you can ]dev the package to download the git repository, @edit a function to open the source in an editor, and push the changes to a PR in about 10 minutes. This makes it much easier for package users to turn into package developers. I've contributed to a bunch of Julia libraries that if I still used python, I wouldn't have contributed to the equivalent python library because contributing to Julia packages is at least an order of magnitude easier.
  - abakus 5y ago
    superficial you say. How about I name these in chinese in my package?
SemanticStrengh 5y ago
The most next gen autodiff library probably is https://github.com/breandan/kotlingrad https://github.com/breandan/kotlingrad because of its features, ergonomy and type safety
- martinsmit 5y ago
  Idk, Enzyme is pretty next gen, all the way down to LLVM code. https://github.com/EnzymeAD/Enzyme https://github.com/EnzymeAD/Enzyme
  - mhh__ 5y ago
    So enzyme is pretty cool but IMO the documentation is terrible and the API is absolutely insane.
    - wmoses 5y ago
      Enzyme dev here. The documentation is not just bad, it is in fact actively missing for several things (like forward mode AD, split mode, recent vector). Having recently finished up a number of big technical challenges (forward, vector, split, BLAS), we're hoping to have a big documentation-athon at the end of summer (all are welcome)!
      - SemanticStrengh 5y ago
        On HN I am often seeing libraries/blog authors react very quickly to references of their projects. I'm curious, how do you achieve this, is it a coincidence and where you reading HN comments, or do you have a crwaler/API than send you a notification when your project github link is found in a new HN comment ?
        wmoses 5y ago
        I was actually socializing with folks at the Cambridge Area Julia meetup where I gave a talk on Enzyme’s Julia bindings no less when someone shared this. So in this case pure coincidence.
tagrun 5y ago
In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-interpolations-as-input-for-a-neural-ode/51224/6 https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is citing is actually a problem or not. Item #2, again another Zygote bug. Item #3, which package? That sounds like an hyperbole, an extrapolation from a small sample, and "I'll avoid naming names" is a lazy excuse that would hide this. It is similarly easy to point to poorly written (or poorly documented) JAX code or Python code as well, so that doesn't prove that "Julia is lacklustre and JAX shines". Also, as an academician, I strongly disagree that learning_rate=... is better than η=..., but that's a matter of convention & taste, with no bearing on the correctness or performance of the package/language. It's bikeshedding. I agree that errors are usually not "instructable" in Julia ML packages (which needs to be improved), so monkey typing is less likely to succeed. Item #4 is such a nitpicker. Sure, Julia may not have a special syntax for that particular fringe array slicing like Numpy does and you'll instead need to make a function call, but matrix/tensor code in Numpy is usually filled with calls to zips and cats with explicit indices, whereas in Julia, no explicit indexing needs to be done usually. Also, one can nitpick similarly in the opposite direction: Julia has many language features lacking from JAX or Python, why not talk about those as well? I'm not a huge fan of Julia either (mainly because of it's garbage collected nature), but this is such a low-effort criticism of it.
- blindseer 5y ago
  I’ve been using Julia since 2017 and still do on a day to day basis, and I agree with the author in a lot of cases, even his subjective naming conventions gripes. The author’s biggest criticism is that Julia doesn’t have tooling to make the developer experience better. There’s Revise, JuliFormatter, LanguageServer and Jet, but the development experience in Python is enviable. There’s like 3 different REPLs, at least two competing linters and auto formatters. It’s okay to admit that these are places Julia is lacking. I think your kind of response to criticism about Julia is what gives the Julia community a bad name, in my opinion. What is wrong with saying these things suck and need improvements? Would you rather Julia not improve and stay the way it is right now forever? Surely I hope not.
longemen3000 5y ago
I feel called out on the academic part hahaah. I simply want to code state of the art (thermodynamic) models, and at least julia helps by providing easy testing and publishing infraestructure. but obviously we can't compete with a corporation in code quality (we are trying!) Unrelated, but for small sizes, i really prefer to use forward mode in julia (Via ForwardDiff.jl) instead of Zygote. the overhead of reverse ADing over an arbitrary function with mutation is not worth it.
- tagrun 5y ago
  In the context of neural networks with differential equations (which appears to be the original poster's field), the trade-off depends: https://diffeqflux.sciml.ai/dev/ControllingAdjoints/ https://diffeqflux.sciml.ai/dev/ControllingAdjoints/
  - longemen3000 5y ago
    yeah, my systems are really small in comparison (1-20) but with higher order derivatives (up to 4th order), so reverse AD is not the best in that regard
blindseer 5y ago
Another massive frustration for me is that Julia has no formal way to say "here are public functions and these are private" but does have a completely orthogonal way of saying "these are functions that will populate your global namespace if you use `using`", i.e. `export variable_name`, and people absolutely confuse the hell out of these. I don't think there's even agreement in the Julia community if you should use `export` for your public API or not. And if you misspell the exports or change the variable in question, Julia won't even warn you about it. That is straight crazy behavior to me, and I still don't understand how that hasn't been changed. The `using` + `import` packages in Julia combined with how `export`s work make it SUCH a confusing and frustrating experience for beginners in Julia. I personally like mathematical symbols when I'm writing and reading code in my domain, but I do feel very lost when I'm reading Julia code outside of my area of expertise. All my colleagues hate it too (hard to grep, hard to type if you don't know the math and are just a software engineer) and I'm coming around to the idea of not using it or documenting it explicitly. The fact that mutable structs are easier to use but immutable structs are more performant, the lack of composition of fields the way Go handles it, the lack of traits or interfaces, the sorry state of compilation time, the non existent tooling all lead a beginner / intermediate Julia developer in the wrong direction in my opinion. It's very easy to write code that is straight up broken or just not efficient in Julia, and that's probably why I won't pick it for a big project going forward. But I'm still keeping my eye on it. Maybe in 5 years it'll be the language for a lot of the jobs?
- mountainriver 5y ago
  Yeah I really want to like Julia but for all the reasons you mentioned I’ll just use python. Also python is making a ton of improvements and looks better by the day
- socialdemocrat 5y ago
  I use export for my public functions. What is wrong with that? How exactly does Python signal what is public? Nothing but conventions like underscore, which btw Julia developers use as well. If I really really don't want somebody to use a function, I might underscore it but generally I don't see the point.
xico 5y ago
Tangentially related but there is an effort to get some of the features of JAX into PyTorch: https://pytorch.org/functorch/ https://pytorch.org/functorch/
patrick451 5y ago
> It’s pretty common to see posts on the Julia Discourse saying “XYZ library doesn’t work”, followed by a reply from one of the library maintainers stating something like “This is an upstream bug in the new version a.b.c of the ABC library, which XYZ depends upon. We’ll get a fix pushed ASAP.” And > First of all, it’s not in the Flux documentation. Instead it’s in the documentation for a separate component library – Zygote. So you have to check both, and you have to know to check both. People love to rave about how the julia ecosystem is made up of tons of tiny composable libraries instead of large monolithic libraries like scipy. To me, that sounds like a major headache, and the quotes above are partly why. I'd rather read docs for one library than two. The chances that the maintainers of a large monolithic library break internal compatibility is way smaller than the chance that two disconnected projects become incompatible.
- rg111 5y ago
  > People love to rave about how the julia ecosystem is made up of tons of tiny composable libraries instead of large monolithic libraries like scipy. To me, that sounds like a major headache Absolutely agree. First and foremost, I need to get my work done. I do not want to learn all those libraries. The API design, its quality, philosophy (or lack thereof) of framework will be different across all the frameworks. And I am not sure I want too many libraries in my codebase. It is a whole different level of technical debt.
alphachloride 5y ago
One of the biggest hurdles to my use of Jax is that it doesn't work on Windows, my daily driver. It's inconvenient when playing around with a new tool that requires a whole other OS to run. I'm hoping that gets figured out soon.
- make3 5y ago
  does it not work on WSL ?
  - rsfern 5y ago
    I think WSL is the official workaround There is also https://github.com/cloudhan/jax-windows-builder https://github.com/cloudhan/jax-windows-builder
socialdemocrat 5y ago
Ouch! As a big Julia fan I cannot exactly complain about these criticism. They seem overall very fair. Although some things like tracking down problems I think is affected a lot by what you are used to. I am way slower figuring out problems in Python than in Julia. But the heavily academic code and often poor user documentation for third party packages is a very real problem. I would like to point out that I think the Julia core libraries are better documented than Python. But third party stuff often has a long way to go. I also think Julia may suffer a bit from people writing a lot of CLEVER code. I am computer science guy who feel lost in the midst of all these brainy Julia PhDs, postdocs and researchers. I think maybe many of us more regular programmers have a bit healthier habits in writing code a little bit less clever and and perhaps more verbose. It is not all bad though. I think that the APIs to a lot of Julia libraries are quite well thought out. It is more the inside of a lot of these packages which can look a bit messy. Perhaps Julia code quality and documentation can improve when the language starts expanding more outside academia and more among general programmers, industry etc.
ChrisRackauckas 5y ago
The main idea (both pro and con) with Jax is its "automatic performance" model. Essentially, Jax uses a tracer down to XLA which then does some linear algebra optimizations to generate code. For the DSL case where Jax is generally used, this tends to be "good enough" for a good number of use cases. The issue is that when it's not "good enough", it doesn't give package developers much recourse to really dig into the details and optimize for the things which the compiler (and compilers in general) is unable to do. For example, we released the library SimpleChains.jl that was optimized for a lot of cases for small neural networks (https://julialang.org/blog/2022/04/simple-chains/ https://julialang.org/blog/2022/04/simple-chains/). There was discussion about Jax, and we saw that for any case where individuals had a modern multi-core CPU threaded machine setup, SimpleChains.jl had about a 10x performance improvement on CPU over the Jax equinox library. This shouldn't be surprising to anyone because the details of how that was achieved was quite clear (memory handling, manual SIMD bits, etc.), though those of course are not the kinds of things you get "automatically" with either language. The main difference of having a full programming language instead of a DSL is that such code can be written to manually optimize what the compilers are not doing. And this is the pattern that tends to repeat. For small static-like differential equation solves, Jax is fine, but for example the PDE code optimizations done in this tutorial (https://diffeq.sciml.ai/dev/tutorials/faster_ode_example/#Example-Accelerating-Linear-Algebra-PDE-Semi-Discretization https://diffeq.sciml.ai/dev/tutorials/faster_ode_example/#Ex...) are completely incompatible with the Jax programming model and accelerate the PDE solve by more than 10x. And if you look into detail at things showing Jax is fine for PDEs, they are using the slower code style and comparing to that slower Julia code, not the faster iteration form. Of course, these are optimizations most people don't know to do so they are the kind of extra order of magnitude you see in Julia packages over Jax, but not as much in user code. Another repeat of this pattern is the sub-optimality of vmap. It doesn't run code completely independently so it's not optimal in CPU cases where you'd want to effectively use multithreading or MPI, but it's also not generating kernels so it's not optimal in the case of GPUs (for example, we have a new GPU-based ODE solver coming out soon that outperforms the older Julia form and the Jax form by like 100x. Again, no surprises here because special-built GPU kernels have already demonstrated this as possible from C++ directly in this context). So vmap will always get you parallelism, but when you build a library you have to leave it behind at some point if you really want to keep optimizing. I think what it does is admirable. If you can get a code to compile with Jax (which is a big if, it does require pure functions and a lot of extra things Python code normally doesn't satisfy) then it does get you something relatively performant. But there's a reason why what's mentioned here is Flux and Zygote, not SimpleChains and Enzyme, the real tools the Julia community is pushing towards with the full mutation support and the full speed behind it. Jax can compare in the former case, but not the latter case.
dklend122 5y ago
There are several repeating (valid) critiques. There is also a sense that things won't get better. I don't think that's necessarily the case. I can't give timelines, but I'll list the themes and point to the very active work going on in those areas: 1. Static analysis/ it's hard to write large codebases of correct Julia code. That's being addressed on several fronts: a. Work on both traits/interfaces and sound static typing is progressing in Jan Vitek's group at Northwestern b. User extensible type lattice for correctness proof of programs. See here: https://news.ycombinator.com/item?id=26136212 https://news.ycombinator.com/item?id=26136212 c. Jet.jl already catches method errors at JIT time. It's under-used IMO. 2. Compilation time and package dev latency. a. We're getting close to native code caching, and more: https://discourse.julialang.org/t/precompile-why/78770/8 https://discourse.julialang.org/t/precompile-why/78770/8 As you'll also read, the difficulty is due to important tradeoffs Julia made with composability and aggressive specialization...but it's not fundamental and can be surmounted. Yes there's been some pain, but in the end hopefully we'll have something approximating the best of both worlds. b. Runtime free static compilation: https://www.youtube.com/watch?v=YsNC4oO0rLA https://www.youtube.com/watch?v=YsNC4oO0rLA c. Semantics for separate compilation: https://gist.github.com/JeffBezanson/dd86043ef867954bd7e2163ab66f8b4e https://gist.github.com/JeffBezanson/dd86043ef867954bd7e2163... Some combination of the above should also address deployment scenarios, whether that be CLI, mobile, browser or even embedded (Yes, with an (expanding) subset of code, StaticCompiler can produce binaries down to the 10s of Kbs with real working Julia programs that use non trivial abstractions like subtyping and higher order functions) 3. Zygote is rough a. Enzyme will do (almost) whole language AD, including mutation. b. Diffractor.jl
- catelm 5y ago
  Do you have a link to some information regarding the work on traits/interfaces and static typing, please?
g42gregory 5y ago
>> Where does JAX shine vs. Julia? ...Documentation...Code Quality... I am not sure I need to go beyond that. The decision is somewhat easy here. I am surprised that integration is not mentioned. JAX is just Python (library) so it could be easily integrated into the rest of the system. It fits very well into Python/C skillset.

141 points by sebg 5y ago

""" Even in the major well-known well-respected Julia packages – I’ll avoid naming names – the source code has very obvious cases of unused local variables, dead code branches that can never be reached, etc. """ Please name names. that way we can fix stuff. Other than that, great post!

lern_too_spel 5y ago
I think the author is pushing for better code quality tooling in Julia instead of having people manually fix these problems.
- adgjlsfhk1 5y ago
  we absolutely should, but in the short term, fixing issues is good.

machinekob 5y ago

I strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means so its easier to read some omega then for example learning_rate or lr. But for us mortal who cross multiple domains its just getting extremely frustrating to read full math based notation without any extra info about notation in package/functions etc. so debugging multiple sub-packages is just getting too time consuming as you have to learn both person style of writing code, whole scientific notation and get domain knowledge before you can even touch the code.

bobbylarrybobby 5y ago
And if you ever do want to edit the code, you have to know the name of every non-ASCII symbol the codebase uses if you want to type out those same symbols without copying and pasting them. If you're not familiar with the material, entering a character like ξ can be a real challenge, and is actually more keystrokes than just typing "xi".
- tagrun 5y ago
  Difference is 1 keystroke: you type \xi. Why is that a "real challenge"?
  - animal_spirits 5y ago
    The challenge comes from a person like me, who doesn't know off the top of my head which Greek letter ξ is. So for each of these symbols I'd have to google them and learn it, or have some notepad where I can copy paste the needed symbol
    - tagrun 5y ago
      So you don't know the Greek alphabet, but write high-performance computing code involving non-trivial math? (Julia's main use case is HPC)
      - belval 5y ago
        The author example is using the greek letter "eta" instead of spelling out "learning_rate", this is a pretty damning example.
      - ShamelessC 5y ago
        What? Did you not read the parent of this thread? > strongly agree with readability in my opinion its cause Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters means In any case, I program HPC stuff myself with pytorch and no - I don't know the Greek alphabet and probably don't understand "non trivial math". The assumption that these people can't contribute is pretty off-putting honestly. More engineers would join such efforts if there wasn't so much gatekeeping.
        tagrun 5y ago
        So you think there's a cabal of "gatekeeper" scientists who are trying to hold the hordes of capable (yet somehow unfamiliar with the corresponding mathematical literature) engineers from making useful contributions to their libraries, because they use the conventional notation from the scientific literature? They should instead tread carefully and dispose of all Greek letters in their code to ensure you are not put off, because that's the real reason that stops you from making useful contributions?
aaplok 5y ago
> Academia people live in "bubbles" and they assume everyone knew what a domain specific terms and greek letters Naming things by their English name is not more universal than using Greek letters. It's just serving amother group of people who live in a different bubble.
- belval 5y ago
  Yes and no, the example that the author gives is actually a very good one: > Many Julia APIs look like Optimiser(η=...) rather than Optimiser(learning_rate=...). This is a pretty unreadable convention. The learning rate is a well known name that basically every one will understand, on the other hand, "η" or eta, is not even used everywhere in the literature with some papers using alpha instead. This just looks clever, it's a pretty bad parameter name.
  - melissalobos 5y ago
    > The learning rate is a well known name that basically every one will understand Absolutely! Because as we all know, everyone speaks English. The GP's point was that greek letters are used in lots and lots of papers even written in other languages. I have read quite a few papers in Japanese that used exactly the same conventions with respect to the greek letters and latin letters used.
    - belval 5y ago
      How many researchers in the ML/DL community don't speak English? I don't have hard numbers but I highly doubt that it's a significant proportion. What is the reach of your Japanese papers when almost no-one outside of Japan can read Japanese? Even China, despite their best effort to de-westernize their culture still uses English in their research papers. And if all the above wasn't enough, Julia's libraries are still all in English so if an hypothetical researcher's English is so poor that they don't know what "learning rate" is, I'd venture that they'll have trouble programming in Julia/JAX/PyTorch.
      - 5y ago
agumonkey 5y ago
I understand both camps but I believe, these are superficial problems. It's like worrying about the comfort of seat in the operating room of a nuclear plant.
- nextos 5y ago
  Exactly. I actually find Julia's ecosystem (not the language) way more approachable than Python's. In Python, most libraries are big monoliths. Whereas in Julia, libraries are small and composable. Furthermore, it's the same language all the way down. Python's libraries are superb, but the learning curve to develop (not to use them) is really steep.
  - jstx1 5y ago
    I don't understand. What do you mean by "learning curve to develop" an existing Python library?
    - borodi 5y ago
      I imagine it has to do with the fact that lots of python libraries are mostly c/c++ or fortran, while Julia packages are usually just Julia.
    - adgjlsfhk1 5y ago
      Python's story for package development is a mess compared to Julia. If you know what you're doing you can make a new Julia package in the general registry with unit tests, CI, and code coverage in about 30 minutes. To contribute to an existing repository, you can ]dev the package to download the git repository, @edit a function to open the source in an editor, and push the changes to a PR in about 10 minutes. This makes it much easier for package users to turn into package developers. I've contributed to a bunch of Julia libraries that if I still used python, I wouldn't have contributed to the equivalent python library because contributing to Julia packages is at least an order of magnitude easier.
- abakus 5y ago
  superficial you say. How about I name these in chinese in my package?

SemanticStrengh 5y ago

The most next gen autodiff library probably is https://github.com/breandan/kotlingrad https://github.com/breandan/kotlingrad because of its features, ergonomy and type safety

martinsmit 5y ago
Idk, Enzyme is pretty next gen, all the way down to LLVM code. https://github.com/EnzymeAD/Enzyme https://github.com/EnzymeAD/Enzyme
- mhh__ 5y ago
  So enzyme is pretty cool but IMO the documentation is terrible and the API is absolutely insane.
  - wmoses 5y ago
    Enzyme dev here. The documentation is not just bad, it is in fact actively missing for several things (like forward mode AD, split mode, recent vector). Having recently finished up a number of big technical challenges (forward, vector, split, BLAS), we're hoping to have a big documentation-athon at the end of summer (all are welcome)!
    - SemanticStrengh 5y ago
      On HN I am often seeing libraries/blog authors react very quickly to references of their projects. I'm curious, how do you achieve this, is it a coincidence and where you reading HN comments, or do you have a crwaler/API than send you a notification when your project github link is found in a new HN comment ?
      - wmoses 5y ago
        I was actually socializing with folks at the Cambridge Area Julia meetup where I gave a talk on Enzyme’s Julia bindings no less when someone shared this. So in this case pure coincidence.

tagrun 5y ago

In his item #1, he links to https://discourse.julialang.org/t/loaderror-when-using-interpolations-as-input-for-a-neural-ode/51224/6 https://discourse.julialang.org/t/loaderror-when-using-inter... The issue is actually a Zygote bug, a Julia package for auto-differentiation, and is not directly related to Julia codebase (or Flux package) itself. Furthermore, the problematic code is working fine now, because DiffEqFlux has switched to Enzyme, which doesn't have that bug. He should first confirm whether the problem he is citing is actually a problem or not. Item #2, again another Zygote bug. Item #3, which package? That sounds like an hyperbole, an extrapolation from a small sample, and "I'll avoid naming names" is a lazy excuse that would hide this. It is similarly easy to point to poorly written (or poorly documented) JAX code or Python code as well, so that doesn't prove that "Julia is lacklustre and JAX shines". Also, as an academician, I strongly disagree that learning_rate=... is better than η=..., but that's a matter of convention & taste, with no bearing on the correctness or performance of the package/language. It's bikeshedding. I agree that errors are usually not "instructable" in Julia ML packages (which needs to be improved), so monkey typing is less likely to succeed. Item #4 is such a nitpicker. Sure, Julia may not have a special syntax for that particular fringe array slicing like Numpy does and you'll instead need to make a function call, but matrix/tensor code in Numpy is usually filled with calls to zips and cats with explicit indices, whereas in Julia, no explicit indexing needs to be done usually. Also, one can nitpick similarly in the opposite direction: Julia has many language features lacking from JAX or Python, why not talk about those as well? I'm not a huge fan of Julia either (mainly because of it's garbage collected nature), but this is such a low-effort criticism of it.

blindseer 5y ago
I’ve been using Julia since 2017 and still do on a day to day basis, and I agree with the author in a lot of cases, even his subjective naming conventions gripes. The author’s biggest criticism is that Julia doesn’t have tooling to make the developer experience better. There’s Revise, JuliFormatter, LanguageServer and Jet, but the development experience in Python is enviable. There’s like 3 different REPLs, at least two competing linters and auto formatters. It’s okay to admit that these are places Julia is lacking. I think your kind of response to criticism about Julia is what gives the Julia community a bad name, in my opinion. What is wrong with saying these things suck and need improvements? Would you rather Julia not improve and stay the way it is right now forever? Surely I hope not.

longemen3000 5y ago

I feel called out on the academic part hahaah. I simply want to code state of the art (thermodynamic) models, and at least julia helps by providing easy testing and publishing infraestructure. but obviously we can't compete with a corporation in code quality (we are trying!) Unrelated, but for small sizes, i really prefer to use forward mode in julia (Via ForwardDiff.jl) instead of Zygote. the overhead of reverse ADing over an arbitrary function with mutation is not worth it.

tagrun 5y ago
In the context of neural networks with differential equations (which appears to be the original poster's field), the trade-off depends: https://diffeqflux.sciml.ai/dev/ControllingAdjoints/ https://diffeqflux.sciml.ai/dev/ControllingAdjoints/
- longemen3000 5y ago
  yeah, my systems are really small in comparison (1-20) but with higher order derivatives (up to 4th order), so reverse AD is not the best in that regard

blindseer 5y ago

Another massive frustration for me is that Julia has no formal way to say "here are public functions and these are private" but does have a completely orthogonal way of saying "these are functions that will populate your global namespace if you use `using`", i.e. `export variable_name`, and people absolutely confuse the hell out of these. I don't think there's even agreement in the Julia community if you should use `export` for your public API or not. And if you misspell the exports or change the variable in question, Julia won't even warn you about it. That is straight crazy behavior to me, and I still don't understand how that hasn't been changed. The `using` + `import` packages in Julia combined with how `export`s work make it SUCH a confusing and frustrating experience for beginners in Julia. I personally like mathematical symbols when I'm writing and reading code in my domain, but I do feel very lost when I'm reading Julia code outside of my area of expertise. All my colleagues hate it too (hard to grep, hard to type if you don't know the math and are just a software engineer) and I'm coming around to the idea of not using it or documenting it explicitly. The fact that mutable structs are easier to use but immutable structs are more performant, the lack of composition of fields the way Go handles it, the lack of traits or interfaces, the sorry state of compilation time, the non existent tooling all lead a beginner / intermediate Julia developer in the wrong direction in my opinion. It's very easy to write code that is straight up broken or just not efficient in Julia, and that's probably why I won't pick it for a big project going forward. But I'm still keeping my eye on it. Maybe in 5 years it'll be the language for a lot of the jobs?

mountainriver 5y ago
Yeah I really want to like Julia but for all the reasons you mentioned I’ll just use python. Also python is making a ton of improvements and looks better by the day
socialdemocrat 5y ago
I use export for my public functions. What is wrong with that? How exactly does Python signal what is public? Nothing but conventions like underscore, which btw Julia developers use as well. If I really really don't want somebody to use a function, I might underscore it but generally I don't see the point.

xico 5y ago

Tangentially related but there is an effort to get some of the features of JAX into PyTorch: https://pytorch.org/functorch/ https://pytorch.org/functorch/

patrick451 5y ago

> It’s pretty common to see posts on the Julia Discourse saying “XYZ library doesn’t work”, followed by a reply from one of the library maintainers stating something like “This is an upstream bug in the new version a.b.c of the ABC library, which XYZ depends upon. We’ll get a fix pushed ASAP.” And > First of all, it’s not in the Flux documentation. Instead it’s in the documentation for a separate component library – Zygote. So you have to check both, and you have to know to check both. People love to rave about how the julia ecosystem is made up of tons of tiny composable libraries instead of large monolithic libraries like scipy. To me, that sounds like a major headache, and the quotes above are partly why. I'd rather read docs for one library than two. The chances that the maintainers of a large monolithic library break internal compatibility is way smaller than the chance that two disconnected projects become incompatible.

rg111 5y ago
> People love to rave about how the julia ecosystem is made up of tons of tiny composable libraries instead of large monolithic libraries like scipy. To me, that sounds like a major headache Absolutely agree. First and foremost, I need to get my work done. I do not want to learn all those libraries. The API design, its quality, philosophy (or lack thereof) of framework will be different across all the frameworks. And I am not sure I want too many libraries in my codebase. It is a whole different level of technical debt.

alphachloride 5y ago

One of the biggest hurdles to my use of Jax is that it doesn't work on Windows, my daily driver. It's inconvenient when playing around with a new tool that requires a whole other OS to run. I'm hoping that gets figured out soon.

make3 5y ago
does it not work on WSL ?
- rsfern 5y ago
  I think WSL is the official workaround There is also https://github.com/cloudhan/jax-windows-builder https://github.com/cloudhan/jax-windows-builder

socialdemocrat 5y ago

Ouch! As a big Julia fan I cannot exactly complain about these criticism. They seem overall very fair. Although some things like tracking down problems I think is affected a lot by what you are used to. I am way slower figuring out problems in Python than in Julia. But the heavily academic code and often poor user documentation for third party packages is a very real problem. I would like to point out that I think the Julia core libraries are better documented than Python. But third party stuff often has a long way to go. I also think Julia may suffer a bit from people writing a lot of CLEVER code. I am computer science guy who feel lost in the midst of all these brainy Julia PhDs, postdocs and researchers. I think maybe many of us more regular programmers have a bit healthier habits in writing code a little bit less clever and and perhaps more verbose. It is not all bad though. I think that the APIs to a lot of Julia libraries are quite well thought out. It is more the inside of a lot of these packages which can look a bit messy. Perhaps Julia code quality and documentation can improve when the language starts expanding more outside academia and more among general programmers, industry etc.

ChrisRackauckas 5y ago

The main idea (both pro and con) with Jax is its "automatic performance" model. Essentially, Jax uses a tracer down to XLA which then does some linear algebra optimizations to generate code. For the DSL case where Jax is generally used, this tends to be "good enough" for a good number of use cases. The issue is that when it's not "good enough", it doesn't give package developers much recourse to really dig into the details and optimize for the things which the compiler (and compilers in general) is unable to do. For example, we released the library SimpleChains.jl that was optimized for a lot of cases for small neural networks (https://julialang.org/blog/2022/04/simple-chains/ https://julialang.org/blog/2022/04/simple-chains/). There was discussion about Jax, and we saw that for any case where individuals had a modern multi-core CPU threaded machine setup, SimpleChains.jl had about a 10x performance improvement on CPU over the Jax equinox library. This shouldn't be surprising to anyone because the details of how that was achieved was quite clear (memory handling, manual SIMD bits, etc.), though those of course are not the kinds of things you get "automatically" with either language. The main difference of having a full programming language instead of a DSL is that such code can be written to manually optimize what the compilers are not doing. And this is the pattern that tends to repeat. For small static-like differential equation solves, Jax is fine, but for example the PDE code optimizations done in this tutorial (https://diffeq.sciml.ai/dev/tutorials/faster_ode_example/#Example-Accelerating-Linear-Algebra-PDE-Semi-Discretization https://diffeq.sciml.ai/dev/tutorials/faster_ode_example/#Ex...) are completely incompatible with the Jax programming model and accelerate the PDE solve by more than 10x. And if you look into detail at things showing Jax is fine for PDEs, they are using the slower code style and comparing to that slower Julia code, not the faster iteration form. Of course, these are optimizations most people don't know to do so they are the kind of extra order of magnitude you see in Julia packages over Jax, but not as much in user code. Another repeat of this pattern is the sub-optimality of vmap. It doesn't run code completely independently so it's not optimal in CPU cases where you'd want to effectively use multithreading or MPI, but it's also not generating kernels so it's not optimal in the case of GPUs (for example, we have a new GPU-based ODE solver coming out soon that outperforms the older Julia form and the Jax form by like 100x. Again, no surprises here because special-built GPU kernels have already demonstrated this as possible from C++ directly in this context). So vmap will always get you parallelism, but when you build a library you have to leave it behind at some point if you really want to keep optimizing. I think what it does is admirable. If you can get a code to compile with Jax (which is a big if, it does require pure functions and a lot of extra things Python code normally doesn't satisfy) then it does get you something relatively performant. But there's a reason why what's mentioned here is Flux and Zygote, not SimpleChains and Enzyme, the real tools the Julia community is pushing towards with the full mutation support and the full speed behind it. Jax can compare in the former case, but not the latter case.

dklend122 5y ago

There are several repeating (valid) critiques. There is also a sense that things won't get better. I don't think that's necessarily the case. I can't give timelines, but I'll list the themes and point to the very active work going on in those areas: 1. Static analysis/ it's hard to write large codebases of correct Julia code. That's being addressed on several fronts: a. Work on both traits/interfaces and sound static typing is progressing in Jan Vitek's group at Northwestern b. User extensible type lattice for correctness proof of programs. See here: https://news.ycombinator.com/item?id=26136212 https://news.ycombinator.com/item?id=26136212 c. Jet.jl already catches method errors at JIT time. It's under-used IMO. 2. Compilation time and package dev latency. a. We're getting close to native code caching, and more: https://discourse.julialang.org/t/precompile-why/78770/8 https://discourse.julialang.org/t/precompile-why/78770/8 As you'll also read, the difficulty is due to important tradeoffs Julia made with composability and aggressive specialization...but it's not fundamental and can be surmounted. Yes there's been some pain, but in the end hopefully we'll have something approximating the best of both worlds. b. Runtime free static compilation: https://www.youtube.com/watch?v=YsNC4oO0rLA https://www.youtube.com/watch?v=YsNC4oO0rLA c. Semantics for separate compilation: https://gist.github.com/JeffBezanson/dd86043ef867954bd7e2163ab66f8b4e https://gist.github.com/JeffBezanson/dd86043ef867954bd7e2163... Some combination of the above should also address deployment scenarios, whether that be CLI, mobile, browser or even embedded (Yes, with an (expanding) subset of code, StaticCompiler can produce binaries down to the 10s of Kbs with real working Julia programs that use non trivial abstractions like subtyping and higher order functions) 3. Zygote is rough a. Enzyme will do (almost) whole language AD, including mutation. b. Diffractor.jl

catelm 5y ago
Do you have a link to some information regarding the work on traits/interfaces and static typing, please?

g42gregory 5y ago

>> Where does JAX shine vs. Julia? ...Documentation...Code Quality... I am not sure I need to go beyond that. The decision is somewhat easy here. I am surprised that integration is not mentioned. JAX is just Python (library) so it could be easily integrated into the rest of the system. It fits very well into Python/C skillset.