4 ms·

Julia could be a great language for ML. It needs more mindshare and developer attention though

singularity2001 1y ago
What's the current state of time to first plot and executable size? Last time it was several seconds to get a 200 MB hello world. I'm sure they are moving in the right direction the only questions is are they there yet?
- moelf 1y ago
  with juliac.jl and --trim, hello world is now 1MB and compiles in a second. more realistic examples of compiling a Julia package into .so: https://indico.cern.ch/event/1515852/contributions/6599313/attachments/3104673/5502050/Jerry_Ling_StaticJulia.pdf https://indico.cern.ch/event/1515852/contributions/6599313/a...
- postflopclarity 1y ago
  improving, slowly. 5 steps forward 3 steps back. 1.9 and 1.10 made huge gains in package precompilation and native code caching. then attentions shifted and there were some regressions in compile times due to unrelated things in 1.11 and the upcoming 1.12. but at the same time, 1.12 will contain an experimental new feature `--trim` as well as some further standardization around entry points to run packages as programs, which is a big step towards generating self-contained small binaries. also nearly all efforts in improving tooling are focused on providing static analysis and helping developers make their program more easily compilable. it's also important a bit to distinguish between a few similar but related needs. most of what I just described applies to generating binaries for arbitrary programs. but for the example you stated "time to first plot" of existing packages, this is already much improved in 1.10 and users (aka non-package-developers) should see sub-second TTFP, and TTFX for most packages they use that have been updated to use the precompilation goodies in recent versions
- adgjlsfhk1 1y ago
  julia> @time begin using Plots display(plot(rand(8))) end 1.074321 seconds On Julia 1.12 (currently at release candidate stage), <1mb hello world is possible with juliac (although juliac in 1.12 is still marked experimental)
- ModernMech 1y ago
  I recently looked into making Julia binaries, and it's not at all a good process. They say it's supported, but it's not exactly as easy as "cargo build" to get a Julia binary out. And the build process involves creating this minimal version of Julia you're expected to ship with your binary, so build times were terrible. I don't know if that gets amortized though. As far as the executable size, it was only 85kb in my test, a bouncing balls simulation. However, it required 300MB of Julia libraries to be shipped with it. About 2/3 of that is in libjulia-codegen.dll, libLLVM-16jl.dll. So you're shipping this chunky runtime and their LLVM backend. If you're willing to pay for that, you can ship a Julia executable. It's a better story than what Python offers, but it's not great if you want small, self-contained executables.
  - postflopclarity 1y ago
    note that as a few other commenters have pointed out, this situation will improve greatly in 1.12 (although still many rough edges)
    - ModernMech 1y ago
      Yeah, that's what I've been hearing about Julia for about 10 years now: "situation will improve greatly in the next version, but still many rough edges remain". I've been involved in a few programming language projects, so I'm sympathetic as to how much work goes into one and how long they can take. At the same time, it makes me wary of Julia, because it highlights that progress is very slow. I think Julia is trying to be too much at once. It's hard enough to be a dynamic, interactive language, but they also want to claim to be performant and compiled. That's a lot of complexity for a small team to handle and deliver on.
      - adgjlsfhk1 1y ago
        Different things have been improving for the past 10 years. TTFX is in a good spot now, multithreading is in a pretty good spot now, GC issues for most people are basically solved. AOT compilation was pretty much the last big item that a lot of people wanted, and that has recently (within the last year) gotten to the point where the code exists, it's merged, and it will be releasing soon (and then spending the next year or two getting improved)
numbers_guy 1y ago
What makes Julia "great" for ML?
- postflopclarity 1y ago
  I would use the term "potentially great" rather than plain "great" but all the normal marketing words: in my opinion it is fast, expressive, and has particularly good APIs for array manipulation
  - numbers_guy 1y ago
    Interesting. I am experimenting with different ML ecosystems and wasn't really considering Julia at all but I put it on the list now.
    - postflopclarity 1y ago
      Glad to hear. I've found it's a very welcoming community. I'll warn you that Julia's ML ecosystem has the most competitive advantage on "weird" types of ML, involving lots of custom gradients and kernels, integration with other pieces of a simulation or diffeq, etc. if you just want to throw some tensors around and train a MLP, you'll certainly end up finding more rough edges than you might in PyTorch
      - salty_biscuits 1y ago
        Yes, my experience has been that it is great if you need to do something particularly weird, but less smooth to do something ordinary.
    - blurbleblurble 1y ago
      If I wanted to get into research ML, I'd pick Julia no doubt. It allows both conventional ML techniques where we throw tons of parameters at the problem, but additionally a more nimble style where we can train over ordinary functions. Combine that with all the cutting edge applied math packages often being automatically compatible with the autodiff and GPU array backends, even if the library authors didn't think about that... it's a recipe for a lot of interesting possibilities.
mdaniel 1y ago
I don't understand why in the world someone would go from one dynamically typed language to another. Even the kernels example cited below is "eh, the types are whatever you want them to be" https://cuda.juliagpu.org/stable/tutorials/introduction/#Writing-your-first-GPU-kernel https://cuda.juliagpu.org/stable/tutorials/introduction/#Wri... Then again, I am also open to the fact that I'm jammed up by the production use of dynamically typed languages, and maybe the "for ML" part means "I code in Jupyter notebooks" and thus give no shits about whether person #2 can understand what's happening
- postflopclarity 1y ago
  It's very important that readers, writers, maintainers, etc. of code are able to easily understand what that code is doing. explicit and strict types on arguments to functions is one way, but certainly not the only way, nor probably the best way to effect that
  - mdaniel 1y ago
    I would actually be curious to hear your perspective on the "best way" that isn't typechecking. I literally cannot comprehend why someone would write such a thing I readily admit that I am biased in that I believe that having a computer check that every reference to every relationship does what it promises, all the time
    - postflopclarity 1y ago
      first and foremost great documentation & design docs cannot be surpassed as a tool to explain and understand code. and that is entirely language agnostic. more generally, the most important bits of a particular function to understand is * what should it be called with * what should it return * what side effects might it have and the "what" here refers to properties in a general sense. types are a good shortcut to signify certain named collections of properties (e.g., the `Int` type has arithmetic properties). but there are other ways to express traits, preconditions, postconditions, etc. besides types
rvz 1y ago
> It needs more mindshare and developer attention though That is the problem. Julia could not compete against Python's mindshare. A competitor to Python needs to be 100% compatible with its ecosystem.

singularity2001 1y ago

What's the current state of time to first plot and executable size? Last time it was several seconds to get a 200 MB hello world. I'm sure they are moving in the right direction the only questions is are they there yet?

moelf 1y ago
with juliac.jl and --trim, hello world is now 1MB and compiles in a second. more realistic examples of compiling a Julia package into .so: https://indico.cern.ch/event/1515852/contributions/6599313/attachments/3104673/5502050/Jerry_Ling_StaticJulia.pdf https://indico.cern.ch/event/1515852/contributions/6599313/a...
postflopclarity 1y ago
improving, slowly. 5 steps forward 3 steps back. 1.9 and 1.10 made huge gains in package precompilation and native code caching. then attentions shifted and there were some regressions in compile times due to unrelated things in 1.11 and the upcoming 1.12. but at the same time, 1.12 will contain an experimental new feature `--trim` as well as some further standardization around entry points to run packages as programs, which is a big step towards generating self-contained small binaries. also nearly all efforts in improving tooling are focused on providing static analysis and helping developers make their program more easily compilable. it's also important a bit to distinguish between a few similar but related needs. most of what I just described applies to generating binaries for arbitrary programs. but for the example you stated "time to first plot" of existing packages, this is already much improved in 1.10 and users (aka non-package-developers) should see sub-second TTFP, and TTFX for most packages they use that have been updated to use the precompilation goodies in recent versions
adgjlsfhk1 1y ago
julia> @time begin using Plots display(plot(rand(8))) end 1.074321 seconds On Julia 1.12 (currently at release candidate stage), <1mb hello world is possible with juliac (although juliac in 1.12 is still marked experimental)
ModernMech 1y ago
I recently looked into making Julia binaries, and it's not at all a good process. They say it's supported, but it's not exactly as easy as "cargo build" to get a Julia binary out. And the build process involves creating this minimal version of Julia you're expected to ship with your binary, so build times were terrible. I don't know if that gets amortized though. As far as the executable size, it was only 85kb in my test, a bouncing balls simulation. However, it required 300MB of Julia libraries to be shipped with it. About 2/3 of that is in libjulia-codegen.dll, libLLVM-16jl.dll. So you're shipping this chunky runtime and their LLVM backend. If you're willing to pay for that, you can ship a Julia executable. It's a better story than what Python offers, but it's not great if you want small, self-contained executables.
- postflopclarity 1y ago
  note that as a few other commenters have pointed out, this situation will improve greatly in 1.12 (although still many rough edges)
  - ModernMech 1y ago
    Yeah, that's what I've been hearing about Julia for about 10 years now: "situation will improve greatly in the next version, but still many rough edges remain". I've been involved in a few programming language projects, so I'm sympathetic as to how much work goes into one and how long they can take. At the same time, it makes me wary of Julia, because it highlights that progress is very slow. I think Julia is trying to be too much at once. It's hard enough to be a dynamic, interactive language, but they also want to claim to be performant and compiled. That's a lot of complexity for a small team to handle and deliver on.
    - adgjlsfhk1 1y ago
      Different things have been improving for the past 10 years. TTFX is in a good spot now, multithreading is in a pretty good spot now, GC issues for most people are basically solved. AOT compilation was pretty much the last big item that a lot of people wanted, and that has recently (within the last year) gotten to the point where the code exists, it's merged, and it will be releasing soon (and then spending the next year or two getting improved)

numbers_guy 1y ago

What makes Julia "great" for ML?

postflopclarity 1y ago
I would use the term "potentially great" rather than plain "great" but all the normal marketing words: in my opinion it is fast, expressive, and has particularly good APIs for array manipulation
- numbers_guy 1y ago
  Interesting. I am experimenting with different ML ecosystems and wasn't really considering Julia at all but I put it on the list now.
  - postflopclarity 1y ago
    Glad to hear. I've found it's a very welcoming community. I'll warn you that Julia's ML ecosystem has the most competitive advantage on "weird" types of ML, involving lots of custom gradients and kernels, integration with other pieces of a simulation or diffeq, etc. if you just want to throw some tensors around and train a MLP, you'll certainly end up finding more rough edges than you might in PyTorch
    - salty_biscuits 1y ago
      Yes, my experience has been that it is great if you need to do something particularly weird, but less smooth to do something ordinary.
  - blurbleblurble 1y ago
    If I wanted to get into research ML, I'd pick Julia no doubt. It allows both conventional ML techniques where we throw tons of parameters at the problem, but additionally a more nimble style where we can train over ordinary functions. Combine that with all the cutting edge applied math packages often being automatically compatible with the autodiff and GPU array backends, even if the library authors didn't think about that... it's a recipe for a lot of interesting possibilities.

mdaniel 1y ago

I don't understand why in the world someone would go from one dynamically typed language to another. Even the kernels example cited below is "eh, the types are whatever you want them to be" https://cuda.juliagpu.org/stable/tutorials/introduction/#Writing-your-first-GPU-kernel https://cuda.juliagpu.org/stable/tutorials/introduction/#Wri... Then again, I am also open to the fact that I'm jammed up by the production use of dynamically typed languages, and maybe the "for ML" part means "I code in Jupyter notebooks" and thus give no shits about whether person #2 can understand what's happening

postflopclarity 1y ago
It's very important that readers, writers, maintainers, etc. of code are able to easily understand what that code is doing. explicit and strict types on arguments to functions is one way, but certainly not the only way, nor probably the best way to effect that
- mdaniel 1y ago
  I would actually be curious to hear your perspective on the "best way" that isn't typechecking. I literally cannot comprehend why someone would write such a thing I readily admit that I am biased in that I believe that having a computer check that every reference to every relationship does what it promises, all the time
  - postflopclarity 1y ago
    first and foremost great documentation & design docs cannot be surpassed as a tool to explain and understand code. and that is entirely language agnostic. more generally, the most important bits of a particular function to understand is * what should it be called with * what should it return * what side effects might it have and the "what" here refers to properties in a general sense. types are a good shortcut to signify certain named collections of properties (e.g., the `Int` type has arithmetic properties). but there are other ways to express traits, preconditions, postconditions, etc. besides types

rvz 1y ago

> It needs more mindshare and developer attention though That is the problem. Julia could not compete against Python's mindshare. A competitor to Python needs to be 100% compatible with its ecosystem.