6 ms·

I'm excited to see `--trim` finally make it, but it only works when all code from entrypoints are statically inferrable. In any non-toy Julia program that's not

I'm excited to see `--trim` finally make it, but it only works when all code from entrypoints are statically inferrable. In any non-toy Julia program that's not going to be the case. Julia sorely needs a static mode and a static analyzer that can check for correctness. It also needs better sum type support and better error messages (static and runtime). In 2020, I thought Julia would be _the_ language to use in 2025. Today I think that won't happen until 2030, if even then. The community is growing too slowly, core packages have extremely few maintainers, and Python and Rust are sucking the air out of the room. This talk at JuliaCon was a good summary of how developers using Rust are so much more productive in Rust than in Julia that they switched away from Julia: https://www.youtube.com/watch?v=gspuMS1hSQo https://www.youtube.com/watch?v=gspuMS1hSQo Which is pretty telling. It takes a overcoming a certain inertia to move from any language. Given all that, outside of depending heavily on DifferentialEquations.jl, I don't know why someone would pick Julia over Python + Rust.

tagrun 1y ago
Telling what? Did you actually listen to the talk that you linked to, or read the top comment there by Chris Rackauckas? > Given all that, outside of depending heavily on DifferentialEquations.jl, I don't know why someone would pick Julia over Python + Rust. See his last slide. And no, they didn't replace their Julia use in its entirety with Rust, despite his organization being a Rust shop. Considering Rust as a replacement for Julia makes as much sense to me as to considering C as a replacement for Mathematica; Julia and Mathematica are domain specific (scientific computation) languages, not general systems programming languages. Neither Julia nor Mathematica is a good fit for embedded device programming. I also find it amusing how you criticize Julia while praising Python (which was originally a "toy" scripting language succeeding ABC, but found some accidental "gaps" to fit in historically) within the narrative that you built. > In any non-toy Julia program that's not going to be the case. Why?
- wolvesechoes 1y ago
  Python has useful and rich ecosystem that grows every day. Julia is mostly pile of broken promises (it neither reads as Python, nor it runs as C, at least not without significant effort required to produce curated benchmarks) and desperate hype generators.
  - tagrun 1y ago
    Since you have a rosy picture of Python, I assume you're young. Python has been mostly a fringe/toy language for 2 decades, until around ~2010, when a Python fad started not too different from the Rust fad of today, and at some point Google started using it seriously and thought they can fix Python but gave up eventually. The fad lived on and kept evolving and somehow found some popularity with SciPy and then ML. I used it in 90s a little, and I found the language bad for anything other than replacing simple bash scripts or simple desktop applications or a desktop calculator, and I still think it is (but sure, there are people who disagree and think it is a good language). It was slow and didn't have type system, you didn't know whether your code would crash or not until you run that line of code, and the correctness of your program depended on invisible characters. "Ecosystem" is not a part of the language, and in any case, the Python ecosystem is not written in Python, because Python is not a suitable language for scientific computing, which is unsurprising because that's not what it was designed for. It is ironic you bring up hype to criticize Julia while praising Python which found popularity thanks to hype rather than technical merit. What promise are you referring to? Who promised you what? It's a programming language.
    - wolvesechoes 1y ago
      > Ecosystem" is not a part of the language, and in any case, the Python ecosystem is not written in Python, because Python is not a suitable language for scientific computing Doesn't matter. Languages do not matter, ecosystems do, for they determine what is practically achievable. And it doesn't matter that Python ecosystem relies on huge amounts of C/C++ code. Python people made the effort to wrap this code, document it and maintain those wrappers. Other people use such code through Python APIs. Yes, every language with FFI can do the same. For some reason none achieved that. Even people using Julia use PythonCall.jl, that's how much Python is unsuitable. > What promise are you referring to? Who promised you what? It's a programming language. Acting dumb is poor rhetorical strategy, and ignores such a nice rhetorical advice as principle of charity - it is quite obvious that I didn't mean that programming language made any promise. Making a promise is something that only people can do. And Julia creators and people promoting it made quite bombastic claims throughout the years that turned out to not have much support in reality. I leave your assumptions about my age or other properties to you.
- blindseer 1y ago
  > Telling what? Did you actually listen to the talk that you linked to, or read the top comment there by Chris Rackauckas? To clarify exactly where I'm coming from, I'm going to expand on my thoughts here. What is Julia's central conceit? It aims to solve "the two language" problem, i.e. the problem where prototyping or rapid development is done in a dynamic and interactive language like Python or MATLAB, and then moved for production to a faster and less flexible language like Rust or C++. This is exactly what the speaker in the talk addresses. They are still using Julia for prototyping, but their production use of Julia was replaced with Rust. I've heard several more anecdotal stories of the exact same thing occurring. Here's another high profile instance of Julia not making it to production: https://discourse.julialang.org/t/julia-used-to-prototype-whatsapp-audio/115626/4 https://discourse.julialang.org/t/julia-used-to-prototype-wh... Julia is failing at its core conceit. Julia as a community have to start thinking about what makes a language successful in production. Quote from the talk: > "(developers) really love writing Rust ... and I get where they are coming from, especially around the tooling." Julia's tooling is ... just not good. Try working several hundred thousand line project in Julia and it is painful for so many reasons. If you don't have a REPL open all the time with the state of your program loaded in the REPL and in your head, Julia becomes painful to work in. The language server crashes all the time, completion is slow, linting has so many false positives, TDD is barebones etc. It's far too easy to write type unstable code. And the worst part is you can write code that you think is type stable, but with a minor refactor your performance can just completely tank. Optimizing for maintaining Julia code over a long period of time with a team just feels futile. That said, is Python perfect? Absolutely not. There's so many things I wish were different. But Python was designed (or at the very least evolved) to be a glue language. Being able to write user friendly interfaces to performant C or C++ code was the reason the language took off the way it did. And the Python language keeps evolving to make it easier to write correct Python code. Type hinting is awesome and Python has much better error messages (static and runtime). I'm far more productive prototyping in Python, even if executing code is slower. When I want to make it fast, it is almost trivial to use PyO3 with Rust to make what I want to run fast. Rust is starting to build up packages used for scientific computing. There's also Numba and Cython, which are pretty awesome and have saved me in a pickle. As a glue language Python is amazing. And jumping into a million line project still feels practical (Julia's `include` feature alone would prevent this from being tenable). The community is growing still, and projects like `uv` and `ty` are only going to make Python proliferate more. I do think Julia is ideal for an individual researcher, where one person can keep every line of code in their head and for code that is written to be thrown away. But I'm certainly not betting the near future on this language.
wrathofmonads 1y ago
I don't think Julia was designed for pure overhead projects in memory-constrained environments, or for squeezing out that last 2% of hardware performance to cut costs, like C++, Rust or Zig. Julia is the language to use in 2025 if what you’re looking for is a JIT-compiled, multiple-dispatch language that lets you write high-performance technical computing code to run on a cluster or on your laptop for quick experimentation, while also being metaprogrammable and highly interactive, whether for modelling, simulation, optimisation, image processing etc.
- throwawaymaths 1y ago
  actually I think it sort of was, I remember berkeley squeezing a ton of perf out of their cray for a crazy task because it was easy to specialize some wild semi-sparse matrix computations onto an architecture with strange memory/cache bottlenecks, while being guaranteed that the results are still okay.
  - leephillips 1y ago
    Yes. Remember that the four languages that have been used for very high performance computing are Fortran, C, C++, and Julia: https://www.hpcwire.com/off-the-wire/julia-joins-petaflop-club/ https://www.hpcwire.com/off-the-wire/julia-joins-petaflop-cl... For large-scale physics simulations, Fortran or Julia are the obvious choices.
mccoyb 1y ago
These are exactly the feelings that I left with from the community in ~2021 (along with the AD story, which never really materialized _within_ Julia - Enzyme had to come from outside Julia to “save it” - or materialized in a way (Zygote) whose compilation times were absolutely unacceptable compared to competitors like JAX) More and more over time, I’ve begun to think that the method JIT architecture is a mistake, that subtyping is a mistake. Subtyping makes abundant sense when paired with multiple dispatch — so perhaps my qualms are not precise there … but it also seems like several designs for static interfaces have sort of bounced off the type system. Not sure, and can’t defend my claims very well. Julia has much right, but a few things feel wrong in ways that spiral up to the limitations in features like this one. Anyways, excited to check back next year to see myself proven wrong.
- jakobnissen 1y ago
  I basically agree with subtyping (but not multiple dispatch). More importantly, I think it's important to recognize that Julia has a niche that literally no one can compete with - interactive, dynamic and high performance. Like, what exactly is the alternative? Python? Too slow. Static languages? Unusable for interactive exploration and data science. That leaves you with hybrids, like Python/Cython, or Python/Rust or Numba, but taken on their own term, these are absolutely terrible languages. Python/Rust is not safe (due to FFI), certainly not pleasant to develop in, and no matter how you cut your code between the languages, you always lose. You always want your Python part to be in Rust so you get static analysis, safety and speed. You always want your Rust part to be in Python, so you can experiment with it easier and introspect.
  - blindseer 1y ago
    I think multiple dispatch (useful as it is) is a little overrated. There's a significant portion of the time where I know I have a closed set of cases to cover, and an enum type with a match-like syntax would have worked better for that. For interfaces, multiple dispatch is good but again I would have preferred a trait based approach with static type checking. I largely think multiple dispatch works well in Julia, and it enables writing performant code in an elegant manner. I mostly have smaller gripes about subtyping and the patterns it encourages with multiple dispatch in Julia, and larger gripes about the lack of tooling in Julia. But multiple dispatch is also a hammer where every problem in Julia looks like a nail. And there isn't enough discussion, official or community driven, that expands on this. In my experience the average developer to Julia tends to reach for multiple dispatch without understanding why, mostly because people keep saying it is the best thing since sliced bread. wrt to hybrid languages, honestly, I think Python/Cython is extremely underrated. Sure you can design an entirely new language like Mojo or Julia, but imo it offers only incremental value over Python/Cython. I would love to peek into another universe where all that money, time and effort for Mojo and Julia went to Cython instead. And I personally don't think Python/Rust is as bad. With a little discipline (and some tests), you can ensure your boundary is safe, for you and your team. Rust offers so much value that I would take on the pain of going through FFI. PyO3 simplifies this significantly. The development of `polars` is a good case study for how Rust empowers Python. I think the Julia community could use some reflection on why it hasn't produced the next `polars`. My personal experience with Julia developers (both in-person and online) is that they often believe multiple dispatch is so compelling that any person that "saw the light" would obviously naturally flock to Julia. Instead, I think the real challenge is meeting users where they are and addressing their needs directly. The fastest way to grow Julia as a language is to tag along Python's success. Would I prefer a single language that solves all my problems? Yes. But that single language is not Julia, yet, for me. PS: I really enjoy your blog posts and comments.
pjmlp 1y ago
There is no "_the_ language to use", I always pick the language based on the project delivery requirements, like what is tier 1 support on a specific SDK, not pick the project based on the language.

tagrun 1y ago

Telling what? Did you actually listen to the talk that you linked to, or read the top comment there by Chris Rackauckas? > Given all that, outside of depending heavily on DifferentialEquations.jl, I don't know why someone would pick Julia over Python + Rust. See his last slide. And no, they didn't replace their Julia use in its entirety with Rust, despite his organization being a Rust shop. Considering Rust as a replacement for Julia makes as much sense to me as to considering C as a replacement for Mathematica; Julia and Mathematica are domain specific (scientific computation) languages, not general systems programming languages. Neither Julia nor Mathematica is a good fit for embedded device programming. I also find it amusing how you criticize Julia while praising Python (which was originally a "toy" scripting language succeeding ABC, but found some accidental "gaps" to fit in historically) within the narrative that you built. > In any non-toy Julia program that's not going to be the case. Why?

wolvesechoes 1y ago
Python has useful and rich ecosystem that grows every day. Julia is mostly pile of broken promises (it neither reads as Python, nor it runs as C, at least not without significant effort required to produce curated benchmarks) and desperate hype generators.
- tagrun 1y ago
  Since you have a rosy picture of Python, I assume you're young. Python has been mostly a fringe/toy language for 2 decades, until around ~2010, when a Python fad started not too different from the Rust fad of today, and at some point Google started using it seriously and thought they can fix Python but gave up eventually. The fad lived on and kept evolving and somehow found some popularity with SciPy and then ML. I used it in 90s a little, and I found the language bad for anything other than replacing simple bash scripts or simple desktop applications or a desktop calculator, and I still think it is (but sure, there are people who disagree and think it is a good language). It was slow and didn't have type system, you didn't know whether your code would crash or not until you run that line of code, and the correctness of your program depended on invisible characters. "Ecosystem" is not a part of the language, and in any case, the Python ecosystem is not written in Python, because Python is not a suitable language for scientific computing, which is unsurprising because that's not what it was designed for. It is ironic you bring up hype to criticize Julia while praising Python which found popularity thanks to hype rather than technical merit. What promise are you referring to? Who promised you what? It's a programming language.
  - wolvesechoes 1y ago
    > Ecosystem" is not a part of the language, and in any case, the Python ecosystem is not written in Python, because Python is not a suitable language for scientific computing Doesn't matter. Languages do not matter, ecosystems do, for they determine what is practically achievable. And it doesn't matter that Python ecosystem relies on huge amounts of C/C++ code. Python people made the effort to wrap this code, document it and maintain those wrappers. Other people use such code through Python APIs. Yes, every language with FFI can do the same. For some reason none achieved that. Even people using Julia use PythonCall.jl, that's how much Python is unsuitable. > What promise are you referring to? Who promised you what? It's a programming language. Acting dumb is poor rhetorical strategy, and ignores such a nice rhetorical advice as principle of charity - it is quite obvious that I didn't mean that programming language made any promise. Making a promise is something that only people can do. And Julia creators and people promoting it made quite bombastic claims throughout the years that turned out to not have much support in reality. I leave your assumptions about my age or other properties to you.
blindseer 1y ago
> Telling what? Did you actually listen to the talk that you linked to, or read the top comment there by Chris Rackauckas? To clarify exactly where I'm coming from, I'm going to expand on my thoughts here. What is Julia's central conceit? It aims to solve "the two language" problem, i.e. the problem where prototyping or rapid development is done in a dynamic and interactive language like Python or MATLAB, and then moved for production to a faster and less flexible language like Rust or C++. This is exactly what the speaker in the talk addresses. They are still using Julia for prototyping, but their production use of Julia was replaced with Rust. I've heard several more anecdotal stories of the exact same thing occurring. Here's another high profile instance of Julia not making it to production: https://discourse.julialang.org/t/julia-used-to-prototype-whatsapp-audio/115626/4 https://discourse.julialang.org/t/julia-used-to-prototype-wh... Julia is failing at its core conceit. Julia as a community have to start thinking about what makes a language successful in production. Quote from the talk: > "(developers) really love writing Rust ... and I get where they are coming from, especially around the tooling." Julia's tooling is ... just not good. Try working several hundred thousand line project in Julia and it is painful for so many reasons. If you don't have a REPL open all the time with the state of your program loaded in the REPL and in your head, Julia becomes painful to work in. The language server crashes all the time, completion is slow, linting has so many false positives, TDD is barebones etc. It's far too easy to write type unstable code. And the worst part is you can write code that you think is type stable, but with a minor refactor your performance can just completely tank. Optimizing for maintaining Julia code over a long period of time with a team just feels futile. That said, is Python perfect? Absolutely not. There's so many things I wish were different. But Python was designed (or at the very least evolved) to be a glue language. Being able to write user friendly interfaces to performant C or C++ code was the reason the language took off the way it did. And the Python language keeps evolving to make it easier to write correct Python code. Type hinting is awesome and Python has much better error messages (static and runtime). I'm far more productive prototyping in Python, even if executing code is slower. When I want to make it fast, it is almost trivial to use PyO3 with Rust to make what I want to run fast. Rust is starting to build up packages used for scientific computing. There's also Numba and Cython, which are pretty awesome and have saved me in a pickle. As a glue language Python is amazing. And jumping into a million line project still feels practical (Julia's `include` feature alone would prevent this from being tenable). The community is growing still, and projects like `uv` and `ty` are only going to make Python proliferate more. I do think Julia is ideal for an individual researcher, where one person can keep every line of code in their head and for code that is written to be thrown away. But I'm certainly not betting the near future on this language.

wrathofmonads 1y ago

I don't think Julia was designed for pure overhead projects in memory-constrained environments, or for squeezing out that last 2% of hardware performance to cut costs, like C++, Rust or Zig. Julia is the language to use in 2025 if what you’re looking for is a JIT-compiled, multiple-dispatch language that lets you write high-performance technical computing code to run on a cluster or on your laptop for quick experimentation, while also being metaprogrammable and highly interactive, whether for modelling, simulation, optimisation, image processing etc.

throwawaymaths 1y ago
actually I think it sort of was, I remember berkeley squeezing a ton of perf out of their cray for a crazy task because it was easy to specialize some wild semi-sparse matrix computations onto an architecture with strange memory/cache bottlenecks, while being guaranteed that the results are still okay.
- leephillips 1y ago
  Yes. Remember that the four languages that have been used for very high performance computing are Fortran, C, C++, and Julia: https://www.hpcwire.com/off-the-wire/julia-joins-petaflop-club/ https://www.hpcwire.com/off-the-wire/julia-joins-petaflop-cl... For large-scale physics simulations, Fortran or Julia are the obvious choices.

mccoyb 1y ago

These are exactly the feelings that I left with from the community in ~2021 (along with the AD story, which never really materialized _within_ Julia - Enzyme had to come from outside Julia to “save it” - or materialized in a way (Zygote) whose compilation times were absolutely unacceptable compared to competitors like JAX) More and more over time, I’ve begun to think that the method JIT architecture is a mistake, that subtyping is a mistake. Subtyping makes abundant sense when paired with multiple dispatch — so perhaps my qualms are not precise there … but it also seems like several designs for static interfaces have sort of bounced off the type system. Not sure, and can’t defend my claims very well. Julia has much right, but a few things feel wrong in ways that spiral up to the limitations in features like this one. Anyways, excited to check back next year to see myself proven wrong.

jakobnissen 1y ago
I basically agree with subtyping (but not multiple dispatch). More importantly, I think it's important to recognize that Julia has a niche that literally no one can compete with - interactive, dynamic and high performance. Like, what exactly is the alternative? Python? Too slow. Static languages? Unusable for interactive exploration and data science. That leaves you with hybrids, like Python/Cython, or Python/Rust or Numba, but taken on their own term, these are absolutely terrible languages. Python/Rust is not safe (due to FFI), certainly not pleasant to develop in, and no matter how you cut your code between the languages, you always lose. You always want your Python part to be in Rust so you get static analysis, safety and speed. You always want your Rust part to be in Python, so you can experiment with it easier and introspect.
- blindseer 1y ago
  I think multiple dispatch (useful as it is) is a little overrated. There's a significant portion of the time where I know I have a closed set of cases to cover, and an enum type with a match-like syntax would have worked better for that. For interfaces, multiple dispatch is good but again I would have preferred a trait based approach with static type checking. I largely think multiple dispatch works well in Julia, and it enables writing performant code in an elegant manner. I mostly have smaller gripes about subtyping and the patterns it encourages with multiple dispatch in Julia, and larger gripes about the lack of tooling in Julia. But multiple dispatch is also a hammer where every problem in Julia looks like a nail. And there isn't enough discussion, official or community driven, that expands on this. In my experience the average developer to Julia tends to reach for multiple dispatch without understanding why, mostly because people keep saying it is the best thing since sliced bread. wrt to hybrid languages, honestly, I think Python/Cython is extremely underrated. Sure you can design an entirely new language like Mojo or Julia, but imo it offers only incremental value over Python/Cython. I would love to peek into another universe where all that money, time and effort for Mojo and Julia went to Cython instead. And I personally don't think Python/Rust is as bad. With a little discipline (and some tests), you can ensure your boundary is safe, for you and your team. Rust offers so much value that I would take on the pain of going through FFI. PyO3 simplifies this significantly. The development of `polars` is a good case study for how Rust empowers Python. I think the Julia community could use some reflection on why it hasn't produced the next `polars`. My personal experience with Julia developers (both in-person and online) is that they often believe multiple dispatch is so compelling that any person that "saw the light" would obviously naturally flock to Julia. Instead, I think the real challenge is meeting users where they are and addressing their needs directly. The fastest way to grow Julia as a language is to tag along Python's success. Would I prefer a single language that solves all my problems? Yes. But that single language is not Julia, yet, for me. PS: I really enjoy your blog posts and comments.

pjmlp 1y ago

There is no "_the_ language to use", I always pick the language based on the project delivery requirements, like what is tier 1 support on a specific SDK, not pick the project based on the language.