3 ms·

As much as it pains me to say it, I don't think Julia will. It looks to me like the practical problems with Julia, while addressable, are being addressed too sl

As much as it pains me to say it, I don't think Julia will. It looks to me like the practical problems with Julia, while addressable, are being addressed too slowly. There is simply too many rough edges and usability problems as it is now, and at the current pace it will take maybe 10 or 15 years to address them. On the other hand, the major use case for Julia is to have a fast, dynamic language. And it seems to me the time horizon for Python to become fast, or Rust or C++ to become dynamic is indefinite, so Julia is still the best bet in that space.

__s 3y ago
What's wrong with JS/TS or Lua as a fast dynamic language?
- jakobnissen 3y ago
  Javascript doesn't compile to native code, so isn't as fast. I've never tried LuaJIT, though, that's supposed to be on par with Julia.
  - ReleaseCandidat 3y ago
    > Javascript doesn't compile to native code, so isn't as fast. There are AOT compilers for JS.
    - jakobnissen 3y ago
      So there is for Python, but it's still slow. JS has slow semantics. It can't be made in the same performance league as Julia.
  - yabbs 3y ago
    I think you mean Julia is on par with LuaJIT? ..maybe exceeds?
- markkitti 3y ago
  One of my favorite things recently is using JS/TS with Julia, usually via a Pluto.jl notebook. There are also some nice demonstrations showing Julia compiled to web assembly. https://tshort.github.io/WebAssemblyCompiler.jl/stable/examples/lorenz/?p1=9.6 https://tshort.github.io/WebAssemblyCompiler.jl/stable/examp...
- tonyarkles 3y ago
  I mean, JS, TS, and Python all have at least one major problem: threading. I don’t recall how Lua threads work but I have a feeling it too suffers.
  - __s 3y ago
    There's threading in lua https://github.com/torch/threads https://github.com/torch/threads
    - tonyarkles 3y ago
      Ahhh, interesting. That looks to me like it's more appropriate for "multiple threads working on independent problems" than "multiple threads working on the same problem" due to the potential serialization overhead and limitations. Each thread has its own lua_State. However, we provide a serialization scheme which allows automatic sharing for several Torch objects (storages, tensors and tds types). Sharing of vanilla lua objects is not possible, but instances of classes that support serialization (eg. classic objects with using require 'classic.torch' or those created with torch.class) can be shared, but remember that only the memory in tensor storages and tds objects will be shared by the instances, other fields will be copies. Also if synchronization is required that must be implemented by the user (ie. with mutex). That's not general-purpose multi-processor threading, that's solving it for a very specific sub-problem.
edgyquant 3y ago
Python seems to be making rapid strides towards becoming fast
- oivey 3y ago
  It’s getting faster but not C fast. Its semantics mean it basically never will. You’ll always need C extensions.
  - markkitti 3y ago
    Using Cython via pure Python mode seems pretty compelling if you all you need is compilation. Compilability is just the beginning though.
    - oivey 3y ago
      Cython is pretty nice, especially for wrappers around C. Julia’s ccall is honestly even better. Numba is also really nice, and for just compiling against arrays I like it better than Cython. Neither help if you need data structures or abstractions, though.
- mardifoufs 3y ago
  I don't think so, not the core language at least. In my tests I think I saw a 20-30% improvement at best compared to versions from 5 years ago. Not saying that that's still not a great job from the core maintainers, but I don't think Julia has to fear python getting fast. The issue is more that python is getting to the point where the ML ecosystem has already accepted using a slow language as long as everything around it is fast. Though as I said before, sometimes no amount of c++/c escape hatches can improve performance since you have to use python objects at some point or another and that will be the bottleneck. But by then, you won't needing some of the stuff Julia offers like the REPL and notebooks etc. I haven't used Julia a lot but to me it's in a weird spot where it would be ideal to start projects with in theory since you won't need to outgrow the language you are starting with, since it's fast enough and has a pretty good/maintainable/sane design. But then you are sacrificing so much and will need much more time to get started that you might not ever get to that point anyways. Just as an example, debugging obscure problems or deploying pytorch models that are more custom in prod is already pretty daunting at times, and it's the "best" and most popular ML framework in the world. I can't imagine how much more time consuming it would be when using a much smaller/less used library/framework. So yeah all of that to say that being way faster isn't how Julia will win. Maybe a push from an influent player/big tech might give it the momentum it needs.
ubj 3y ago
Although I mostly agree with this sentiment and have deep concerns about Julia's flaws, I will say that Julia has a very high ceiling. If the right concerns were addressed in a timely manner, I wouldn't be surprised to see its adoption rate accelerate.
deleted 3y ago
[deleted]

__s 3y ago

What's wrong with JS/TS or Lua as a fast dynamic language?

jakobnissen 3y ago
Javascript doesn't compile to native code, so isn't as fast. I've never tried LuaJIT, though, that's supposed to be on par with Julia.
- ReleaseCandidat 3y ago
  > Javascript doesn't compile to native code, so isn't as fast. There are AOT compilers for JS.
  - jakobnissen 3y ago
    So there is for Python, but it's still slow. JS has slow semantics. It can't be made in the same performance league as Julia.
- yabbs 3y ago
  I think you mean Julia is on par with LuaJIT? ..maybe exceeds?
markkitti 3y ago
One of my favorite things recently is using JS/TS with Julia, usually via a Pluto.jl notebook. There are also some nice demonstrations showing Julia compiled to web assembly. https://tshort.github.io/WebAssemblyCompiler.jl/stable/examples/lorenz/?p1=9.6 https://tshort.github.io/WebAssemblyCompiler.jl/stable/examp...
tonyarkles 3y ago
I mean, JS, TS, and Python all have at least one major problem: threading. I don’t recall how Lua threads work but I have a feeling it too suffers.
- __s 3y ago
  There's threading in lua https://github.com/torch/threads https://github.com/torch/threads
  - tonyarkles 3y ago
    Ahhh, interesting. That looks to me like it's more appropriate for "multiple threads working on independent problems" than "multiple threads working on the same problem" due to the potential serialization overhead and limitations. Each thread has its own lua_State. However, we provide a serialization scheme which allows automatic sharing for several Torch objects (storages, tensors and tds types). Sharing of vanilla lua objects is not possible, but instances of classes that support serialization (eg. classic objects with using require 'classic.torch' or those created with torch.class) can be shared, but remember that only the memory in tensor storages and tds objects will be shared by the instances, other fields will be copies. Also if synchronization is required that must be implemented by the user (ie. with mutex). That's not general-purpose multi-processor threading, that's solving it for a very specific sub-problem.

edgyquant 3y ago

Python seems to be making rapid strides towards becoming fast

oivey 3y ago
It’s getting faster but not C fast. Its semantics mean it basically never will. You’ll always need C extensions.
- markkitti 3y ago
  Using Cython via pure Python mode seems pretty compelling if you all you need is compilation. Compilability is just the beginning though.
  - oivey 3y ago
    Cython is pretty nice, especially for wrappers around C. Julia’s ccall is honestly even better. Numba is also really nice, and for just compiling against arrays I like it better than Cython. Neither help if you need data structures or abstractions, though.
mardifoufs 3y ago
I don't think so, not the core language at least. In my tests I think I saw a 20-30% improvement at best compared to versions from 5 years ago. Not saying that that's still not a great job from the core maintainers, but I don't think Julia has to fear python getting fast. The issue is more that python is getting to the point where the ML ecosystem has already accepted using a slow language as long as everything around it is fast. Though as I said before, sometimes no amount of c++/c escape hatches can improve performance since you have to use python objects at some point or another and that will be the bottleneck. But by then, you won't needing some of the stuff Julia offers like the REPL and notebooks etc. I haven't used Julia a lot but to me it's in a weird spot where it would be ideal to start projects with in theory since you won't need to outgrow the language you are starting with, since it's fast enough and has a pretty good/maintainable/sane design. But then you are sacrificing so much and will need much more time to get started that you might not ever get to that point anyways. Just as an example, debugging obscure problems or deploying pytorch models that are more custom in prod is already pretty daunting at times, and it's the "best" and most popular ML framework in the world. I can't imagine how much more time consuming it would be when using a much smaller/less used library/framework. So yeah all of that to say that being way faster isn't how Julia will win. Maybe a push from an influent player/big tech might give it the momentum it needs.

ubj 3y ago

Although I mostly agree with this sentiment and have deep concerns about Julia's flaws, I will say that Julia has a very high ceiling. If the right concerns were addressed in a timely manner, I wouldn't be surprised to see its adoption rate accelerate.

deleted 3y ago

[deleted]