5 ms·

Here's a thorough answer by someone who has worked a lot on types, optimizations and numerical libraries, and who still uses Python as his first language (for a

Here's a thorough answer by someone who has worked a lot on types, optimizations and numerical libraries, and who still uses Python as his first language (for academic purposes): Why not: from Common Lisp to Julia https://gist.github.com/digikar99/24decb414ddfa15a220b27f6748165d7 https://gist.github.com/digikar99/24decb414ddfa15a220b27f674...

ubj 4y ago
> The unnecessarily-long-for-debugging compilation times certainly put me off. I think I do like the semantics of Julia better than Python, but the compilation times is a deal-breaker for me. Perhaps I might make the switch once this gets fixed in the upcoming years :) I like Julia, but this is a point of frustration for me as well. Compilation time is fine for smaller scripts. But whenever I've tried using more powerful packages such as the SciML ones (DifferentialEquations.jl, DiffEqFlux.jl, Symbolics.jl), the precompilation time during the edit/debug cycle can be excruciating. It's a shame because the SciML packages are so amazing. I'm waiting until Julia's static compilation process improves and is standardized. On-the-fly compilation that disappears every time you close the REPL will never be fast enough to give users a good experience when working with highly complex code. EDIT: Just to be clear, I'm overall confident that things will improve. The folks at Julia have already made good progress in reducing precompilation times from where they used to be. Looking forward to seeing further advancements in the future.
- DNF2 4y ago
  I'm not experienced with the SciML ecosystem, but they have apparently put huge efforts into improving compilation lag, and this has been going on for quite some time: https://twitter.com/ChrisRackauckas/status/1561278092043325440 https://twitter.com/ChrisRackauckas/status/15612780920433254... Quote: "Coming very soon: a version of DifferentialEquations.jl that fully precompiles the solvers on Vector{Float64}, virtually eliminating the any #julialang #sciml JIT lag."
  - crabbygrabby 4y ago
    It's still rough. They've been at it for yrs and it's a perpetual bed of sand... A lot of machines will fail to install it because it's so resource intense to install... Promising idea overall, maybe in three yrs or so it'll be worth using for something outside of research.
    - ChrisRackauckas 4y ago
      It's 0.1 seconds now and faster than one can type, as shown by the GIF. What is the speed where it's not rough?
  - ubj 4y ago
    Nice, that Twitter announcement looks fantastic! Looking forward to trying out the new precompiled version. And agreed that the folks at SciML (and the rest of Julia) have put amazing efforts into reducing the compilation lag from where it used to be :) I'm optimistic that things will improve--it'll just take some time.
    - crabbygrabby 4y ago
      Theyve been at it for half a decade or so. Ignoring compilation times they shuffle the code around so frequently it's only real use imo is for the authors to publish papers and stay three steps ahead of any of it's users hoping for a stable tool after a few cycles give up. It's a shame but it's academia at its finest.
      - DNF2 4y ago
        This part of your criticism seems quite disingenuous. You're simultaneously criticizing them for not improving their code quickly enough, and for changing it too much.
        crabbygrabby 4y ago
        I appreciate your view but seeing as how most Julia projects work this way I sometimes wonder if it's just a problem with the language itself. Not trying to be a troll with impossible expectations, but genuinely the code is unstable, and yes they have been working on it for a long time.
- sundarurfriend 4y ago
  > I like Julia, but this is a point of frustration for me as well. It's a point of frustration for most of the community, it's just something we're willing to put up with (and work around) for the other benefits of the language. If the tradeoff doesn't seem worth it to you, it's totally fine to wait it out until it's in a more acceptable place for you. There's a lot of focus on improvements surrounding this in the recent and upcoming versions, precisely because of this frustration, but it's still going to be a gradual process.
forgotpwd16 4y ago
Nice response. Seems to mostly agree with OP. About the "long-for-debugging compilation times" argument, it is usually raised when running a fresh Julia session to test updated code. The expected workflow is actually more of keeping a single Julia session opened indefinitely and including updates in it. This workflow is enhanced further with Revise, and conceptually is similar to how people usually develop on MATLAB and R/RStudio. Python programmers will only really use this approach inside a Jupyter notebook.
- crabbygrabby 4y ago
  Yea but keep in mind anytime you get a bug your session goes away so you can end up eating hours from a day precompiling. Not worth it with heavy pkgs imo. Wish they had incremental compilation like rust because most compiled Lang's are noticeably faster than Julia precompilation ime.
  - sundarurfriend 4y ago
    > anytime you get a bug your session goes away This only happens for Segmentation fault type crashes which kill the entire Julia process, which shouldn't be common at all. Can you describe when you experience these issues?
    - crabbygrabby 4y ago
      It's super easy to get ooms and seg. Faults in Julia. For a while there this past year you couldn't even Ctrl c to stop the Julia process. It's rough for real work imo. Fine for research.
      - mindB 4y ago
        I don't remember C-c being broken in the past year. Could it have just been a specific program you were running with a tight loop that didn't have any yield points? If so, that's not really unique to Julia. I'd also be interested in your workload that was generating lots of seg faults (oom makes some sense if working with large data since Julia's runtime does add an unfortunate amount of memory overhead.
sundarurfriend 4y ago
> If I had to wish something from Julia, it would be to provide a way to turn off runtime optimization to (radically) speed up compile times for purposes of debugging. 100% agreed on that. I've tried a Julia alias with `--compile=min --optimize=0` options passed in to try to say "please give me responsiveness over runtime performance", but it's still not quite the smooth flow I'd like it to be. > Dynamic Binding Beyond performance, it sounds like dynamic binding would have the same hard-to-debug action-at-a-distance problems that global variables often land you in, so I'm not sure it's worth it. (The specific case the author mentions would also lead to type stability problems, but that's maybe beside the point.) > Structural editing It's hard to process things from gifs, especially since I can't tell what the starting point of the gif is. It vaguely gives me the impression of the Emmet plugin for HTML development [1]. > I am aware julia has a --lisp mode, but I have never found any documentation for it. So, I don’t agree that all the things in julia are well-documented either :). Afaik, the `--lisp` mode is intended to be sort of an easter egg, rather than a real mode for practical coding. I doubt many people use it other than Jeff himself. :) The author doesn't say everything in Julia is well-documented by the way, or even mention Julia documentation. There's just complaints about the Lisp ecosystem's lack of documentation, and perhaps from that an implication that Julia has better docs, but I doubt the author would say all the things are well-documented - there's still quite a way to go for that to be the case. Overall, the article left me more curious to explore Lisp, not less. It didn't feel particularly gloomy, and exposed me to many features of the language that make me really want to try it out. I hope there's more articles like this - in the sense of being intended for a general (non-lisp) audience, and talking about specific features rather than just "it expands your mind, it's programming like you've never done before" type statements. [1] https://emmet.io/ https://emmet.io/

ubj 4y ago

> The unnecessarily-long-for-debugging compilation times certainly put me off. I think I do like the semantics of Julia better than Python, but the compilation times is a deal-breaker for me. Perhaps I might make the switch once this gets fixed in the upcoming years :) I like Julia, but this is a point of frustration for me as well. Compilation time is fine for smaller scripts. But whenever I've tried using more powerful packages such as the SciML ones (DifferentialEquations.jl, DiffEqFlux.jl, Symbolics.jl), the precompilation time during the edit/debug cycle can be excruciating. It's a shame because the SciML packages are so amazing. I'm waiting until Julia's static compilation process improves and is standardized. On-the-fly compilation that disappears every time you close the REPL will never be fast enough to give users a good experience when working with highly complex code. EDIT: Just to be clear, I'm overall confident that things will improve. The folks at Julia have already made good progress in reducing precompilation times from where they used to be. Looking forward to seeing further advancements in the future.

DNF2 4y ago
I'm not experienced with the SciML ecosystem, but they have apparently put huge efforts into improving compilation lag, and this has been going on for quite some time: https://twitter.com/ChrisRackauckas/status/1561278092043325440 https://twitter.com/ChrisRackauckas/status/15612780920433254... Quote: "Coming very soon: a version of DifferentialEquations.jl that fully precompiles the solvers on Vector{Float64}, virtually eliminating the any #julialang #sciml JIT lag."
- crabbygrabby 4y ago
  It's still rough. They've been at it for yrs and it's a perpetual bed of sand... A lot of machines will fail to install it because it's so resource intense to install... Promising idea overall, maybe in three yrs or so it'll be worth using for something outside of research.
  - ChrisRackauckas 4y ago
    It's 0.1 seconds now and faster than one can type, as shown by the GIF. What is the speed where it's not rough?
- ubj 4y ago
  Nice, that Twitter announcement looks fantastic! Looking forward to trying out the new precompiled version. And agreed that the folks at SciML (and the rest of Julia) have put amazing efforts into reducing the compilation lag from where it used to be :) I'm optimistic that things will improve--it'll just take some time.
  - crabbygrabby 4y ago
    Theyve been at it for half a decade or so. Ignoring compilation times they shuffle the code around so frequently it's only real use imo is for the authors to publish papers and stay three steps ahead of any of it's users hoping for a stable tool after a few cycles give up. It's a shame but it's academia at its finest.
    - DNF2 4y ago
      This part of your criticism seems quite disingenuous. You're simultaneously criticizing them for not improving their code quickly enough, and for changing it too much.
      - crabbygrabby 4y ago
        I appreciate your view but seeing as how most Julia projects work this way I sometimes wonder if it's just a problem with the language itself. Not trying to be a troll with impossible expectations, but genuinely the code is unstable, and yes they have been working on it for a long time.
sundarurfriend 4y ago
> I like Julia, but this is a point of frustration for me as well. It's a point of frustration for most of the community, it's just something we're willing to put up with (and work around) for the other benefits of the language. If the tradeoff doesn't seem worth it to you, it's totally fine to wait it out until it's in a more acceptable place for you. There's a lot of focus on improvements surrounding this in the recent and upcoming versions, precisely because of this frustration, but it's still going to be a gradual process.

forgotpwd16 4y ago

Nice response. Seems to mostly agree with OP. About the "long-for-debugging compilation times" argument, it is usually raised when running a fresh Julia session to test updated code. The expected workflow is actually more of keeping a single Julia session opened indefinitely and including updates in it. This workflow is enhanced further with Revise, and conceptually is similar to how people usually develop on MATLAB and R/RStudio. Python programmers will only really use this approach inside a Jupyter notebook.

crabbygrabby 4y ago
Yea but keep in mind anytime you get a bug your session goes away so you can end up eating hours from a day precompiling. Not worth it with heavy pkgs imo. Wish they had incremental compilation like rust because most compiled Lang's are noticeably faster than Julia precompilation ime.
- sundarurfriend 4y ago
  > anytime you get a bug your session goes away This only happens for Segmentation fault type crashes which kill the entire Julia process, which shouldn't be common at all. Can you describe when you experience these issues?
  - crabbygrabby 4y ago
    It's super easy to get ooms and seg. Faults in Julia. For a while there this past year you couldn't even Ctrl c to stop the Julia process. It's rough for real work imo. Fine for research.
    - mindB 4y ago
      I don't remember C-c being broken in the past year. Could it have just been a specific program you were running with a tight loop that didn't have any yield points? If so, that's not really unique to Julia. I'd also be interested in your workload that was generating lots of seg faults (oom makes some sense if working with large data since Julia's runtime does add an unfortunate amount of memory overhead.

sundarurfriend 4y ago

> If I had to wish something from Julia, it would be to provide a way to turn off runtime optimization to (radically) speed up compile times for purposes of debugging. 100% agreed on that. I've tried a Julia alias with `--compile=min --optimize=0` options passed in to try to say "please give me responsiveness over runtime performance", but it's still not quite the smooth flow I'd like it to be. > Dynamic Binding Beyond performance, it sounds like dynamic binding would have the same hard-to-debug action-at-a-distance problems that global variables often land you in, so I'm not sure it's worth it. (The specific case the author mentions would also lead to type stability problems, but that's maybe beside the point.) > Structural editing It's hard to process things from gifs, especially since I can't tell what the starting point of the gif is. It vaguely gives me the impression of the Emmet plugin for HTML development [1]. > I am aware julia has a --lisp mode, but I have never found any documentation for it. So, I don’t agree that all the things in julia are well-documented either :). Afaik, the `--lisp` mode is intended to be sort of an easter egg, rather than a real mode for practical coding. I doubt many people use it other than Jeff himself. :) The author doesn't say everything in Julia is well-documented by the way, or even mention Julia documentation. There's just complaints about the Lisp ecosystem's lack of documentation, and perhaps from that an implication that Julia has better docs, but I doubt the author would say all the things are well-documented - there's still quite a way to go for that to be the case. Overall, the article left me more curious to explore Lisp, not less. It didn't feel particularly gloomy, and exposed me to many features of the language that make me really want to try it out. I hope there's more articles like this - in the sense of being intended for a general (non-lisp) audience, and talking about specific features rather than just "it expands your mind, it's programming like you've never done before" type statements. [1] https://emmet.io/ https://emmet.io/