10 ms·

Julia 1.7 Highlights

singularity2001 5y ago
julia> using Pkg julia> Pkg.update()
- rodonn 5y ago
  Alternatively: Type the ']' key (to enter the package management mode) and then type 'up' (short for update) to update all packages. For more info on this: https://pkgdocs.julialang.org/v1/repl/ https://pkgdocs.julialang.org/v1/repl/
adgjlsfhk1 5y ago
IMO, the best part about this is that 1.6 is officially the new LTS. Hopefully this finally ends people trying to use 1.0.x which at this point is really sub-par.
- StefanKarpinski 5y ago
  It's true that 1.0 feels really old at this point. We're also trying to improve messaging that people should generally not be using LTS unless they work in a really deeply risk-averse organization. Almost everyone should just use the latest release. That messaging should hopefully help make what the LTS release is less important. It just shouldn't matter to most people.
eigenspace 5y ago
This release was a long time coming. Very glad it's now here!
- logankilpatrick 5y ago
  So much hard work from the whole Julia community, it's great to see the release go live!
a-dub 5y ago
time to give it another look!
- moelf 5y ago
  try doing this year of Advent of Code in Julia!
  - fault1 5y ago
    Yes! Julia is quite elegant in AoC. I had a blast doing it in Pluto last year: https://github.com/fonsp/Pluto.jl https://github.com/fonsp/Pluto.jl
    - Mandelmus 5y ago
      I’m doing the same this year. AoC in Julia and Pluto.
    - spratzt 5y ago
      I tried it last year using JupyterLab. I found Julia error messages so unhelpful that eventually I gave up and returned to Python.
      - sundarurfriend 5y ago
        The type-based error messages can be pretty opaque until you have a good grasp of the type system, so those can be harsh/seem unhelpful when you're trying to learn the language (and in this case solve daily problems with time constraints too). Were those the problem or did you have some other examples in mind?
- whimsicalism 5y ago
  i seem to be the last person in the world to prefer c-style syntax. but so much cool stuff happening in julia that it seems silly to avoid diving in on such a basic semantic nit.
  - 5y ago
pella 5y ago
> "We hope to be back in a few months to report > on even more progress in version 1.8!" 1.8rc1 ?
- adgjlsfhk1 5y ago
  We're a ways away from 1.8rc1, but we probably will have a feature freeze for 1.8 soonish (next month or so). Hopefully, 1.8 takes less time to release than 1.7 did.
sundarurfriend 5y ago
This is a really nice write up, hope we get one of these at least once every few releases. The multi-; syntax is something that both looks weird at first glance but is also really convenient and satisfying in its consistency. The weirdness factor will likely go down as we get used to seeing this new construct, while the convenience/ease-of-use goes up - so overall a solid positive to the language.
- ViralBShah 5y ago
  We've done such release highlights for the last 3 releases, and certainly hope to continue! https://julialang.org/blog/2021/11/julia-1.7-highlights/ https://julialang.org/blog/2021/11/julia-1.7-highlights/ https://julialang.org/blog/2021/03/julia-1.6-highlights/ https://julialang.org/blog/2021/03/julia-1.6-highlights/ https://julialang.org/blog/2020/08/julia-1.5-highlights/ https://julialang.org/blog/2020/08/julia-1.5-highlights/
thetwentyone 5y ago
Been really pleased with Julia and happy about the continued progress. Package speedups on Windows are expecially nice for me in this release.
- lukego 5y ago
  Likewise. The core language is pretty amazing and this steady stream of improvements is very impressive and reassuring. Being able to easily install, run, and combine bleeding-edge research tools is fantastic. I'm really enjoying exploring the probabilistic-programming corner of the Juliaverse and finding it much smoother to get up and running with than Python/R tooling.
- dagw 5y ago
  "Package speedups on Windows are expecially nice for me in this release." This is huge! Despite my best efforts Julia as been practically unusable on Windows. There are lots of people at work who could probably replace Matlab with Julia, but this has been a complete showstopper.
fault1 5y ago
I'm really excited about Julia 1.8 and diffractor: https://github.com/JuliaDiff/Diffractor.jl https://github.com/JuliaDiff/Diffractor.jl Keno Fisher did a presentation in a discussion moderated by Simon Peyton Jones here: https://www.youtube.com/watch?v=mQnSRfseu0c https://www.youtube.com/watch?v=mQnSRfseu0c Also combined with enzyme which can differentiate through static paths at the llvm bitcode level: https://enzyme.mit.edu/ https://enzyme.mit.edu/ I wonder what this will enable at the frontier of what is computationally tractable in the combo of physics + ml.
- 22c 5y ago
  If anyone else is wondering what the AD stands for in "Next Generation AD" it's "algorithmic differentiation".
  - suchow 5y ago
    Also known as automatic differentiation or autodiff.
- savant_penguin 5y ago
  What's special about this compared to other ADs? Reading the front page it seems to focus on efficient higher order derivatives, is that it?
  - adgjlsfhk1 5y ago
    The basic answer is it (similar to enzyme) is hooked into a compiler so it can optimize before and after AD. This is important because the typical approach can lead to code that is much harder for the compiler to optimize.
  - KenoFischer 5y ago
    Diffractor solves a couple of issues that are inter-related. I'm gonna contrast with Zygote which is the current "standard" AD in Julia and the most fancy one we have (though there are a couple other AD packages that are useful in certain situations that Zygote is bad at - Diffractor will cover some of these but not all). Essentially, what Zygote does is insert itself into the compiler pipeline at the lowered code stage to perform the AD transform. That is it operates on the form of Julia code before we perform any type analysis, devirtualization or optimizations. Essentially, you can think of Zygote as a lisp-style macro applied automatically/dynamically to every function being called starting at a particular entry point. This works pretty well, but has a couple limitations. The first is that it has absolutely no semantic information available (since it operates on non-inferred code), so it can't use that for optimizations or things like data layout planning, which are important optimizations for a production-grade AD system. Essentially, it's not allowed to know that the `+` symbol it sees is actually the `Base.+` function that does addition, so must make the most pessimistic assumptions. This issue doesn't actually show up that much in machine learning use cases, but it's a big issue when you need to AD scalar code (which happens frequently in various differentiable programming contexts). The next issue with running at this stage is that a lot of existing julia code is written with some mental model of the capabilities of type inference and the optimizer. For example, destructuring code like `a, b = f(x)`, people don't think about at all, but semantically that allocates several tuples and then indexes into them to take them apart. By running the AD transform, you basically double (at least), the complexity of every operation, so in a number of cases patterns that used to completely optimize away are now terribly slow, because they are no longer optimized (and then AD transformed on top of that). A related issue is that because you cannot interleave optimization with the AD transform, if you want to perform nested (i.e. higher order) differentiation, you're gonna get exponential code generation, which you then have to hope the optimizer will cut down again for you, which it often can't all the way, but even if it could, generating exponential code in the first place is bad, because you're gonna wait an exponential amount of time for the compiler to be done with it (and super-exponential in practice, because the compiler is not linear in the size of the input problem), so Zygote is basically unusable for anything beyond second order (and even at second order is a struggle). Lastly, there is also a technical issue, which is that code that operates on lowered IR isn't technically allowed to build any closures, but the AD transform has to do that in order to put the code for the backwards pass somewhere. Zygote gets around this by taking advantage of the fact that the AD transform is pure, so it basically runs everything twice (once to generate the forward pass, once to generate the backwards pass when execution gets to it) and it knows that things match up because the input code is the same. That mostly works, but this dependency isn't visible to the runtime system, so you can get into issues where code is updated in between the forwards and the backwards pass, which breaks Zygote in all sorts of ways. Anyway, Diffractor is designed to fix all of this by "simply" moving the AD transform stage from lowered IR to post-inference (where the optimizer sits). The issue is that Julia the language, currently doesn't really allow semantic transformations post inference (after all optimizers are supposed to make things faster, but not change the outcome of things) and in particular, running optimizers is always optional and the language may choose not to do it. So to get there, we need to do a couple of things: 1. We need to have some sort of wedge into the semantics of the language that allows for optimization-time changes that are semantic. For this, I added `OpaqueClosures`, which are essentially like regular closures, except that they do not have semantics as to their capture lists or the code that they run. Now, this may be confusing to people from some functional languages where all closures have such semantics (you can see SPJ ask this question in the linked discussion we had - didn't think about it ahead of time, because I'm so used to our semantics, so my answer was a bit muddled), but essentially in Julia, the contents of closures is semantically visible and optimizations over closure boundaries are mostly prohibited, because ordinary closures do not close over the world age (i.e. if you have `f = x->sin(x)` and then say `sin(x) = 9999`, then the next execution of `f` will return 9999). So opaque closure basically change this and say "nothing in the system is allowed to look at the code they contain, or the capture list, and also we capture the world age". More importantly though, we now have a datastructure (the capture list of an opaque closure) that is allowed to be changed by the optimizer, so we drive basically drive a truck through that. 2. We need the actual mechanism to move the transform to inference time. This isn't super hard, but we haven't quite finished this yet, and it's something I'm hoping to get to over the next couple of months. It's somewhat intertwined with making the compiler in general more accessible to packages outside the core language. There's about 8-10 different packages that want to do compiler-y things on Julia IR and we really need to figure out a good solution to make that generally possible. But as always, designing core language APIs is a bit tricky, because you're gonna be stuck with them for a while. There's also some other nice bits and pieces in Diffractor. The big one is that I did a fair bit of theoretical exploration at the top of this project to really understand higher order AD. When I first took differential geometry (10 years ago now), I used to joke with my classmates that I had no idea what a second derivative was, because all the textbooks basically just say "look, the tangent bundle of a smooth manifold is a smooth manifold" and never actually really go into higher order derivatives at all. Anyway, I really sat down to work all that out and in the course of that I came across a way to do higher order derivatives more efficiently (under suitable assumptions of what the compiler does) than just nesting the first order transform. In retrospect, it seems pretty obvious, but I can honestly say that I didn't think about that until I worked out the theory. In the course of it, I also managed to make very precise the notion that reverse-mode AD and pullbacks of cotangent vectors are the same thing. This connection was pretty well known in the oral tradition, but I never really saw a convincing writeup of it (and my result extends to higher orders of course). Other than that, it directly uses ChainRules.jl as its AD rule system, where Zygote used to keep its own rules (Zygote and ChainRules developed concurrently, and Zygote was later adapted to look in both places, but its a bit of a mess) and it also has a unified forward mode that (in theory, once it's robust enough), subsumes both ForwardDiff.jl and TaylorSeries.jl. In theory, this should all make for a pretty good AD system, but there's a fair bit of work still to be done to make it robust (though all the pieces except the julia-level mechanism for actually moving the AD pass are done and tested) and as always, time is very limited :).
sharikous 5y ago
Maybe this is the right forum to ask... Why the debug system in Julia is so terribly slow? It seems to me that Debug.jl (or whatever runs in VSCode) interprets, rather than running, the code. The result is that for me debugging is just unusable. The standard way to put breakpoints in an executable is to replace the instruction at which to stop with INT3 (or something analogous in other architectures). Then give the system a callback for your debugger when the CPU receives the interrupt. Is there a way to make Julia's debugger do that?
- simeonschaub 5y ago
  Yes, it uses Debugger.jl, which relies on JuliaInterpreter.jl under the hood, so while you can tell the debugger to compile functions in certain modules, it will mostly interpret your code. You might be interested in https://github.com/JuliaDebug/Infiltrator.jl https://github.com/JuliaDebug/Infiltrator.jl, which uses an approach more similar to what you describe.
  - adgjlsfhk1 5y ago
    The other part of the answer is that currently JuliaInterpreter is really slow because it is a very naive interpreter. Speeding it up by a factor of 5 or so should be relatively easy if anyone wants to try.
    - simeonschaub 5y ago
      I would not go as far as calling it very naive, there has certainly been some work put into optimizing performance within the current design. There are probably some gains to be had by using a different storage format for the IR though as proposed in [1], but it is difficult to say how much of a difference that will make in practice. [1] https://github.com/JuliaDebug/JuliaInterpreter.jl/pull/309 https://github.com/JuliaDebug/JuliaInterpreter.jl/pull/309
- KenoFischer 5y ago
  We had a debugger like that a few years ago, but the experience was unsatisfying for people, because you got the "debugging optimized C++ code" experience with unreliable breakpoints and mostly unavailable debug variables. I took the decision to scrap that and instead put out something simple that's slow but robust and reliable. The plan was always to then use the JIT on top of that to create a "debug-specialized" (using statepoints for local variables rather than DWARF) version of the running code, which should give you perfect debuggability at minimal runtime cost, but it's a fair amount of work that nobody has wanted to do yet. In general, traditional debugging has always taken a bit of a backseat in Julia, because people code is usually decently functional, so they just run it in a Revise loop and write their state dumps directly into the code (you could deride that that printf debugging, but I think it has a bit of a bad rap, particularly in a live-reloading system, where you basically get an execution log of your revising expression on every file update). There are still cases where a traditional debugger is useful, so I'm hoping someone will take that on at some point, but so far there've been higher priorities eleswhere. Also do note that you can switch the debugger into compiled mode, which will be faster, but ignore breakpoints.
savant_penguin 5y ago
"Julia v1.7 is also the first release which runs on Apple Silicon, for example the M1 family of ARM CPUs. " I hope those benchmarks are coming in hot
- LolWolf 5y ago
  Still Tier 3 support, but hopefully it'll be Tier 1 very soon :) (As far as I know, the community is really working on it! And I'm phenomenally excited)
- spekcular 5y ago
  I'm extremely excited about this. But there still many problems, for example a ton of tests failing. For example: https://github.com/JuliaLang/julia/issues/43164 https://github.com/JuliaLang/julia/issues/43164.
  - adgjlsfhk1 5y ago
    Yeah, please file bugs if you find them!
- ChrisRackauckas 5y ago
  >I hope those benchmarks are coming in hot M1 is extremely good for PDEs because of its large cache lines. https://github.com/SciML/DiffEqOperators.jl/issues/407#issuecomment-862807968 https://github.com/SciML/DiffEqOperators.jl/issues/407#issue... The JuliaSIMD tools which are internally used for BLAS instead of OpenBLAS and MKL (because they tend to outperform standard BLAS's for the operations we use https://github.com/YingboMa/RecursiveFactorization.jl/pull/28#issuecomment-890267907 https://github.com/YingboMa/RecursiveFactorization.jl/pull/2...) also generate good code for M1, so that was giving us some powerful use cases right off the bat even before the heroics allowed C/Fortran compilers to fully work on M1.
PostThisTooFast 5y ago
Whatever "Julia" is. Another douchily obscure HN post.
matjet 5y ago
Multidimensional array construction is something I had looked forward to in Julia. I am not convinced that the approach taken in julia1.7 compares favorably with other language implementations (Ignoring R) In my view the numpy syntax has more clarity for this task. [[[1,2],[3,4]],[[5,6],[7,8]]] compared with [1 2;3 4;;;5 6;7 8] It is not immediately clear why [1,2,3,4] is equivalent to [1;2;3;4], but [1,2,,3,4] (and [1,2;3,4] vs [1 2;3 4] ect) is not equivalent to [1;2;;3;4]. For creating a 3d slice, I expected that ";", ";;", ";;;" would each refer to incrementing a specific dimension. Eg, It seems intuitive that if you can create a 2d matrix with [1 2;3 4], then you should be able to make a 3d tensor with [1 2;3 4;;5 6;7 8]
- wodenokoto 5y ago
  Yeah, I looked up the manual and completely fail to understand how the syntax is supposed to be read and written.
  - mbauman 5y ago
    I'd love to figure out where the disconnect is — and how we can make the manual more clear. It's a pretty simple rule: the number of semicolons specifies the dimension in which you "move". I've seen two disconnects, but yours might be different * Julia's arrays are column major but when you use spaces to write them, you do so in row major fashion. This new syntax enables a column major input: `[1 2; 3 4]` is equivalent to `[1; 3;; 2; 4]`. * When you're using spaces, it might feel "funny" to jump from one semicolon (which concatenates the rows in the first dimension) to three semicolons (which concatenates the matrices in the third dimension) in an expression like `[1 2; 3 4;;; 5 6; 7 8]`, but the key is that spaces first build rows and then the semicolons concatenate them along a particular dimension. Anyhow, if you can expand on what's causing trouble, it'd be great to figure out how to improve the description in the manual.
    - fault1 5y ago
      There has got to be a way to sprinkle emojis into the documentation: https://github.com/under-Peter/OMEinsum.jl#learn-by-examples https://github.com/under-Peter/OMEinsum.jl#learn-by-examples
pjmlp 5y ago
Lots of nice goodies, quite interesting to follow on Julia's development.
pepoluan 5y ago
I love how the Xoshiro PRNG Family is replacing Mersenne Twister more and more.
tpoacher 5y ago
I used to love Julia, but it increasingly makes this Koan make more and more sense: > A martial arts student went to his teacher and said earnestly, “I am devoted to studying your martial system. How long will it take me to master it?” The teacher’s reply was casual, “Ten years.” Impatiently, the student answered,”But I want to master it faster than that. I will work very hard. I will practice everyday, ten or more hours a day if I have to. How long will it take then?” The teacher thought for a moment, “20 years.” (originally seen in the context of this article: https://brianlui.dog/2020/05/10/beware-of-tight-feedback-loops https://brianlui.dog/2020/05/10/beware-of-tight-feedback-loo...)
- amkkma 5y ago
  In this analogy, Julia is the student? Or are you and Julia is the martial arts? I'd be curious to hear more specific critiques if you don't mind

singularity2001 5y ago

julia> using Pkg julia> Pkg.update()

rodonn 5y ago
Alternatively: Type the ']' key (to enter the package management mode) and then type 'up' (short for update) to update all packages. For more info on this: https://pkgdocs.julialang.org/v1/repl/ https://pkgdocs.julialang.org/v1/repl/

IMO, the best part about this is that 1.6 is officially the new LTS. Hopefully this finally ends people trying to use 1.0.x which at this point is really sub-par.

StefanKarpinski 5y ago
It's true that 1.0 feels really old at this point. We're also trying to improve messaging that people should generally not be using LTS unless they work in a really deeply risk-averse organization. Almost everyone should just use the latest release. That messaging should hopefully help make what the LTS release is less important. It just shouldn't matter to most people.

eigenspace 5y ago

This release was a long time coming. Very glad it's now here!

logankilpatrick 5y ago
So much hard work from the whole Julia community, it's great to see the release go live!

a-dub 5y ago

time to give it another look!

moelf 5y ago
try doing this year of Advent of Code in Julia!
- fault1 5y ago
  Yes! Julia is quite elegant in AoC. I had a blast doing it in Pluto last year: https://github.com/fonsp/Pluto.jl https://github.com/fonsp/Pluto.jl
  - Mandelmus 5y ago
    I’m doing the same this year. AoC in Julia and Pluto.
  - spratzt 5y ago
    I tried it last year using JupyterLab. I found Julia error messages so unhelpful that eventually I gave up and returned to Python.
    - sundarurfriend 5y ago
      The type-based error messages can be pretty opaque until you have a good grasp of the type system, so those can be harsh/seem unhelpful when you're trying to learn the language (and in this case solve daily problems with time constraints too). Were those the problem or did you have some other examples in mind?
whimsicalism 5y ago
i seem to be the last person in the world to prefer c-style syntax. but so much cool stuff happening in julia that it seems silly to avoid diving in on such a basic semantic nit.
- 5y ago

pella 5y ago

> "We hope to be back in a few months to report > on even more progress in version 1.8!" 1.8rc1 ?

adgjlsfhk1 5y ago
We're a ways away from 1.8rc1, but we probably will have a feature freeze for 1.8 soonish (next month or so). Hopefully, 1.8 takes less time to release than 1.7 did.

sundarurfriend 5y ago

This is a really nice write up, hope we get one of these at least once every few releases. The multi-; syntax is something that both looks weird at first glance but is also really convenient and satisfying in its consistency. The weirdness factor will likely go down as we get used to seeing this new construct, while the convenience/ease-of-use goes up - so overall a solid positive to the language.

ViralBShah 5y ago
We've done such release highlights for the last 3 releases, and certainly hope to continue! https://julialang.org/blog/2021/11/julia-1.7-highlights/ https://julialang.org/blog/2021/11/julia-1.7-highlights/ https://julialang.org/blog/2021/03/julia-1.6-highlights/ https://julialang.org/blog/2021/03/julia-1.6-highlights/ https://julialang.org/blog/2020/08/julia-1.5-highlights/ https://julialang.org/blog/2020/08/julia-1.5-highlights/

thetwentyone 5y ago

Been really pleased with Julia and happy about the continued progress. Package speedups on Windows are expecially nice for me in this release.

lukego 5y ago
Likewise. The core language is pretty amazing and this steady stream of improvements is very impressive and reassuring. Being able to easily install, run, and combine bleeding-edge research tools is fantastic. I'm really enjoying exploring the probabilistic-programming corner of the Juliaverse and finding it much smoother to get up and running with than Python/R tooling.
dagw 5y ago
"Package speedups on Windows are expecially nice for me in this release." This is huge! Despite my best efforts Julia as been practically unusable on Windows. There are lots of people at work who could probably replace Matlab with Julia, but this has been a complete showstopper.

fault1 5y ago

I'm really excited about Julia 1.8 and diffractor: https://github.com/JuliaDiff/Diffractor.jl https://github.com/JuliaDiff/Diffractor.jl Keno Fisher did a presentation in a discussion moderated by Simon Peyton Jones here: https://www.youtube.com/watch?v=mQnSRfseu0c https://www.youtube.com/watch?v=mQnSRfseu0c Also combined with enzyme which can differentiate through static paths at the llvm bitcode level: https://enzyme.mit.edu/ https://enzyme.mit.edu/ I wonder what this will enable at the frontier of what is computationally tractable in the combo of physics + ml.

22c 5y ago
If anyone else is wondering what the AD stands for in "Next Generation AD" it's "algorithmic differentiation".
- suchow 5y ago
  Also known as automatic differentiation or autodiff.
savant_penguin 5y ago
What's special about this compared to other ADs? Reading the front page it seems to focus on efficient higher order derivatives, is that it?
- adgjlsfhk1 5y ago
  The basic answer is it (similar to enzyme) is hooked into a compiler so it can optimize before and after AD. This is important because the typical approach can lead to code that is much harder for the compiler to optimize.
- KenoFischer 5y ago
  Diffractor solves a couple of issues that are inter-related. I'm gonna contrast with Zygote which is the current "standard" AD in Julia and the most fancy one we have (though there are a couple other AD packages that are useful in certain situations that Zygote is bad at - Diffractor will cover some of these but not all). Essentially, what Zygote does is insert itself into the compiler pipeline at the lowered code stage to perform the AD transform. That is it operates on the form of Julia code before we perform any type analysis, devirtualization or optimizations. Essentially, you can think of Zygote as a lisp-style macro applied automatically/dynamically to every function being called starting at a particular entry point. This works pretty well, but has a couple limitations. The first is that it has absolutely no semantic information available (since it operates on non-inferred code), so it can't use that for optimizations or things like data layout planning, which are important optimizations for a production-grade AD system. Essentially, it's not allowed to know that the `+` symbol it sees is actually the `Base.+` function that does addition, so must make the most pessimistic assumptions. This issue doesn't actually show up that much in machine learning use cases, but it's a big issue when you need to AD scalar code (which happens frequently in various differentiable programming contexts). The next issue with running at this stage is that a lot of existing julia code is written with some mental model of the capabilities of type inference and the optimizer. For example, destructuring code like `a, b = f(x)`, people don't think about at all, but semantically that allocates several tuples and then indexes into them to take them apart. By running the AD transform, you basically double (at least), the complexity of every operation, so in a number of cases patterns that used to completely optimize away are now terribly slow, because they are no longer optimized (and then AD transformed on top of that). A related issue is that because you cannot interleave optimization with the AD transform, if you want to perform nested (i.e. higher order) differentiation, you're gonna get exponential code generation, which you then have to hope the optimizer will cut down again for you, which it often can't all the way, but even if it could, generating exponential code in the first place is bad, because you're gonna wait an exponential amount of time for the compiler to be done with it (and super-exponential in practice, because the compiler is not linear in the size of the input problem), so Zygote is basically unusable for anything beyond second order (and even at second order is a struggle). Lastly, there is also a technical issue, which is that code that operates on lowered IR isn't technically allowed to build any closures, but the AD transform has to do that in order to put the code for the backwards pass somewhere. Zygote gets around this by taking advantage of the fact that the AD transform is pure, so it basically runs everything twice (once to generate the forward pass, once to generate the backwards pass when execution gets to it) and it knows that things match up because the input code is the same. That mostly works, but this dependency isn't visible to the runtime system, so you can get into issues where code is updated in between the forwards and the backwards pass, which breaks Zygote in all sorts of ways. Anyway, Diffractor is designed to fix all of this by "simply" moving the AD transform stage from lowered IR to post-inference (where the optimizer sits). The issue is that Julia the language, currently doesn't really allow semantic transformations post inference (after all optimizers are supposed to make things faster, but not change the outcome of things) and in particular, running optimizers is always optional and the language may choose not to do it. So to get there, we need to do a couple of things: 1. We need to have some sort of wedge into the semantics of the language that allows for optimization-time changes that are semantic. For this, I added `OpaqueClosures`, which are essentially like regular closures, except that they do not have semantics as to their capture lists or the code that they run. Now, this may be confusing to people from some functional languages where all closures have such semantics (you can see SPJ ask this question in the linked discussion we had - didn't think about it ahead of time, because I'm so used to our semantics, so my answer was a bit muddled), but essentially in Julia, the contents of closures is semantically visible and optimizations over closure boundaries are mostly prohibited, because ordinary closures do not close over the world age (i.e. if you have `f = x->sin(x)` and then say `sin(x) = 9999`, then the next execution of `f` will return 9999). So opaque closure basically change this and say "nothing in the system is allowed to look at the code they contain, or the capture list, and also we capture the world age". More importantly though, we now have a datastructure (the capture list of an opaque closure) that is allowed to be changed by the optimizer, so we drive basically drive a truck through that. 2. We need the actual mechanism to move the transform to inference time. This isn't super hard, but we haven't quite finished this yet, and it's something I'm hoping to get to over the next couple of months. It's somewhat intertwined with making the compiler in general more accessible to packages outside the core language. There's about 8-10 different packages that want to do compiler-y things on Julia IR and we really need to figure out a good solution to make that generally possible. But as always, designing core language APIs is a bit tricky, because you're gonna be stuck with them for a while. There's also some other nice bits and pieces in Diffractor. The big one is that I did a fair bit of theoretical exploration at the top of this project to really understand higher order AD. When I first took differential geometry (10 years ago now), I used to joke with my classmates that I had no idea what a second derivative was, because all the textbooks basically just say "look, the tangent bundle of a smooth manifold is a smooth manifold" and never actually really go into higher order derivatives at all. Anyway, I really sat down to work all that out and in the course of that I came across a way to do higher order derivatives more efficiently (under suitable assumptions of what the compiler does) than just nesting the first order transform. In retrospect, it seems pretty obvious, but I can honestly say that I didn't think about that until I worked out the theory. In the course of it, I also managed to make very precise the notion that reverse-mode AD and pullbacks of cotangent vectors are the same thing. This connection was pretty well known in the oral tradition, but I never really saw a convincing writeup of it (and my result extends to higher orders of course). Other than that, it directly uses ChainRules.jl as its AD rule system, where Zygote used to keep its own rules (Zygote and ChainRules developed concurrently, and Zygote was later adapted to look in both places, but its a bit of a mess) and it also has a unified forward mode that (in theory, once it's robust enough), subsumes both ForwardDiff.jl and TaylorSeries.jl. In theory, this should all make for a pretty good AD system, but there's a fair bit of work still to be done to make it robust (though all the pieces except the julia-level mechanism for actually moving the AD pass are done and tested) and as always, time is very limited :).

sharikous 5y ago

Maybe this is the right forum to ask... Why the debug system in Julia is so terribly slow? It seems to me that Debug.jl (or whatever runs in VSCode) interprets, rather than running, the code. The result is that for me debugging is just unusable. The standard way to put breakpoints in an executable is to replace the instruction at which to stop with INT3 (or something analogous in other architectures). Then give the system a callback for your debugger when the CPU receives the interrupt. Is there a way to make Julia's debugger do that?

simeonschaub 5y ago
Yes, it uses Debugger.jl, which relies on JuliaInterpreter.jl under the hood, so while you can tell the debugger to compile functions in certain modules, it will mostly interpret your code. You might be interested in https://github.com/JuliaDebug/Infiltrator.jl https://github.com/JuliaDebug/Infiltrator.jl, which uses an approach more similar to what you describe.
- adgjlsfhk1 5y ago
  The other part of the answer is that currently JuliaInterpreter is really slow because it is a very naive interpreter. Speeding it up by a factor of 5 or so should be relatively easy if anyone wants to try.
  - simeonschaub 5y ago
    I would not go as far as calling it very naive, there has certainly been some work put into optimizing performance within the current design. There are probably some gains to be had by using a different storage format for the IR though as proposed in [1], but it is difficult to say how much of a difference that will make in practice. [1] https://github.com/JuliaDebug/JuliaInterpreter.jl/pull/309 https://github.com/JuliaDebug/JuliaInterpreter.jl/pull/309
KenoFischer 5y ago
We had a debugger like that a few years ago, but the experience was unsatisfying for people, because you got the "debugging optimized C++ code" experience with unreliable breakpoints and mostly unavailable debug variables. I took the decision to scrap that and instead put out something simple that's slow but robust and reliable. The plan was always to then use the JIT on top of that to create a "debug-specialized" (using statepoints for local variables rather than DWARF) version of the running code, which should give you perfect debuggability at minimal runtime cost, but it's a fair amount of work that nobody has wanted to do yet. In general, traditional debugging has always taken a bit of a backseat in Julia, because people code is usually decently functional, so they just run it in a Revise loop and write their state dumps directly into the code (you could deride that that printf debugging, but I think it has a bit of a bad rap, particularly in a live-reloading system, where you basically get an execution log of your revising expression on every file update). There are still cases where a traditional debugger is useful, so I'm hoping someone will take that on at some point, but so far there've been higher priorities eleswhere. Also do note that you can switch the debugger into compiled mode, which will be faster, but ignore breakpoints.

savant_penguin 5y ago

"Julia v1.7 is also the first release which runs on Apple Silicon, for example the M1 family of ARM CPUs. " I hope those benchmarks are coming in hot

LolWolf 5y ago
Still Tier 3 support, but hopefully it'll be Tier 1 very soon :) (As far as I know, the community is really working on it! And I'm phenomenally excited)
spekcular 5y ago
I'm extremely excited about this. But there still many problems, for example a ton of tests failing. For example: https://github.com/JuliaLang/julia/issues/43164 https://github.com/JuliaLang/julia/issues/43164.
- adgjlsfhk1 5y ago
  Yeah, please file bugs if you find them!
ChrisRackauckas 5y ago
>I hope those benchmarks are coming in hot M1 is extremely good for PDEs because of its large cache lines. https://github.com/SciML/DiffEqOperators.jl/issues/407#issuecomment-862807968 https://github.com/SciML/DiffEqOperators.jl/issues/407#issue... The JuliaSIMD tools which are internally used for BLAS instead of OpenBLAS and MKL (because they tend to outperform standard BLAS's for the operations we use https://github.com/YingboMa/RecursiveFactorization.jl/pull/28#issuecomment-890267907 https://github.com/YingboMa/RecursiveFactorization.jl/pull/2...) also generate good code for M1, so that was giving us some powerful use cases right off the bat even before the heroics allowed C/Fortran compilers to fully work on M1.

PostThisTooFast 5y ago

Whatever "Julia" is. Another douchily obscure HN post.

matjet 5y ago

Multidimensional array construction is something I had looked forward to in Julia. I am not convinced that the approach taken in julia1.7 compares favorably with other language implementations (Ignoring R) In my view the numpy syntax has more clarity for this task. [[[1,2],[3,4]],[[5,6],[7,8]]] compared with [1 2;3 4;;;5 6;7 8] It is not immediately clear why [1,2,3,4] is equivalent to [1;2;3;4], but [1,2,,3,4] (and [1,2;3,4] vs [1 2;3 4] ect) is not equivalent to [1;2;;3;4]. For creating a 3d slice, I expected that ";", ";;", ";;;" would each refer to incrementing a specific dimension. Eg, It seems intuitive that if you can create a 2d matrix with [1 2;3 4], then you should be able to make a 3d tensor with [1 2;3 4;;5 6;7 8]

wodenokoto 5y ago
Yeah, I looked up the manual and completely fail to understand how the syntax is supposed to be read and written.
- mbauman 5y ago
  I'd love to figure out where the disconnect is — and how we can make the manual more clear. It's a pretty simple rule: the number of semicolons specifies the dimension in which you "move". I've seen two disconnects, but yours might be different * Julia's arrays are column major but when you use spaces to write them, you do so in row major fashion. This new syntax enables a column major input: `[1 2; 3 4]` is equivalent to `[1; 3;; 2; 4]`. * When you're using spaces, it might feel "funny" to jump from one semicolon (which concatenates the rows in the first dimension) to three semicolons (which concatenates the matrices in the third dimension) in an expression like `[1 2; 3 4;;; 5 6; 7 8]`, but the key is that spaces first build rows and then the semicolons concatenate them along a particular dimension. Anyhow, if you can expand on what's causing trouble, it'd be great to figure out how to improve the description in the manual.
  - fault1 5y ago
    There has got to be a way to sprinkle emojis into the documentation: https://github.com/under-Peter/OMEinsum.jl#learn-by-examples https://github.com/under-Peter/OMEinsum.jl#learn-by-examples

pjmlp 5y ago

Lots of nice goodies, quite interesting to follow on Julia's development.

pepoluan 5y ago

I love how the Xoshiro PRNG Family is replacing Mersenne Twister more and more.

tpoacher 5y ago

I used to love Julia, but it increasingly makes this Koan make more and more sense: > A martial arts student went to his teacher and said earnestly, “I am devoted to studying your martial system. How long will it take me to master it?” The teacher’s reply was casual, “Ten years.” Impatiently, the student answered,”But I want to master it faster than that. I will work very hard. I will practice everyday, ten or more hours a day if I have to. How long will it take then?” The teacher thought for a moment, “20 years.” (originally seen in the context of this article: https://brianlui.dog/2020/05/10/beware-of-tight-feedback-loops https://brianlui.dog/2020/05/10/beware-of-tight-feedback-loo...)

amkkma 5y ago
In this analogy, Julia is the student? Or are you and Julia is the martial arts? I'd be curious to hear more specific critiques if you don't mind