5 ms·

Back in the day I had trouble convincing my C++ friends to give Julia a try, because Julia's garbage collector was a showstopper. But if you follow these perfor

Back in the day I had trouble convincing my C++ friends to give Julia a try, because Julia's garbage collector was a showstopper. But if you follow these performance tips related to pre-allocation, in-place mutation, avoiding temporary allocations (and maybe avoiding cycling references), you don't struggle with GC performance issues. Looking back, I think the tooling Julia had from the start, combined with the REPL, made it actually really nice to fix these performance issues. Much better than compiling and linking a binary and running it through some heap profiler tool. You could simply do julia> @time f() x.y seconds (N allocations: M bytes) and iteratively improve part of the code base instead of profiling the entire application. (To be fair: back then escape analysis was not implemented in the compiler, and it was hard to avoid silly allocations)

VorpalWay 8mo ago
The issue with writing high performance code in GC languages is that you end up going out of your way writing things in a strange way to avoid the GC. At that point you might as well use a non-GC language. In my experience, the most natural way to write things in Rust is usually the fastest (or close enough) as well. Note: I don't know Julia specifically, but this does apply to other GC languages like Ocaml and Java. Try reading code in those languages that avoid the GC. It looks very strange. In python it is even worse: people try to avoid for loops because they are slower than list comprehensions for example (or at least used to, I haven't written much python for some years now).
- SatvikBeri 8mo ago
  There are some big advantages to having it in the same language. You can write the easy, non-performant version quickly and gradually refactor while having an easy test case. This is especially nice in situations where you expect to throw away a lot of the code you write, e.g. research. Also, you don't have to write most code in a super performance obsessed way – Julia makes it really easy to find the key 5% that needs to be rewritten. We've ported some tens of thousands of lines of numpy-heavy Python, and in practice our Julia code is actually more concise while being about 10x-100x more performant.
  - VorpalWay 8mo ago
    > There are some big advantages to having it in the same language. Sure, but why not write it all in Rust or similar then? (Not writing it all in C++ I would understand.) > This is especially nice in situations where you expect to throw away a lot of the code you write, e.g. research. Right, that is very different from what I do. There is code I wrote 15 years ago that is still in use. And I expect the same would be true in 15 years from now. Though that is also code where a GC is a no-go entirely (the code is hard real-time).
    - SatvikBeri 8mo ago
      > Sure, but why not write it all in Rust or similar then? (Not writing it all in C++ I would understand.) I don't know Rust as well as I'd like, but when we've worked with some strong Rust programmers, their versions of the code are something like 4x as long as equivalent Julia code with minimal performance improvements. And since we primarily care about trading algorithms that don't have binary results, it's pretty helpful to be able to understand those at a glance. Also, Rust's ecosystem for numeric computing still seems pretty underdeveloped, though it's getting better.
    - wolvesechoes 8mo ago
      > Sure, but why not write it all in Rust or similar then? Because it is clunky, ugly and unreadable mess. Which is something you may be willing to pay in some circumstances, but not while researching algorithms, doing data science, writing simulations etc. > Though that is also code where a GC is a no-go entirely (the code is hard real-time). There are GC systems with hard real time guarantees. There is more out there than just OpenJDK and Go. Also, hard real-time systems rarely use runtime allocations, at least with off the shelf allocators.
- eigenspace 8mo ago
  Julia has a big culture and a lot of interfaces built around writing non-allocating code. We sometimes even overemphasize eliminating GC allocations from stuff. Generally, the code ends up looking rather similar to non-GC languages. You create some buffers outside of your performance-sensitive parts, and then thread them through your code so they can be accessed and re-used in the hot loop or whatever. It could be better, e.g. C++ and Rust both have some nice utilities for this stuff that are a bit hard to replicate in Julia, but it's not auch a huge difference, and there's also a lot of advantages on the julia side. E.g. it's really nice to have the GC available for the non-performance critical parts of your code.
  - VorpalWay 8mo ago
    Since I don't know Julia, let me ask: how easy is it to add a use of the GC by mistake in a critical part of the code (maybe a junior dev does it)? Are there any tools to lint against that (some sort of function attribute to deny uses of GC in the function perhaps)? If it happens, how hard is it find the culript lines of code? Because from what I have seen in other GC languages, the answers to any of those questions haven't been great.
    - SatvikBeri 8mo ago
      You can use AllocCheck.jl to guarantee your code doesn't allocate. It's conservative, so it'll sometimes throw false positives, but shouldn't throw false negatives. You apply the checks to function definitions. You can profile memory usage line by line in detail pretty easily with tools like @timeit or @btime In practice I've found it pretty easy to get inner loops down to few or 0 allocations when needed for parallelization
    - dandanua 8mo ago
      It's easy to add, yes. However, the basic @time macro outputs the info about allocations by default. So it is also easy to see if there is a problem.
    - eigenspace 8mo ago
      Tooling for this in julia is evolving, but currently works fairly well. It could be better, but could be a lot worse. There's static analysis tools that might be over-eager finding allocations, and there's also runtime measurement tools which should go into your test suite if it's of vital importance to monitor.
- pjmlp 8mo ago
  Depends on which GC language, people keep forgeting many have C++ like capabilities, besides having a GC. D, C#, Swift, Nim,..... Agree that in Julia's case the flexiblity is not quite there, still much better than using Python and then going to write most of the work in C, C++, Fortran,..... Which is a thing that gets lost quite often in these discussions, just because the last 5% might be a bit harder, doesn't mean we have to throw everything away and start from scratch in another programming language, with its own set of problems.
- dpc_01234 8mo ago
  > In my experience, the most natural way to write things in Rust is usually the fastest (or close enough) as well. Well, a lot of C/Odin/Zig people will point out that Rust's stdlib encourages heap allocations. For actually best performance you typically want to store your data in some data-oriented data model, avoid allocations and so on, which is not exactly against idiomatic Rust, but more than just a typical straighforward Rust just throwing allocations around.
  - VorpalWay 8mo ago
    Fair point, I don't tend to do much with heap in Rust (or any language) as I do hard realtime. You allocate all you will need up front. As for data oriented design: yes, and it depends on your usage pattern. E.g. arrays of structs can still be better than struct of arrays if that matches your cache line access pattern. Zig probably makes SOA easier for the cases where that is more efficient (I haven't used Zig, but I have read a bit about it. It has some cool ideas (comptime), but also things I can't get behind such as lack of RAII, and generally lack of memory safety.)

VorpalWay 8mo ago

The issue with writing high performance code in GC languages is that you end up going out of your way writing things in a strange way to avoid the GC. At that point you might as well use a non-GC language. In my experience, the most natural way to write things in Rust is usually the fastest (or close enough) as well. Note: I don't know Julia specifically, but this does apply to other GC languages like Ocaml and Java. Try reading code in those languages that avoid the GC. It looks very strange. In python it is even worse: people try to avoid for loops because they are slower than list comprehensions for example (or at least used to, I haven't written much python for some years now).

SatvikBeri 8mo ago
There are some big advantages to having it in the same language. You can write the easy, non-performant version quickly and gradually refactor while having an easy test case. This is especially nice in situations where you expect to throw away a lot of the code you write, e.g. research. Also, you don't have to write most code in a super performance obsessed way – Julia makes it really easy to find the key 5% that needs to be rewritten. We've ported some tens of thousands of lines of numpy-heavy Python, and in practice our Julia code is actually more concise while being about 10x-100x more performant.
- VorpalWay 8mo ago
  > There are some big advantages to having it in the same language. Sure, but why not write it all in Rust or similar then? (Not writing it all in C++ I would understand.) > This is especially nice in situations where you expect to throw away a lot of the code you write, e.g. research. Right, that is very different from what I do. There is code I wrote 15 years ago that is still in use. And I expect the same would be true in 15 years from now. Though that is also code where a GC is a no-go entirely (the code is hard real-time).
  - SatvikBeri 8mo ago
    > Sure, but why not write it all in Rust or similar then? (Not writing it all in C++ I would understand.) I don't know Rust as well as I'd like, but when we've worked with some strong Rust programmers, their versions of the code are something like 4x as long as equivalent Julia code with minimal performance improvements. And since we primarily care about trading algorithms that don't have binary results, it's pretty helpful to be able to understand those at a glance. Also, Rust's ecosystem for numeric computing still seems pretty underdeveloped, though it's getting better.
  - wolvesechoes 8mo ago
    > Sure, but why not write it all in Rust or similar then? Because it is clunky, ugly and unreadable mess. Which is something you may be willing to pay in some circumstances, but not while researching algorithms, doing data science, writing simulations etc. > Though that is also code where a GC is a no-go entirely (the code is hard real-time). There are GC systems with hard real time guarantees. There is more out there than just OpenJDK and Go. Also, hard real-time systems rarely use runtime allocations, at least with off the shelf allocators.
eigenspace 8mo ago
Julia has a big culture and a lot of interfaces built around writing non-allocating code. We sometimes even overemphasize eliminating GC allocations from stuff. Generally, the code ends up looking rather similar to non-GC languages. You create some buffers outside of your performance-sensitive parts, and then thread them through your code so they can be accessed and re-used in the hot loop or whatever. It could be better, e.g. C++ and Rust both have some nice utilities for this stuff that are a bit hard to replicate in Julia, but it's not auch a huge difference, and there's also a lot of advantages on the julia side. E.g. it's really nice to have the GC available for the non-performance critical parts of your code.
- VorpalWay 8mo ago
  Since I don't know Julia, let me ask: how easy is it to add a use of the GC by mistake in a critical part of the code (maybe a junior dev does it)? Are there any tools to lint against that (some sort of function attribute to deny uses of GC in the function perhaps)? If it happens, how hard is it find the culript lines of code? Because from what I have seen in other GC languages, the answers to any of those questions haven't been great.
  - SatvikBeri 8mo ago
    You can use AllocCheck.jl to guarantee your code doesn't allocate. It's conservative, so it'll sometimes throw false positives, but shouldn't throw false negatives. You apply the checks to function definitions. You can profile memory usage line by line in detail pretty easily with tools like @timeit or @btime In practice I've found it pretty easy to get inner loops down to few or 0 allocations when needed for parallelization
  - dandanua 8mo ago
    It's easy to add, yes. However, the basic @time macro outputs the info about allocations by default. So it is also easy to see if there is a problem.
  - eigenspace 8mo ago
    Tooling for this in julia is evolving, but currently works fairly well. It could be better, but could be a lot worse. There's static analysis tools that might be over-eager finding allocations, and there's also runtime measurement tools which should go into your test suite if it's of vital importance to monitor.
pjmlp 8mo ago
Depends on which GC language, people keep forgeting many have C++ like capabilities, besides having a GC. D, C#, Swift, Nim,..... Agree that in Julia's case the flexiblity is not quite there, still much better than using Python and then going to write most of the work in C, C++, Fortran,..... Which is a thing that gets lost quite often in these discussions, just because the last 5% might be a bit harder, doesn't mean we have to throw everything away and start from scratch in another programming language, with its own set of problems.
dpc_01234 8mo ago
> In my experience, the most natural way to write things in Rust is usually the fastest (or close enough) as well. Well, a lot of C/Odin/Zig people will point out that Rust's stdlib encourages heap allocations. For actually best performance you typically want to store your data in some data-oriented data model, avoid allocations and so on, which is not exactly against idiomatic Rust, but more than just a typical straighforward Rust just throwing allocations around.
- VorpalWay 8mo ago
  Fair point, I don't tend to do much with heap in Rust (or any language) as I do hard realtime. You allocate all you will need up front. As for data oriented design: yes, and it depends on your usage pattern. E.g. arrays of structs can still be better than struct of arrays if that matches your cache line access pattern. Zig probably makes SOA easier for the cases where that is more efficient (I haven't used Zig, but I have read a bit about it. It has some cool ideas (comptime), but also things I can't get behind such as lack of RAII, and generally lack of memory safety.)