3 ms·
Julia was brought in largely organically. We often need performant non-vectorizable code. Numba is actually a real pain to work with if the code is not explicit
by short_sells_poo 7y ago
Julia was brought in largely organically. We often need performant non-vectorizable code. Numba is actually a real pain to work with if the code is not explicitly numeric only. It works well for optimizing a hot loop here and there, but anything more complex becomes an exercise in frustration.
So we started experimenting with Julia and it was literally like the best of both worlds. Compile times can still be a struggle sometimes, but we are happy to eat a few secs startup cost when doing research, as most of the time we'd just have an ipython/julia repl going and keep the session open for hours/days.
Most people (me included) weren't prepared to pay the mental tax of writing Rust on potentially throwaway experimental code, so as we worked we realized that the core Rust libs can be easily replaced with much simpler Julia code without any loss in performance.
While the PyO3 library is awesome, it was actually quite difficult to reconcile the safety of Rust with the dynamism of Python and the friction can be really felt in any code dealing with the interop. This is mainly on the Rust side, as it ended up being littered with casting and type checks when communicating with Python. Rust being compiled AOT, a lot of the generics power goes out the window in the interop purely because it is impossible to know at compile time the type of objects coming from python. This has negative implications for performance, because the rust code can't be neatly specialized either, but has to resort to dynamic dispatch and trait objects. Julia wins here due to the JIT compiler that auto-specializes code at run time when the types are known.
- jhayward 7y agoI really appreciate folks like the parent here who take the time to highlight specific details like these that arose from their experience with different languages and paradigms. These are the details that matter.