3 ms·

I am a researcher at an algo trading shop. We are moving our core libraries from rust+python to Julia. It really is amazingly powerful and has great interop wi

I am a researcher at an algo trading shop. We are moving our core libraries from rust+python to Julia. It really is amazingly powerful and has great interop with python (eg seamless zero copy array sharing). In the past one had to muck about with cython/c/numpy api to speed things up, now one can just write the functionality in Julia and make it available in both ecosystems. Python will likely remain the premier language to do research in, but more and more work can move to Julia which is much better at anything not immediately vectorizable.

socialdemocrat 7y ago
Great news for me! I actually do Julia training. My colleagues do Python training. But we don't seem to be quit at the critical mass yet to gain a lot of training requests for Julia. But I am an optimist. There seems to be a shift going on. 2020 could be a breakout year for Julia. A lot of my job the way I see it at the moment is to try to learn about people's experience switching to Julia to help explain to people why it would benefit them. How did you guy actually end up switching to Julia? Was it a careful analysis ordered from the top, or was there more like some Julia evangelists who bugged people until they tried it an realized it was actually quite useful? I used to work in the oil industry, and tried to convince people to try Julia. It would have been a huge advantage over our Python interface in terms of performance but it was a very hard sell. People are conservative. They are very reluctant to try new things. So I am always curious how other people pull off making a change happen.
- short_sells_poo 7y ago
  Julia was brought in largely organically. We often need performant non-vectorizable code. Numba is actually a real pain to work with if the code is not explicitly numeric only. It works well for optimizing a hot loop here and there, but anything more complex becomes an exercise in frustration. So we started experimenting with Julia and it was literally like the best of both worlds. Compile times can still be a struggle sometimes, but we are happy to eat a few secs startup cost when doing research, as most of the time we'd just have an ipython/julia repl going and keep the session open for hours/days. Most people (me included) weren't prepared to pay the mental tax of writing Rust on potentially throwaway experimental code, so as we worked we realized that the core Rust libs can be easily replaced with much simpler Julia code without any loss in performance. While the PyO3 library is awesome, it was actually quite difficult to reconcile the safety of Rust with the dynamism of Python and the friction can be really felt in any code dealing with the interop. This is mainly on the Rust side, as it ended up being littered with casting and type checks when communicating with Python. Rust being compiled AOT, a lot of the generics power goes out the window in the interop purely because it is impossible to know at compile time the type of objects coming from python. This has negative implications for performance, because the rust code can't be neatly specialized either, but has to resort to dynamic dispatch and trait objects. Julia wins here due to the JIT compiler that auto-specializes code at run time when the types are known.
  - jhayward 7y ago
    I really appreciate folks like the parent here who take the time to highlight specific details like these that arose from their experience with different languages and paradigms. These are the details that matter.

socialdemocrat 7y ago

Great news for me! I actually do Julia training. My colleagues do Python training. But we don't seem to be quit at the critical mass yet to gain a lot of training requests for Julia. But I am an optimist. There seems to be a shift going on. 2020 could be a breakout year for Julia. A lot of my job the way I see it at the moment is to try to learn about people's experience switching to Julia to help explain to people why it would benefit them. How did you guy actually end up switching to Julia? Was it a careful analysis ordered from the top, or was there more like some Julia evangelists who bugged people until they tried it an realized it was actually quite useful? I used to work in the oil industry, and tried to convince people to try Julia. It would have been a huge advantage over our Python interface in terms of performance but it was a very hard sell. People are conservative. They are very reluctant to try new things. So I am always curious how other people pull off making a change happen.

short_sells_poo 7y ago
Julia was brought in largely organically. We often need performant non-vectorizable code. Numba is actually a real pain to work with if the code is not explicitly numeric only. It works well for optimizing a hot loop here and there, but anything more complex becomes an exercise in frustration. So we started experimenting with Julia and it was literally like the best of both worlds. Compile times can still be a struggle sometimes, but we are happy to eat a few secs startup cost when doing research, as most of the time we'd just have an ipython/julia repl going and keep the session open for hours/days. Most people (me included) weren't prepared to pay the mental tax of writing Rust on potentially throwaway experimental code, so as we worked we realized that the core Rust libs can be easily replaced with much simpler Julia code without any loss in performance. While the PyO3 library is awesome, it was actually quite difficult to reconcile the safety of Rust with the dynamism of Python and the friction can be really felt in any code dealing with the interop. This is mainly on the Rust side, as it ended up being littered with casting and type checks when communicating with Python. Rust being compiled AOT, a lot of the generics power goes out the window in the interop purely because it is impossible to know at compile time the type of objects coming from python. This has negative implications for performance, because the rust code can't be neatly specialized either, but has to resort to dynamic dispatch and trait objects. Julia wins here due to the JIT compiler that auto-specializes code at run time when the types are known.
- jhayward 7y ago
  I really appreciate folks like the parent here who take the time to highlight specific details like these that arose from their experience with different languages and paradigms. These are the details that matter.