4 ms·

I will attempt to, but do let me know if this is not the type of examples you meant: Julia idioms: Multimethods and multiple dispatch instead of python/java st

I will attempt to, but do let me know if this is not the type of examples you meant: Julia idioms: Multimethods and multiple dispatch instead of python/java style OOP; Loops are fast, with zero-cost-abstractions and automatic SIMD, unlike python's need for vectorized libraries; "type-inferability" is incredibly important for good performance, which is quite different from python's duck typing (but the language is dynamic, so it is still an option); measuring how often your code allocates is the most trivial way to increase its performance by an order of magnitude. Julia workflow: you *have* to change your workflow, compared to coming from python, otherwise the latency of the compiler would cause you deep enraging frustration: e.g. use Revise.jl/Pluto.jl/REPL, organize your code so that small pieces can be recompiled without having to restart your session; keep your workspace clean and your computational pipeline organized, so that reproducibility testing is easy.

p1esk 4y ago
I'm curious what made you more productive. Was it not having to use those vectorized libraries? I write machine learning code, so anything compute intensive is done with specialized libraries and the Python glue code usually has little overhead. Sometimes when writing complicated pipelines I spend a lot of effort on optimizing data transfers (between CPU and GPUs, between GPUs on the same server, between GPUs on different servers in a cluster, etc), but that effort has little to do with Python limitations (or so I think).
- krastanov 4y ago
  A couple of things that made a difference for me: - I actually like vectorized code, I find it easier to read, but especially in python, vectorized code still has some fixed overhead that can be prohibitive for smallish vectors. - Some of the numerical code I run is not easy to batch, so tools like tensorflow are cumbersome. With Julia it is easy to make fast non-allocating code while writing in easy to read style like in Python. In Python you need to use C/Cython/Numba to achieve this, but then you lose interoperability with other Python packages. E.g. auto-diff through fast cython code does not work, while autodiff through my equally fast non-batched Julia code is trivial. - Ecosystem: I work with differential equations a lot. Julia's SciML library (and its DifferentialEquations.jl sub-library) is light years ahead compared to any other tool in Python/R/Mathematica/Matlab/Maple. Speaking of python, solving differential equations in python feels like I am using technology that is quarter of a century old. - In python I might write a nice and legible slowish implementation of something and then piece by piece I will start replacing it with cython. That requires a lot of boilerplate. In Julia I just write the slowish implementation with the same ease with which I would do it in python, but then I need to perform extremely minimal profile-guided changes with zero boilerplate to achieve the cython/numba type of performance. And again, I do not lose interoperability with other parts of the ecosystem when I do that. - It is trivial to inspect the actual machine code that was generated for every single function you write. It is trivial to inspect exactly where allocations happen. Most of the ecosystem is in pure Julia, so it is trivial to edit fairly foundational pieces of the ecosystem and submit patches. On the other hand, the levels of abstraction in something like Jax/Tensorflow/Numba makes it relatively difficult to contribute. - It is wild how much code reuse you can have with the multiple dispatch paradigm. Say I find someone's pet project on performing a complicated linear algebra algorithm. That code was probably written just to work with Float64, but thanks to multiple dispatch the compiler can generate specializations for esoteric higher-precision real number representations, or real numbers with error bars, or unitful numbers. I still need to do some sanity checks, because hidden assumptions might be problematic, but this level of interoperability is possible only with multiple dispatch (thinking of it as one of the "standard solutions" for the Expression Problem[1] really helped; [2] was also informative). I think the GPU libraries for Julia are a good example of multiple dispatch working well in that sense. [1]: https://en.wikipedia.org/wiki/Expression_problem https://en.wikipedia.org/wiki/Expression_problem [2]: https://www.youtube.com/watch?v=kc9HwsxE1OY https://www.youtube.com/watch?v=kc9HwsxE1OY

p1esk 4y ago

I'm curious what made you more productive. Was it not having to use those vectorized libraries? I write machine learning code, so anything compute intensive is done with specialized libraries and the Python glue code usually has little overhead. Sometimes when writing complicated pipelines I spend a lot of effort on optimizing data transfers (between CPU and GPUs, between GPUs on the same server, between GPUs on different servers in a cluster, etc), but that effort has little to do with Python limitations (or so I think).

krastanov 4y ago
A couple of things that made a difference for me: - I actually like vectorized code, I find it easier to read, but especially in python, vectorized code still has some fixed overhead that can be prohibitive for smallish vectors. - Some of the numerical code I run is not easy to batch, so tools like tensorflow are cumbersome. With Julia it is easy to make fast non-allocating code while writing in easy to read style like in Python. In Python you need to use C/Cython/Numba to achieve this, but then you lose interoperability with other Python packages. E.g. auto-diff through fast cython code does not work, while autodiff through my equally fast non-batched Julia code is trivial. - Ecosystem: I work with differential equations a lot. Julia's SciML library (and its DifferentialEquations.jl sub-library) is light years ahead compared to any other tool in Python/R/Mathematica/Matlab/Maple. Speaking of python, solving differential equations in python feels like I am using technology that is quarter of a century old. - In python I might write a nice and legible slowish implementation of something and then piece by piece I will start replacing it with cython. That requires a lot of boilerplate. In Julia I just write the slowish implementation with the same ease with which I would do it in python, but then I need to perform extremely minimal profile-guided changes with zero boilerplate to achieve the cython/numba type of performance. And again, I do not lose interoperability with other parts of the ecosystem when I do that. - It is trivial to inspect the actual machine code that was generated for every single function you write. It is trivial to inspect exactly where allocations happen. Most of the ecosystem is in pure Julia, so it is trivial to edit fairly foundational pieces of the ecosystem and submit patches. On the other hand, the levels of abstraction in something like Jax/Tensorflow/Numba makes it relatively difficult to contribute. - It is wild how much code reuse you can have with the multiple dispatch paradigm. Say I find someone's pet project on performing a complicated linear algebra algorithm. That code was probably written just to work with Float64, but thanks to multiple dispatch the compiler can generate specializations for esoteric higher-precision real number representations, or real numbers with error bars, or unitful numbers. I still need to do some sanity checks, because hidden assumptions might be problematic, but this level of interoperability is possible only with multiple dispatch (thinking of it as one of the "standard solutions" for the Expression Problem[1] really helped; [2] was also informative). I think the GPU libraries for Julia are a good example of multiple dispatch working well in that sense. [1]: https://en.wikipedia.org/wiki/Expression_problem https://en.wikipedia.org/wiki/Expression_problem [2]: https://www.youtube.com/watch?v=kc9HwsxE1OY https://www.youtube.com/watch?v=kc9HwsxE1OY