3 ms·
I've run into some pretty straight-forward problems (eg: modeling physical systems, with linear algebra and some other basic math) where avoiding python for loo
by ssivark 3y ago
I've run into some pretty straight-forward problems (eg: modeling physical systems, with linear algebra and some other basic math) where avoiding python for loops meant allocating memory up front for massive numpy, and then doing multiple separate vectorized operations on them rather than fusing all of them into a single for loop. Required several hundred GB of RAM and took a few hours IIRC. This was of course still better than Python for loops... /facepalm
I wrote the same thing in Julia in about the same number of lines of code -- avoiding the sins above -- and I could run it on my laptop in a few seconds.
- geysersam 3y agoFor stuff like that I think Numba really shines. But yes I get the issue. Julia is great, but sometimes a bit less user friendly than Python, it's quite easy to shoot oneself in the foot performance wise using Julia.
- ssivark 3y agoI hate the fact that if I choose Numba, then I can’t easily compose with other tools like autodiff, etc. Maybe Jax improves the situation in some aspects. Generally though… While Python might have many distinct popular/well-developed tools, they tend to compose very badly. IME, it feels relatively painful if you’re trying to do something new which hasn’t been solved and wrapped for you. As for Julia footguns, it has been relatively trivial IME to get substantial improvements compared to Python. Whether that is the best one can do, I dunno.