4 ms·
> almost always crystal clear and self explanatory and slow. Most of the time the idiomatic pandas answer takes advantage of its vectorization capabilities whi
by aulin 4y ago
> almost always crystal clear and self explanatory
and slow. Most of the time the idiomatic pandas answer takes advantage of its vectorization capabilities while plain python would require nested loops.
- bqmjjx0kac 4y agoThis is one of the advantages of languages with optimizing compilers. In Python, there's a tradeoff between clarity and performance. But in C++ for example, your for-loop is just as fast as any library's for-loop. Edit: My assumption is that many Python libraries are written in C for performance, but maybe that's not the case.
- behnamoh 4y agoExcept that your C loop can be WRITTEN slower than some library’s loop.
- kaba0 4y agoIn case of these libraries it is mostly the case of painstaking optimizations in low level code. No common language has sufficiently smart auto-vectorization to reliably convert for loops in most cases to SIMD instructions - so no, your average C++ for loop wouldn’t be that much faster, unless you spend a lot of time optimizing it for the specific task.
- andrepd 4y ago>But in C++ for example, your for-loop is just as fast as any library's for-loop. This is most definitely not true. Even stuff like subtle aliasing guarantees can make code run an order of magnitude faster or slower.
- gus_massa 4y agoAnother example is the order in which you access memory. To multiply two matrices, in one you move by row and in the other you move by columns. One makes the cache happy and the other unhappy, so it's faster to transpose one of them (the correct one, I never remember) and them "multiply" them. We notice this in Fortran with -O2, but the difference disappears in Fortran with -O3, so the compiler is doing some magic.