3 ms·
Numpy probably calls Fortran and C libraries under the hood. Is Fortran still considerably faster than Numpy?
by aborsy 3y ago
Numpy probably calls Fortran and C libraries under the hood.
Is Fortran still considerably faster than Numpy?
- bee_rider 3y agoThe overhead of a function call should be O(1) (I mean it must be, right?), and then most of the functions are probably O(N), where N is the size of the vector, at least. So, as long as most of the work is in the C or Fortran library, it should be about the same.
- adgjlsfhk1 3y agothe problem is that O(1) here can be hundreds of nanoseconds which has a pretty notable impact for arrays shorter than a few thousand
- chihuahua 3y agoBut constant factors matter. We don't expect there to be a difference in the big-O complexity between languages. If you have two O(N) implementations, but one is 10x faster than the other (or 1.5x or whatever), that's still a significant difference.
- ok123456 3y agoIt depends on how much conversion is needed to feed it into the FFI. Duck typing in python means there are a lot of implicit conversions that will take place.
- bee_rider 3y agoTrue. But if the code is reasonably well written, and you are just doing Numpy operations on Numpy vectors and matrices, it should basically be equivalent to doing those operations in Fortran (because most of the actual compute should be performed by code written in Fortran and C). They asked if Fortran was faster than Numpy, not if it was faster than Python.
- synergy20 3y agoI think yes, numpy uses fortran.
- abdullahkhalids 3y agoThe passage of data from python -> fotran -> python takes some time. Right now, I am converting some numpy code to rust for exactly this sort of speed advantage. I barely know any rust, and don't know the first thing about writing optimized rust code, but I am already getting constant factor improvements.
- ilayn 3y agoNo, NumPy has tools for Fortran inclusion, f2py, for the users but itself has no Fortran code. SciPy has Fortran77 code about 15% but slowly rewriting it in Cython/C++. Both use BLAS from OpenBLAS library externally, that is mostly Assembly/C for BLAS. NumPy use a flavor of LAPACK that is called LAPACK Lite, written in C, SciPy vendors LAPACK that comes with OpenBLAS if you are pip installing, or if you are using conda, it comes with MKL library from Intel.