2 ms·
A slight gap in knowledge in the piece: Most Python libraries for numerical computation are written in C/C++ or... Fortran. Last time I had to compile scipy fro
by rcarmo 9y ago
A slight gap in knowledge in the piece: Most Python libraries for numerical computation are written in C/C++ or... Fortran. Last time I had to compile scipy from scratch I had to install gfortran.
- nine_k 9y agoSome of the Numpy code is coming from 1970s: it uses BLAS that originates from 1979. https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprogra... Immense amount of effort was put in performance, correctness, feature set, and numerical stability of such widely used libraries. Replacing the without a very good reason is hardly feasible.
- quadruplebond 9y agoBlas is an interface, All performant blas libraries need to written to specifically take advantage of modern hardware. Things like NetLib exists as reference implementations, they are not high performance. The point I am trying to make is that Numpy is unlikely to still be using the exact same code that was written in the 1970's unless its performance isn't critical. It is making the same function calls but the actual code will look pretty different.
- pletnes 9y agoNumpy can use optimised BLAS but I believe it ships with an f2c transpiled reference BLAS so it can run (although slower) without Fortran where necessary.
- quadruplebond 9y agoSure but that will likely be much much slower.
- cm2187 9y agoI needed a .net math library. My IT department insisted on my trying a wrapper around a Fortran library (I believe purely because they had a license already). All the classes and variables names are short hexadecimals strings. None of the interface is idiomatic to .net (the library only has void returning methods which pass error codes as byref arguments instead of using exceptions, requesting a parameter for the length of any array passed as argument instead of reading it from the array), etc... Basically I refused to touch this thing, using a library which makes the code unreadable is going to be a bug magnet. I would be surprised if I was the only one having this reaction to 1960s coding convention surfacing in modern code. So selling new licenses should be a good enough reason.
- ryl00 9y agoSounds like your IT department botched the wrapper. Wouldn't a more successful candidate likely just have used an abstraction layer between the .net conventions you were expecting and whatever the native code was doing?
- cm2187 9y agoNope, the library itself is sold that way. See the documentation: https://www.nag.co.uk/numeric/DT/nagdotnet_dtw02/html/contents.html https://www.nag.co.uk/numeric/DT/nagdotnet_dtw02/html/conten...
- ryl00 9y agoAh I see, makes more sense. It would be a massive undertaking to abstract that API into something else.
- krallja 9y agoYour .NET wrapper doesn't need to expose the same name or calling convention. That's why you use a wrapper: to abstract away distasteful non-idiomatic decisions in the wrapped API.
- eesmith 9y agoSome from the 1960s. Quoting numpy/lib/function_base.py: We use the algorithm published by Clenshaw [1]_ and referenced by Abramowitz and Stegun [2]_, for which the function domain is ... .. [1] C. W. Clenshaw, "Chebyshev series for mathematical functions", in *National Physical Laboratory Mathematical Tables*, vol. 5, London: Her Majesty's Stationery Office, 1962. .. [2] M. Abramowitz and I. A. Stegun, *Handbook of Mathematical Functions*, 10th printing, New York: Dover, 1964, pp. 379. http://www.math.sfu.ca/~cbm/aands/page_379.htm Quoting numpy/core/src/npymath/npy_math_complex.c.src: /* Algorithm 312, CACM vol 10, Oct 1967. */ if (a >= 0) { t = npy_sqrt@c@((a + npy_hypot@c@(a, b)) * 0.5@c@); result = npy_cpack@c@(t, b / (2 * t)); }
- fiftyacorn 9y agoYeah i remember that too - there is a answer in the scipy FAQ - https://www.scipy.org/scipylib/faq.html#how-can-scipy-be-fast-if-it-is-written-in-an-interpreted-language-like-python https://www.scipy.org/scipylib/faq.html#how-can-scipy-be-fas... "Actually, the time-critical loops are usually implemented in C or Fortran. "