5 ms·

The code backing things like Numpy and Scipy is not small, has been well optimised over many years, and would be extremely difficult to reimplement at speed eve

The code backing things like Numpy and Scipy is not small, has been well optimised over many years, and would be extremely difficult to reimplement at speed even with a faster Python. The freedom high level languages give tends to prevent many low level optimisations that are really useful for big maths and data routines. For the language I work on we were able to write macros and functions to allow our C libraries to continue to work when we changed our VM, but we've spent considerable effort optimising things so the C has to reach back into objects as little as possible, and we could only do this because recompilation and refactoring was okay.

ris 12y ago
"The code backing things like Numpy and Scipy is not small, has been well optimised over many years, and would be extremely difficult to reimplement at speed even with a faster Python." I assume you're unaware of the numpypy project then, which is doing exactly that, with reasonable results. I also think you overestimate the performance of numpy/scipy when compared to, for instance, C++ template-based libraries which are able to make large performance gains from compiling specializations for the specific array shapes being used. JITs can potentially go further, being able to compile specializations in places where array shapes are not known at compile-time.
- gaius 12y ago
  They are rewriting BLAS in Python? Seriously? Links please. You may not be aware that NumPy does it's heavy lifting by offloading the work to C and FORTRAN, and that's where the speed comes from. You set up your data in Python, use the nice high level language to parse files, etc, then you pass chunks of memory containing matrices off to the FORTRAN to crunch, then use Python again to say write it back to a file.
  - masklinn 12y ago
    > They are rewriting BLAS in Python? Nah. Note that these are optional even in the original numpy.
  - ris 12y ago
    It is yet to be seen whether they're going to defer to BLAS or try rewriting the routines in python for the linear algebra.
  - gnufx 12y ago
    Yes to the basic point, but note that (specifically) if you use BLAS in Fortran, you're typically losing a factor of several in speed, I'm sorry to say. Competitive serial numerical kernels typically still end up needing assembler (e.g. OpenBLAS, FFTW, ELPA). I'm not sure why Fortran wouldn't be considered a high level language, if that's the implication (and it hasn't been SHOUTED since the 70s).

ris 12y ago

"The code backing things like Numpy and Scipy is not small, has been well optimised over many years, and would be extremely difficult to reimplement at speed even with a faster Python." I assume you're unaware of the numpypy project then, which is doing exactly that, with reasonable results. I also think you overestimate the performance of numpy/scipy when compared to, for instance, C++ template-based libraries which are able to make large performance gains from compiling specializations for the specific array shapes being used. JITs can potentially go further, being able to compile specializations in places where array shapes are not known at compile-time.

gaius 12y ago
They are rewriting BLAS in Python? Seriously? Links please. You may not be aware that NumPy does it's heavy lifting by offloading the work to C and FORTRAN, and that's where the speed comes from. You set up your data in Python, use the nice high level language to parse files, etc, then you pass chunks of memory containing matrices off to the FORTRAN to crunch, then use Python again to say write it back to a file.
- masklinn 12y ago
  > They are rewriting BLAS in Python? Nah. Note that these are optional even in the original numpy.
- ris 12y ago
  It is yet to be seen whether they're going to defer to BLAS or try rewriting the routines in python for the linear algebra.
- gnufx 12y ago
  Yes to the basic point, but note that (specifically) if you use BLAS in Fortran, you're typically losing a factor of several in speed, I'm sorry to say. Competitive serial numerical kernels typically still end up needing assembler (e.g. OpenBLAS, FFTW, ELPA). I'm not sure why Fortran wouldn't be considered a high level language, if that's the implication (and it hasn't been SHOUTED since the 70s).