3 ms·

PyPy is really impressive, I love the idea of getting all these optimisations (SIMD, STM) for free... but as you say, numerical work means numpy, scipy, pandas,

PyPy is really impressive, I love the idea of getting all these optimisations (SIMD, STM) for free... but as you say, numerical work means numpy, scipy, pandas, which don't work with PyPy. Even if the NumPyPy project was able to fully match the Numpy api, you still have a lot of large projects like Pandas that depend on the c api. it would be stupid to copy everything. Perhaps something can be worked out between pypy, cffi, and cython. In the short term Numba is much more practical for numerics. In the longer term Pyston looks promising - it's actually similar to Numba in that it also uses LLVM, I imagine there could be synergy between the two...

sitkack 11y ago
NumPy is a protocol that dictates the layout of multidimensional arrays with really fast Fortran code that knows that layout. What needs to be copied is that memory layout protocol so that we get n:m sharing instead of n^2 duplication.
- mangecoeur 11y ago
  My point was that even if you copy the numpy protocol, you still have huge projects that depend on c-extensions that you wouldn't want to port. Pandas is one, others include scikit-learn, scikit-image, Astropy, Bioinformatics libraries, stats libraries, etc... which all have heavy C/Cython use and depend to varying degrees on the Python C-api. Porting NumPy barely scratches the surface of scientific python.
  - dalke 11y ago
    Biopython runs on pypy. It also runs under Jython. While it has C use, it is not "heavy" C use. Going on a tangent, and though I realize it's a lost battle, I wish people would stop saying that NumPy is the base of scientific programming in Python. As Biopython shows, it isn't required for at least some of bioinformatics. My own research[1] deals with chemical graphs, and NumPy/SciPy/etc. are nearly irrelevant to that research. [1] For example, given a set of 100 structures, what is the largest substructure (based on the number of bonds) which is in at least 90 of the structures?
Alphasite_ 11y ago
Last I heard there were efforts to support the c api in pypy: https://bitbucket.org/pypy/compatibility/wiki/c-api https://bitbucket.org/pypy/compatibility/wiki/c-api

sitkack 11y ago

NumPy is a protocol that dictates the layout of multidimensional arrays with really fast Fortran code that knows that layout. What needs to be copied is that memory layout protocol so that we get n:m sharing instead of n^2 duplication.

mangecoeur 11y ago
My point was that even if you copy the numpy protocol, you still have huge projects that depend on c-extensions that you wouldn't want to port. Pandas is one, others include scikit-learn, scikit-image, Astropy, Bioinformatics libraries, stats libraries, etc... which all have heavy C/Cython use and depend to varying degrees on the Python C-api. Porting NumPy barely scratches the surface of scientific python.
- dalke 11y ago
  Biopython runs on pypy. It also runs under Jython. While it has C use, it is not "heavy" C use. Going on a tangent, and though I realize it's a lost battle, I wish people would stop saying that NumPy is the base of scientific programming in Python. As Biopython shows, it isn't required for at least some of bioinformatics. My own research[1] deals with chemical graphs, and NumPy/SciPy/etc. are nearly irrelevant to that research. [1] For example, given a set of 100 structures, what is the largest substructure (based on the number of bonds) which is in at least 90 of the structures?

Alphasite_ 11y ago

Last I heard there were efforts to support the c api in pypy: https://bitbucket.org/pypy/compatibility/wiki/c-api https://bitbucket.org/pypy/compatibility/wiki/c-api