6 ms·
I have a question and then a general vent 1. Does anyone know the latest update on NumPyPy? PyPy for me is just not a usable proposition because I heavily use
by boultonmark 9y ago
I have a question and then a general vent
1. Does anyone know the latest update on NumPyPy? PyPy for me is just not a usable proposition because I heavily use Numpy (and Scipy et al). So I am forced to use slow Python + fast Numpy or slow Numpy + fast Python. Very saddening. The C-Extension is just so off the pace, NumPyPy was meant to solve that quandry.
And I know some smart Alec will trot out the usual 'downshift into C' line that everyone (including Guido) use as the final goto solution for performance but that is simply a disgrace in 2017. Even JavaScript is fast. Why can I not choose to write Python and it be fast?? And yet Python 3 is getting slower. Don't agree?
Look at these benchmarks of Python heaps written in Python (not using the C based builtin heapq) https://github.com/MikeMirzayanov/binary-heap-benchmark https://github.com/MikeMirzayanov/binary-heap-benchmark Python generally is off the pace but Python 3 is about twice as slow as 2 and miles off JavaScript.
But PyPy is proof that Python can be fast. It makes quote/unquote "Pure Python" within striking distance of Go and and when I run that test suit on PyPy, its similar to the Node.js score. Why does this matter?
Because I want to write bloody Python not C.
And it is so tantalisingly close - look at a blog post like: https://dnshane.wordpress.com/2017/02/14/benchmarking-python-heaps/ https://dnshane.wordpress.com/2017/02/14/benchmarking-python... The performance of the Fibonacci Heap that someone wrote in quote/unquote "Pure Python", when run in CPython can never compete with HeapQ (the C based builtin lib), but on PyPy it can. Fast code written in Python. So what are the problems holding back PyPy? I think possibly money and number of devs working on stuff. Javascript had Mozilla, Google, Microsoft and Apple in a browser war + loads of open source input.
But is the biggest stumbling block not Guido himself and the core Python devs? Do they just philosophically not agree with PyPy or is it just disinterest?
Well whatever it is, it is heart-breaking to want to write fast code in my favourite language and leverage all its power including Numpy/Scipy etc and not be able to. And yes my use-case is perhaps quite unique, a very CPU intensive service that ideally computes and returns a real-time calculation (that includes 500k function calls) in 10-50ms.
But getting fast Numpy in the PyPy mix (i.e all the speed of the JIT + no worse Numpy) would be a HUGE step forward for me in PyPy adoption. What is the latest? How can I help?
- petters 9y agoI agree writing C extensions is not a solution. It is evidently too hard. Even the standard pickle library in Python 3 has a memory corruption bug. http://bugs.python.org/issue23655 http://bugs.python.org/issue23655
- mkj 9y agoI can't answer how numpypy is going, but Numba works pretty well for me to write fast numeric code in python. A bit restricted language and installing llvm is a bit of a hassle, but overall it's great.
- ant6n 9y agoThat's CPython and the reason you're not noticing its slow is probably because most of your time is spent computing inside compiled library functions.
- maxerickson 9y agoNumba is a Python compiler: http://numba.pydata.org/ http://numba.pydata.org/ So the fast functions they are using are likely compiled, they aren't necessarily library functions.
- mattbillenstein 9y agoI think the core devs are doing a fine job with CPython as the reference implementation and developing PyPy takes a different set of expertise; JIT and compilers specifically. I think sponsorship of PyPy would be welcome -- but it seems non-obvious where that would come from. Javascript has the fortune of being the language that drives a very important platform -- and Chrome has been a particularly strategic investment for Google to have more control over the web than it ever has before. Java has Android... Python unfortunately doesn't have that sort of standing in any area that I'm aware of.
- pjmlp 9y agoJava has much more than Android. Besides the enterprise space, there are lots of embedded devices running it.