10 ms·
PyPy.js: Now faster than CPython
- cft 12y agoI felt it was a grave mistake for Python to direct the community effort into backwards incomparable Python 3, rather than on focusing on the speed (and better profiling instrumentation) of Python 2 (perhaps adding a standard JIT compiler). As a result, had our company started now, we would have written our Twisted server in C or Go instead of Python like we did.
- rdtsc 12y agoSadly I agree. Looking back Python 3 came too late, didn't offer much in terms of an incentive to justify switching to it, divided the community and in the process also left other things people worry about by the wayside -- performance, packaging, better concurrency handling (no not via Tulip or Twisted). I think, like you point out, many before they decided to sink the time into upgrading to Python 3 to get better unicode support for example, would also take the time to evaluate and go with something else altogether -- Go for example.
- pjmlp 12y agoI still don't get the point of using Go instead of PyPy or Cython.
- rdtsc 12y agoThe point was that once people are looking at restructuring the code base and re-writing parts of it, it also becomes a point of evaluation other technologies and frameworks. So there is a high chance many will not switch Python 2 -> Python 3 but Python 2 -> Go or something else. Then well ok, the answer to "What did Python 3 bring to the Python community?" becomes "Yeah it ended up driving a lot of people away", which I am sure wasn't the intended purpose of Python 3.0
- pjmlp 12y agoSo switching Python 2 -> Python 3 is a higher cost than Python 2 -> Go?!?
- rdtsc 12y agoYes if you add in amortized benefit of extra performance and static type checking during compilation. benefit(2->3 + performance benefit + maintainability) < benefit(2->Go + performance benefit + maintainability) The time cost of 2->3 will be less but if I know I will get better memory footprint, and faster response times I might be willing to invest some more time into it. Or another way to look at it, once one sits down and rolls their sleeves looking to re-factor/renew/refresh the code base there is a good chance they will look around them and see what else is out there.
- berkut 12y agoOr something else, yes. If the cost is potentially going to be big enough to convert for a large code base (keeping in mind testing might be a huge part of it - having to potentially re-test everything), and for very little benefit (well, unicode possibly, but it's not like Python 3's really any faster, and the GIL's still there), it makes you evaluate if it's worth enduring a bit more transition pain to convert to something else completely which would bring the possibility of new features / better speed. It's also quite an issue for COTS software - sometimes to jump forwards and clean up APIs they have to do a huge refactor, sometime even a re-write of the API. If this is troublesome enough, it's worth taking the time to look into the competition.
- dmbaggett 12y agoCython is really impressive, but it produces incredibly huge C files, which then compile into enormous binaries. So it's not terribly suitable when code size matters.
- pjmlp 12y agoSure because Go binaries produced by Go static linking are small... https://code.google.com/p/go/issues/detail?id=6853 https://code.google.com/p/go/issues/detail?id=6853
- stcredzero 12y agoClean and efficient concurrency primitives, greater memory efficiency, and the ability to finely control memory layout, combined with just enough "high level" in the language to write complex systems. For that, you give up some (but not too much) of the rapidity of dynamic typing, and you don't have the marvelous ecosystem of libraries Python has built up. (yet)
- pjmlp 12y agoYou can get that with PyPy and Cython.
- stefantalpalaru 12y agoI like Cython, but how do you do CSP (Communicating Sequential Processes) with it?
- pjmlp 12y agoWith PyCSP, as one possible example https://code.google.com/p/pycsp/ https://code.google.com/p/pycsp/
- rohansingh 12y agoClean and efficient concurrency primitives? I think not? I would be very interested if that is indeed the case.
- a13xnet 12y agoStackless Python has very clean and efficient concurency primitives, something lot of people dont seem to know..
- ggchappell 12y ago> Stackless Python has very clean and efficient concurency primitives, .... Sounds like a good topic for a post (or at least a link - hint, hint).
- pecanpieyw 12y agoBecause Go is at least as fast as PyPy or Cython, have a similar coding feeling to Python, have language level concurrency support, and has a clear/consistent development roadmap moving forward. PyPy is not really a good example here. It is still highly experimental, takes long time to compile, it's not compatible with quite a few libraries in normal Python, and still has all relative downsides of dynamic typing. To be frank, I don't get the point of using PyPy instead of Go. Previously most Python libraries already use C to optimize critical components. If you can use just one language to get similar performance, why do you bother using two then?
- pjmlp 12y agoThe issue is not about writing clean new programs in Go, but rather re-writing whole code bases from Python into Go, instead of trying out Python implementations that offer better performance than CPython.
- fijal 12y agoDid you consider PyPy? It kinda works and it kinda focuses effort on being a faster replacement.
- rdtsc 12y agoPyPy is good. I wish there was more moment and more _pull_ for it from the main Python team. I feel the big push for the last 3-4 years should have been for PyPy (or performance and tooling in general). PyPy is good. I wish there was more moment and more _pull_ for it from the main Python team. I feel the big push for the last 3-4 years should have been for PyPy (or performance and tooling in general). EDIT: To expand, I remember feeling so excited during the PyCon-s in Chicago and then Atlanta. Unladen Swallow was making some progress but PyPy was fantastic, great speed improvements. Everyone looked in awe at you guys, and your presentations. We talked about STM and all the great things. And then...Guido announced in a rather annoyed tone how PyPy will never be a part of main Python distro and that's that. "Stop dreaming folks". And that made me pretty disappointed. Then the drumbeat for Python 3.0 started. Unicode, unicode, unicode... it got louder and louder. The wall of shame for "incompatible" libraries that were not being ported to Python 3.0. Lots of how to migrate and to Python 3 talks. Lots of oh look at the great view iterators. Things that would make me get off my seat and start porting to Python 3 would be: greenlet integration for low memory, concurrency green threads (gevent and eventlet). Great CPU speed improvement with included PyPy. Notice what is not on the list -- unicode, view iterators and renaming Queue module to queue. Sorry if I am being too sarcastic here, it is because I am bitter about it. It is also important to realize that Python 2.0 is just very good that it becomes hard for Python 3.0 to beat it. There were not big hairy warts that just bothered me about it. So it isn't that Python 3.0 is bad, it is not, it is just not better enough Python 2.0.
- rectangletangle 12y agoI think you're a little hard on Guido. CPython had amassed quite a few C extensions by that point. PyPy changes the low-level API in numerous significant ways. If Guido were to merge CPython with PyPy it would break all of those extensions. These extensions are in no small part the reason why Python is so popular. If he were to break all of this work, he'd piss off quite a few people. Many of the same people that helped him make Python what it is today. Now I agree with you having PyPy's performance improvements in the main implementation would be awesome, and PyPy itself is quite impressive. If they could get both NumPy running on PyPy and support Python 3's changes, it would keep Python a strong competitor for the foreseeable future. And to your last point, I entirely agree Python 2.0 had so many strengths that it's really hard to improve upon it. Though I think that it's unicode support was the big hairy wart.
- 1ris 12y agoI feel quite the other way round. They should have cleaned up their language before so many tools where developed on it.
- frik 12y agoMeanwhile, refactored PHP makes Wordpress 20% faster. http://news.php.net/php.internals/73888 http://news.php.net/php.internals/73888 , https://wiki.php.net/phpng https://wiki.php.net/phpng and https://news.ycombinator.com/item?id=7699322 https://news.ycombinator.com/item?id=7699322 HHVM and now PHP 5.7-dev, both with JIT - the future of PHP looks bright. Starting a new project, we would choose PHP, Node.js or Go according to the requirements.
- deleted 12y ago[deleted]
- frik 12y agoPlease read the page again (search for "JIT") and follow the link for more info.
- mantrax5 12y agoYeah, so refactored PHP makes a PHP app 20% faster. But you might actually not like to hear Node's V8 engine is between 50 and 150 times faster than PHP in synthetic benchmarks, and about 20 times faster in more diversified applications (as long as they don't sit an eternity blocking waiting on a DB call which is the way people did web apps 10 years ago). That's 2000% faster than PHP, just so we use the same units here. I'm very happy PHP is getting faster and all, but just don't juxtapose it next to JS engines (which the article talks about), let alone something more serious like the JVM, because the "bright future" of PHP starts looking quite dim in comparison.
- frik 12y agoIt depends, for things like Web Sockets Node.js is really good. But for other things, HHVM (Facebook's HipHop JIT) and the new PHP 5.7 dev with JIT are really fast. Check out an independent benchmark for comparision: http://www.techempower.com/benchmarks/#section=data-r9&hw=i7&test=update http://www.techempower.com/benchmarks/#section=data-r9&hw=i7...
- tdees40 12y agoI just ran the same test and got 80.8K pystones/sec (CPython) vs. 234.3K/sec (PyPy.js - Chrome, after a few warm-ups), so yeah, wow.
- zo1 12y agoAm I the only one that considers the title of this very "bait"-y? I mean, if you go on and read the article, it clearly shows that native PyPy is leaps and bounds faster than the JS version. So in essence, it re-iterates what most of us already know... That the PyPy implementation is much faster than the standard CPython one. It's just now we get to "prove" it in JS as well, as if it's some "suck it, python users" flaunt. On a side note: To the PyPy.js authors/devs, great job! Looking forward to playing around with this.
- PetitPrince 12y agoAccording to the author, it is deliberately bait-y: OK OK, I couldn't resist that title but it probably goes a bit far. Let me try for a little more nuance: PyPy.js: Now faster than CPython, on a single carefully-tuned benchmark, after JIT warmup.
- jkrems 12y agoYes, it is baity and the article clearly admits to it. But it also clearly has PyPy and CPython in the title which is important. I don't read as "suck it python users" but as "great news, python users: if your code runs okay in CPython today, you could be running it in the browser with decent performance soon!". If anything it's a "suck it JS, we can now run real languages in the browser!"
- DanWaterworth 12y agoI don't think the article is saying: "pypy.js is really fast, why not use it instead of CPython?". It's saying: "Consider using pypy.js on the client, it's comparable in performance to native CPython; you consider that fast enough."
- knackernews 12y agoSo for how long will the python community pretend that performance isn't a problem? (Inb4 "all the intensive parts of my app are written in C", addendum: "my app isn't CPU bound", "we have libraries written in C") The interpreters (esp CPython) need some hardcore engineering (like how javascript got V8). Languages that have the same theoretical performance limits of Python are now blowing it out of the water. The stories about teams that were able to go down from N servers to 1 server by switching from Python to Go/Scala/node.js will hurt it in the long run.
- TillE 12y agoLots of people use Python for tasks that are not CPU-bound. Scalable web-whatever and scientific computing aren't the whole world.
- apendleton 12y agoEven scalable web-whatever is often IO-bound... of my several apps in production, there's only one where CPU is a serious concern, and I run that one on PyPy. cPython is fine for the others.
- stcredzero 12y agoI'll go so far as to say that we haven't done enough with making things fast on multicore processors except with IO-bound tasks. The current architecture of multicore processors stinks for achieving anything but the "embarrassingly parallel."
- th3iedkid 12y ago>>The stories about teams that were able to go down from N servers to 1 server by switching from Python to Go/Scala/node.js will hurt it in the long run. concurrency designs and architecture play a greater role than the language , most of the times. Bottle-necks are often in bad-sync patterns , data-structures or overall designs. Language plays the role for semantic verification,formal models (for model-comparison...) ,type-system designs , proof systems etc .GIL is just one! Can't compare language vs another. After which the buck passes on to underlying implementation but cpython in our discussion is only thrown as a reference implementation and for production grade interpreters we might have to look outside like one by enthought. However language's built-in patterns to help advertise concurrency is another question altogether.
- panarky 12y agoI get 157K/sec in the terminal and 309K/sec in Chrome. As the article says, Chrome's performance shows wide variation, even after JIT warmup. It's ranging from 50K/sec to 320K/sec. Bizarre.
- azakai 12y agoJITs are great but can be unpredictable, that's probably the cause. As mentioned in the article, JITs vs AOT have upsides and downsides.
- steveklabnik 12y agoDoesn't Chrome specifically _not_ optimize asm.js, as well? Wonder if that has anything to do with it.
- zurn 12y agoIt doesn't special case the asm.js JS subset same way as Firefox does, but V8 guys are paying attention to optimizing asm.js-style code.
- coldcode 12y agoDoes this work at all in Safari? Is Safari's crappy performance of native->js ever going to catch up to FF and Chrome?
- d0mine 12y agotest.pystone.main() pypyjs on ios safari gives -- 9K/s (50K/s on Python for ios) Chrome on desktop -- 200K/s pypy on desktop -- 2000K/s
- azakai 12y agoYes, Safari has been working on an LLVM-based JIT called FTL. It now outperforms v8 on almost all asm.js benchmarks, http://arewefastyet.com/#machine=12&view=breakdown&suite=asmjs-apps http://arewefastyet.com/#machine=12&view=breakdown&suite=asm... That's not in release versions of Safari yet, it was just enabled on trunk. But it will make Safari very competitive once more.
- stcredzero 12y agoWhat if LLVM based JITs with NaCL style security were available in all browsers except IE? (I'm just assuming Microsoft will drag their heels.) Then an LLVM IR based bytecode could become a standard bytecode for browsers.
- hajile 12y agoI believe that in many ways, this would be ideal. Not only could any language be used (the implementers being responsible for interfacing with the browser API), but the LLVM pseudo-asm could be pre-optimized saving the browser a lot of work. In the meantime, browsers would still be free to compete based on their JIT implementations. With rust and by proxy, Mozilla/Samsung's experimental Servo layout engine targeting the LLVM, I can also see potential synergy in this situation.
- frik 12y agoApple's webkit development has slowed down, as it seems. Only two blog entries in the last half year: https://www.webkit.org/blog/ https://www.webkit.org/blog/ What a petty, it seems Google's forking of Webkit (named Blink) has hurt Webkit. Google's behavior of removing "CSS regions" was an unacceptable move.
- vanderZwan 12y agoI am colourblind and have problems with your choice of red and green for graphs.
- icebraining 12y agoNot that it shouldn't be fixed, but have you tried http://www.ryobi-sol.co.jp/visolve/en/ http://www.ryobi-sol.co.jp/visolve/en/ ?
- vanderZwan 12y agoI have on Windows, didn't really "stick". But I'll try it out on mobile though, might be more useful as a form of augmented reality, thanks!
- rfk 12y agoI had not considered this, it's currently just using whatever default colours matplotlib selected. Thanks for the heads-up!
- vanderZwan 12y agoThanks for the fix, it is greatly appreciated!
- bayesianhorse 12y agoPython in the browser so far is a big disappointment. I still find it a lot easier to just split frontend and backend development. In the future, PyPy.js might become a viable alternative for big browser applications.
- omaranto 12y agoIt's not faster on my computer, a 32bit netbook with an Intel Atom N450 chip (all scores average of five warm runs): CPython 2.7.5: 27032 PyPy.js: 20103 PyPy 2.0.2: 444011 I ran PyPy.js on Firefox 29.
- garindra 12y agoIn the browser environment, people usually use quite a few inline anonymous functions. Javascript is great at this since you can define full-blown anonymous functions anywhere. Python, on the other hand, is definitely one of the worst at this. You can only create one-line, single-expression anonymous function with its lambda keyword. Of all the languages you want to convert to JS so that it runs on the browser, Python is probably the last.
- zo1 12y agoI really don't understand this obsession with anonymous functions. If you're doing anything longer than one line, there is absolutely no reason why you should be sticking the entire function definition as a parameter. I've had great usage from Javascript where I have separate functions with proper names and formatting, and then pass a reference to them into the code that asks for them. Can someone please enlighten me about this whole anonymous functions thing that Javascript seems to have popularized? What are some of the pro's of using it over regular named/defined functions that sit by themselves and are reusable.
- hajile 12y agoI suggest you read SICP and gain an appreciation for higher-order functions. Once you do, you will probably have answered your own question. http://mitpress.mit.edu/sicp/ http://mitpress.mit.edu/sicp/
- inportb 12y agoWhy would zo1 want to read SICP? zo1 is clearly quite happy with Python's named functions, and has no immediate incentive to "read SICP and gain an appreciation for higher-order functions."
- hajile 12y agoI either assume that the parent poster was serious about the question or rhetorical. Given the apparent lack of knowledge, I assume the question was serious. If the question was serious, does it not deserve a serious response? Faced with squeezing an explanation into a limited post or recommending one of the best books on computer science (written by teachers with many years experience in teaching these self-same principles), should I not recommend the book that will allow the poster to expand his/her horizons? The poster was sufficiently motivated to ask the question. I was willing to post what I believed at the time to be the best answer to the question. If the poster's desire to know is sufficient, then he/she will take the time to explore the suggestion. If not, then things will stay as they are. In either case, my only interaction is my choice to share a suggestion or remain silent.
- camus2 12y agoStill trying to find a suitable "python to js compiler written in js that runs in the browser" project.PyPy.js is unfortunatly too big and all the other projects i found are either written in python or dont pass the browserify step.I need a transpiler that runs in the browser.