15 ms·
Python 3.11 beta vs. 3.10 benchmark
- baq 4y agoimpressive gains for such a mature project.
- SemanticStrengh 4y ago
- FartyMcFarter 4y agoI think it's more fair to say that Python became widely used for purposes it wasn't designed for. Originally Python was meant as a language to write small to medium scripts in, not a language for complex programs benefiting from high performance.
- SemanticStrengh 4y agoin other words, a historical accident
- Spivak 4y agoOutside of Java, Kotlin, and Go are there any languages designed specifically for large scale corporate/enterprisey use?
- ThomasBHickey 4y agoAda
- FartyMcFarter 4y agoCOBOL ?
- SemanticStrengh 4y agoC# indeed. But even javascript has a bearable JIT nowadays. Also no need for the vague enterprisey connotation, only features and performance are needed, which e.g. scala provide and Go do not (feature wise).
- Spivak 4y agoOhh and Swift! I just mean that lots of popular languages were historical accidents when it comes to their use on large scale projects and it’s pretty rare (I suppose less now it seems) for someone to go out and invent languages specifically for large scale repos and “toy language is retrofitted for scale” is a more common narrative.
- SemanticStrengh 4y agoYes historically this has been true but we are past this time, no need for each new language to duplicate it VM, its JIT, its GC.. GraalVM solve it all and bring ecosystems interop.
- aeyes 4y agoErlang/OTP but you are probably asking about the usability once you have 100+ developers working on a project so it doesn't quite fit in. Even Go is questionable here when compared to Java, C# and even C++.
- stdbrouw 4y agoI mean, I guess technically it started as an "educational language", but much of its early-ish development was aimed at turning it into an enterprise language that could rival Java. Guido worked for Zope Corporation from 2000-2003, Zope is a big hairy enterprisey CMS. 10-15 years ago, people at Google for a very long time toyed with the idea of making Python fast enough so they could just use it for everything, cf. Unladen Swallow. That said, it's true that Python has ended up being used in places that were not envisioned by its original creators, but the implication that it is therefore a bad fit for those settings does not automatically follow. Ruby was never meant to be a "web language" but many seem to enjoy using it that way.
- chippiewill 4y agoIt's not _that_ surprising. The CPython developers have never historically prioritised performance. Simplicity and readability of implementation have typically been higher priorities. In fact that's a key quality of Python in general that development speed is more important than runtime performance. PHP had a similar performance binge about 6 years ago which is why it tends to run circles around similar class interpreted languages these days
- Spivak 4y agoAnd I can really appreciate that. Reading the CPython source is extremely easy for me and I have only “I learned it in college” level C experience.
- 323 4y agoThere was no serious money invested until recently in Python performance. Imagine what could happen if billions of dollars were invested in performance like it was done for C/C++/Java/Javascript in aggregate by various companies. As more Python is being run in datacenters that starts to make sense, since 1% of CPU usage improvement can mean tens of millions of dollars per year in power costs.
- pjmlp 4y agoI am quite sure PyPy has gotten lots of money, specially from EU research grants. As did IronPython, while Microsoft cared to burn money on it.
- 323 4y agoQuick search shows PyPy received 1.5 mil euros from EU. It also received 200 k dollars from Mozilla. That's enough to fund about 10 developers for a year. From what I remember IronPython had Microsoft hire something like 5 developers for a few years.
- Game_Ender 4y agoThis comes from the following enhancements: - Cheaper, lazy Python frames: 3-7% - Inlined Python function calls: 1-3% - PEP 659: Specializing Adaptive Interpreter: variable The interpreter changes work by generating specialized code so this might have a memory impact which they are working to offset and expect to cap at 20% more. See the in progress release notes for details: https://docs.python.org/3.11/whatsnew/3.11.html#faster-cpython https://docs.python.org/3.11/whatsnew/3.11.html#faster-cpyth...
- SemanticStrengh 4y agoAre you saying python never inlined functions before??
- giancarlostoro 4y agoI'll assume this is a good faith question and answer it by quoting the "What's new in Python 3.11?" page (linked below the quote). > During a Python function call, Python will call an evaluating C function to interpret that function’s code. This effectively limits pure Python recursion to what’s safe for the C stack. > In 3.11, when CPython detects Python code calling another Python function, it sets up a new frame, and “jumps” to the new code inside the new frame. This avoids calling the C interpreting function altogether. > Most Python function calls now consume no C stack space. This speeds up most of such calls. In simple recursive functions like fibonacci or factorial, a 1.7x speedup was observed. This also means recursive functions can recurse significantly deeper (if the user increases the recursion limit). We measured a 1-3% improvement in pyperformance. https://docs.python.org/3.11/whatsnew/3.11.html#inlined-python-function-calls https://docs.python.org/3.11/whatsnew/3.11.html#inlined-pyth...
- SemanticStrengh 4y agoBTW It'd be nice if python implemented support for tail call recursive or even better, a growable/segmented stack in order to allow arbitrary pythons functions be stack overflow safe. e.g. https://gcc.gnu.org/wiki/SplitStacks#:~:text=Split%20Stacks%20in%20GCC&text=The%20goal%20of%20split%20stacks,as%20required%20by%20the%20program https://gcc.gnu.org/wiki/SplitStacks#:~:text=Split%20Stacks%....
- rich_sasha 4y agoIs there some headline reason for these improvements? I remember reading about the GILectomy, and how actually many of the improvements, aimed at making GILectomy feasible, make single-threaded Python runs faster too. There was even the trepidation that PSF might accept these changes, but still say no to GILectomy. Is this at all related?
- bpicolo 4y agoMicrosoft is funding the work with Mark Shannon / GVR / Eric Snow. The project is called "Faster Cpython". Pretty excited for the 3.12 work, where they plan to introduce JIT compilation I believe? I'm not sure about the progress on the new form of GIL removal, but as far as I've seen it's a separate effort. https://github.com/faster-cpython/ https://github.com/faster-cpython/ https://www.theregister.com/2021/05/13/guido_van_rossum_cpython_3_11/ https://www.theregister.com/2021/05/13/guido_van_rossum_cpyt...
- Alex3917 4y ago> Pretty excited for the 3.12 work, where they plan to introduce JIT compilation I believe? "A JIT, according to Shannon, will probably not arrive until 3.13 at the earliest, given the amount of lower-hanging fruit that is still to be worked on. The first step towards a JIT, he explained, would be to implement a trace interpreter, which would allow for better testing of concepts and lay the groundwork for future changes." https://pyfound.blogspot.com/2022/05/the-2022-python-language-summit_2.html https://pyfound.blogspot.com/2022/05/the-2022-python-languag...
- KptMarchewa 4y agoRemoving GIL naively would decrease single thread performance. Every project aiming at removing GIL failed because it could not get performance comparable with GILed Python.
- sumtechguy 4y agoRemoving the GIL is an idea (and as you point out not working very well). When optimizing do not depend on 'that one cool trick' to fix everything. In this case it looks like they are removing extra work and doing work once and keeping a copy around (caching).
- SemanticStrengh 4y agoWould be interesting to compare with GraalPython
- kmod 4y agoGraalPython only supports a relatively-small subset of the Python language, making it impossible to collect numbers on realistic benchmarks
- azinman2 4y agoWill that change? I thought the promise of graal is to massively speed up languages like Python and Ruby.
- SemanticStrengh 4y agoGraalJS has ~99% support of the JS language features, as for GraalPython it's unclear to me what's missing but it should get feature parity in the next few years. It already support modern python versions so that's a good sign.
- freediver 4y agoDid that python-speed v1.3 using python v3.8.5 string/mem: 3032.69 ms pi calc/math: 6652.63 ms regex: 2442.87 ms fibonnaci/stack: 382.66 ms multiprocess: 11573.28 ms total: 24084.13 ms (lower is better) Compare to results in the first comment in this post. Graal looks much slower except for stack manipulation.
- SemanticStrengh 4y agoThanks a lot, hmm kinda disappointing, except for fibonnaci/stack. Was it in native (binary) of JIT form ? Also GraalVM EE has additional optimizations. (I'm assuming you used GraalVM 2022.1 edition) Anyway thanks for sharing :)
- freediver 4y ago
- FartyMcFarter 4y agoI remember noticing that Python 3 interpreter startup was significantly slower than Python 2 a couple of years ago. I wonder if these improvements have reversed the situation.
- BiteCode_dev 4y agoI think this is more of a job for PEP 690: https://peps.python.org/pep-0690/ https://peps.python.org/pep-0690/
- digisign 4y agoSomewhat, yes.
- hexomancer 4y agoPython is so slow that I ironically don't care about performance improvement that much, because if you are writing any performance-sensitive part of your software in python you are screwed anyway, 50% faster code isn't going to help that much.
- freediver 4y agoPython is a joy to use and is used by a large number of apps as a tool of choice to get things done. Almost all of "retail" web crawling and machine learning in the world runs on Python. In practice it is not slow at all. Plus the development iteration speed is probably second to none of all the programming languages. If I would have my kid learn two programming languages it would be HTML and Python.
- hexomancer 4y agoI never said that python is a bad language. In fact it is my favorite language and I probably have written more python code than any other language. That has nothing to do with it being slow though. And as I said, all those machine learning code rely on numpy (which relies on LAPACK which is not written in python) or highly optimized cuda kernels (which again is not written in python).
- solarkraft 4y agoWhich is the optimal split! Write application/composing code in Python and highly performance sensitive parts in some FFI language (nowadays probably Rust?). It's not great if you do performance sensitive stuff all the time, but amazing if you just want to build something.
- zekrioca 4y agoI understand and somehow agree with your point, however, it would also be nice for someone to have the ability to build something performant with a language like Python. But this is not really possible today, so the narrative that Python is for prototyping will keep holding.
- freediver 4y agoCompared speed to Python 3.9 using python-speed for those who want a simpler, more straight-forward benchmark. [1] Basically one can expect overall 24% increase in performance "for free" in a typical application. Improvements across the board in all major categories. Seriously impressive. Stack usage and multiprocessing had the largest performance increase. Even regex had 21% increase. Just wow! And this may be the first Python3 that will actually be faster (about 5%) than Python 2.7. We've waited 12 years for this... Excited about Python's future! ----- python-speed v1.3 using python v3.9.2 string/mem: 2400.67 ms pi calc/math: 2996.1 ms regex: 3201.59 ms fibonnaci/stack: 2487.13 ms multiprocess: 812.37 ms total: 11897.85 ms (lower is better) ----- python-speed v1.3 using python v3.11.0 string/mem: 2234.78 ms pi calc/math: 2667.84 ms regex: 2548.81 ms fibonnaci/stack: 1149.57 ms multiprocess: 480.25 ms total: 9081.25 ms (lower is better) ----- [1] https://github.com/vprelovac/python-speed https://github.com/vprelovac/python-speed
- bjt2n3904 4y agoJust on the python3/python2.7 speed... This one has always killed me on BeagleBones... `$ time python3 -c "print('hello')"` I have a handful of utility scripts written in python, and the overhead of starting/stopping is just massive! (Sadly, I don't have one on hand to actually print a metric...)
- freediver 4y agoLuckily in most cases a python web app is 'served' so it does not have this start-up overhead.
- londons_explore 4y agoI'd like to see some kind of financial marketplace for this kind of speedup. There are tens of thousands of people who wish their python code would run a bit quicker. Many of those stand to earn/save actual money if the code was quicker, so would be happy to pay some of that towards making optimisations. If that could be pooled together, with some kind of "$100k per 1% speed up on this set of benchmarks" metric, then developers could get properly paid for the work, and everyone would walk away happy. As it is, everyone wishes it was faster, but realises that they alone can't pay a developer to make a dent, so nobody does it.
- allendoerfer 4y agoOr we could continue to expect big tech monopolies, which are extracting the most value out of Open Source, to also fund it.
- londons_explore 4y agoSuch a system works to an extent, but I suspect the globally economically optimal amount of effort to put into these widely used opensource projects is far higher than the actual effort put in.
- properdine 4y agoCould the interpreter have a "restricted" mode and a "dynamic" mode, where restricted mode can be faster because it skips certain lookups? And then if your code _engages_ in a "dynamic" behavior, it then drops out of "restricted" mode? (this could also be in a class by class basis too). Django and friends might always have to be in "dynamic" mode, but possibly you could allow for some complex logic to run quickly if you use subset of the language. (e.g., like RPython but with less overhead to set up)
- Someone 4y agoIt could, but detecting whether it has to “drop out” has a performance impact. Engineering such a feature without horrendously impacting performance is complex. Also, users would not like to see performance drop permanently, only because they redefined some function (e.g. to log it’s arguments in a debugging session), so they’ll expect the system to (eventually) re-optimize code using the new state. Languages such as JavaScript and Java do this kind of thing (Java not because programs can redefine what len means, but because the JITter makes assumptions such as “there’s only one implementation of interface Foo” or “the Object passed to this function always is an integer”), but I think both have it easier to detect the points in the code where they need to change their assumptions. I also guess both have had at least an order of magnitude more development effort poured into them.
- nomel 4y ago> It could, but detecting whether it has to “drop out” has a performance impact. I would be ok with a flag that raised an exception/halted if the restricted dynamicism was encountered, if it meant appreciable performance gains. For every project I've worked on, and every project I've really become familiar with, the "magic" that requires these crazy levels of dynamicism can relatively easily be avoided, with more "standard" interfaces, and possibly a very slight increase in complexity presented to the library user. I used to play with python magic frequently, but eventually realized my motivation was just some meta code-flex game I was playing, and it's almost certainly never worth it, if other developers are involved.
- kortex 4y ago
- jadbox 4y agoHow does real world Python performance compare with Node.js these days? Anyone have recent experience with the semi-latest?
- kirbyfan64sos 4y agoAFAIK, it depends on your bottlenecks. In terms of actual compute speed, Python is still significantly slower, and although 3.11's changes will help quite a bit, V8 is also just insane. In terms of I/O speed, uvloop (https://magic.io/blog/uvloop-blazing-fast-python-networking/ https://magic.io/blog/uvloop-blazing-fast-python-networking/) can beat Node quite handily, so if you're more concerned with being able to handle requests than doing anything major during those requests, Python might be comparable.
- lenkite 4y agoHow is this faster than nodejs if both are built on libuv ?