13 ms·
Faster CPython at PyCon, part one
- cfiggers 3y agoNooooo if Python gets faster then all the Mojo demos are going to sound so much less impressive nooo
- albertzeyer 3y agoJAX or torch.compile also do sth similar as Mojo, right?
- npalli 3y agoI don't know, looks like ~15% improvement [1]. Doubt the Mojo guys are going to lose much sleep (they claimed 35,000x improvement). [1] https://speed.python.org/comparison/?exe=12%2BL%2B3.11%2C12%2BL%2Bmaster&ben=741%2C742%2C743%2C744%2C745%2C746%2C747%2C748%2C749%2C799%2C800%2C750%2C751%2C802%2C752%2C753%2C754%2C755%2C756%2C757%2C758%2C759%2C760%2C761%2C762%2C763%2C764%2C765%2C766%2C767%2C768%2C769%2C770%2C771%2C772%2C773%2C774%2C775%2C776%2C777%2C778%2C779%2C780%2C781%2C782%2C783%2C784%2C785%2C786%2C788%2C787%2C789%2C790%2C791%2C792%2C793%2C794%2C797%2C796%2C795%2C798&env=1&hor=true&bas=12%2BL%2B3.11&chart=normal+bars https://speed.python.org/comparison/?exe=12%2BL%2B3.11%2C12%...
- TwentyPosts 3y agoSomehow I'm still skeptical! Feels like "strict superset of Python, all Python code works" and "several orders of magnitude faster" just sounds like you're trying to have your cake and eat it, too. I doubt that the Mojo developers have some sort of 'secret sauce' or 'special trick' that will get them there. And even if they have something, I don't see why the Python devs wouldn't just implement the same approach, considering they're currently trying to make Python faster. I assume that (as long as Mojo wants to stick to its goal of being a strict superset of Python), there will be a lot of things that just cannot be 'fixed'. For example, I'd be surprised if Python's Global Interpreter Lock isn't entangled with the language in a few nasty ways that'd make it really difficult to replace or improve upon (while retaining compatibility). Then again, the developer of Swift is working on it, right? I guess he's got the experience, at least.
- School-Cotton 3y agoIDK about 35,000x but Python really is outrageously slow relative to other popular languages, even other interpreted ones. It's more comparable to something like Ruby than to something like JavaScript.
- williamstein 3y agoFrom their FAQ: "How compatible is Mojo with Python really? Mojo already supports many core features of Python including async/await, error handling, variadics, etc, but… it is still very early and missing many features - so today it isn’t very compatible. Mojo doesn’t even support classes yet!". Overall, pure Python seems to be about 100x slower than what you can reasonably get with a compiled language and some hard work. It's about 10x slower than what you can get from JITs like Pypy and Javascript, when such comparisons makes sense. I agree that Mojo remind me of Cython, but with more marketing and less compatibility with Python. Cython aspired to be a nearly 99% superset of Python, at least that's exactly what I pushed my postdocs and graduate students to make it be back in 2009 (e.g., there were weeks of Robert Bradshaw and Craig Citro pushing each other to get closures to fully work). Mojo seems to be a similar modern idea for doing the same sort of thing. It could be great for the ecosystem; time will tell. Cython could still also improve a lot too -- there is a major new 3.0 release just around the corner!: https://pypi.org/project/Cython/#history https://pypi.org/project/Cython/#history
- deleted 3y ago[deleted]
- wiseowise 3y agoIsn’t Mojo compiled? While Python needs to be interpreted.
- TwentyPosts 3y agoMojo bothers offers Just-In-Time and Ahead-Of-Time models. Proper AOT compilation is going to offer a significant speedup, but probably not enough to get to their stated goals. And good luck carrying all of Python's dynamic features across the gap.
- tgma 3y agoThat comparison seems quite cherry picked. Unlikely that it is generalizable. In my tests with Mojo, it doesn’t seem to behave like Python at all so far. Once they add the dynamism necessary we can see where they land. I’m still optimistic (mostly since Python has absolute garage performance and there’s low hanging fruit) but it’s no panacea. I feel their bet is not to have to run Python as is but have Python developers travel some distance and adapt to some constraints. They just need enough momentum to make that transition happen but it’s a transition to a distinct language, not a Python implementation. Sort of what Hack is to PHP.
- SOLAR_FIELDS 3y agoHack as a comparison is probably not a great thing for Mojo, since no one outside of Facebook uses Hack.
- tgma 3y agoSafe to say no one outside of Modular uses Mojo either at the moment.
- KeplerBoy 3y agoI don't buy the mojo hype. The concept of compiling python has been tried again and again. It has its moments, but anything remotely important is glued in from compiled code anyways.
- ptx 3y agoFrom a quick reading of the Mojo website, it sounds like gluing in compiled code is exactly what they're doing, except this time the separate compiled language happens to look sort of like Python a bit. For the actual Python code it still uses CPython, so that part doesn't get any faster.
- morelisp 3y agoPyrex, Cython, mypyc, Codon... we've been down this road before plenty too. If Mojo succeeds it will be purely based on quality of implementation, not a spark of genius.
- pdpi 3y agoThe "spark of genius" might very well be a novel implementation strategy.
- neolefty 3y agoIt leverages the Multi-Level Intermediate Representation (MLIR) https://mlir.llvm.org/ https://mlir.llvm.org/ which is a follow-on from LLVM, with lots of lessons learned: https://www.hpcwire.com/2021/12/27/lessons-from-llvm-an-sc21-fireside-chat-with-chris-lattner/ https://www.hpcwire.com/2021/12/27/lessons-from-llvm-an-sc21...
- morelisp 3y agoAgain, "compile to an IR" isn't exactly ground-breaking. The devil will be in the details.
- deleted 3y ago
- klyrs 3y agoI've been rewriting Python->C for nearly 20 years now. The expected speedup is around 100x, or 1000x for numerical stuff or allocation-heavy work that can be done statically. Whenever you get 10,000x or above, it's because you've written a better algorithm. You can't generalize that. A 35k speedup is a cool demo but should be regarded as hype.
- nomel 3y ago> The expected speedup is around 100x, or 1000x for numerical stuff What if you stay in the realm of numpy? What's the biggest offender that you see?
- klyrs 3y ago> What if you stay in the realm of numpy? You mean, what if you're only doing matrix stuff? Then it's probably easier to let numpy do the heavy lifting. You'll probably take less than a 5x performance hit, if you're doing numpy right. And if you're doing matrix multiplication, numpy will end up faster because it's backed by a BLAS, which mortals such as myself know better than to compete with. > What's the biggest offender that you see? Umm... every line of Python? Member access. Function calls. Dictionaries that can fundamentally be mapped to int-indexed arrays. Reference counting. Tuple allocation. One fun exercise is to take your vanilla python code, compile it in Cython with the -a flag to produce an HTML annotation. Click on the yellowest lines, and it shows you the gory details of what Cython does to emulate CPython. It's not exactly what CPython is doing (for example, Cython elides the virtual machine), but it's close enough to see where time is spent. Put the same code through the python disassembler "dis" to see what virtual machine operations are emitted, and paw through the main evaluation loop [1]; or take a guided walkthrough at [2]. [1] https://github.com/python/cpython/blob/v3.6.14/Python/ceval.c#L1274 https://github.com/python/cpython/blob/v3.6.14/Python/ceval.... (note this is an old version, you can change that in the url) [2] https://leanpub.com/insidethepythonvirtualmachine/read https://leanpub.com/insidethepythonvirtualmachine/read
- vbarrielle 3y agoDue to the possibility to fuse multiple operations in C++ (whereas you often have intermediate arrays in numpy), I routinely get 20x speedups when porting from numpy to C++. Good libraries like eigen help a lot.
- ehsankia 3y ago35,000x suddenly becomes 30,000x, not as impressive!
- Alex3917 3y agoHow does that reconcile with the benchmarks here, which say that Python 2.12 is currently somewhere between 5% slower to 5% faster? https://github.com/faster-cpython/benchmarking-public https://github.com/faster-cpython/benchmarking-public
- make3 3y agoif it's an improvement that big it needs to be with GPUs, & gpus can be used normally with torch etc
- traverseda 3y agoMeh, mypyc and codon both do the same thing as mojo. Codon seems to have similar speed improvements and the source code is available.
- jjtheblunt 3y agoDo they compile to MLIR so when Alteryx writes a code generator you can target FPGAs, for example?
- traverseda 3y agoLLVM ir. So probably there are some flags somewhere. Also supports openmp and gpu compute through decorators.
- brrrrrm 3y agoOnce fused into super-instructions/adaptive instructions, does `disas` still allow users to manipulate the bytecode as if it were not optimized? That'd be a very nice feature and I'm not sure it exists in other languages.
- kmod 3y agoI haven't followed things recently but I know that at least initially they purposefully kept the bytecode intact so that existing debuggers and compilers and what not would still work. I believe they put the optimized bytecode in a different field, and accept this duplication as the cost of supporting the feature you requested.
- deleted 3y ago[deleted]
- Spivak 3y agoSpeaking of Python speed you gotta hand it to the core Python devs having some really dedicated numerical analysis folks. A really good gentle introduction talk https://www.youtube.com/watch?v=wiGkV37Kbxk https://www.youtube.com/watch?v=wiGkV37Kbxk Python really has earned its place as the language of choice for scientific, research, and ML. I've stolen quite a few algos from CPython.
- wheelerof4te 3y ago"Speaking of Python speed you gotta hand it to the core Python devs having some really dedicated numerical analysis folks." Well, they are Python devs after all.
- deleted 3y ago[deleted]
- albertzeyer 3y ago> LOAD_ATTR_ADAPTIVE operation can recognize that it is in the simple case How can it do that? The `self` variable could be anything, or not?
- Sohcahtoa82 3y agoIf you define a .__getattr__ method, then "self.y" is no longer simple.
- Jasper_ 3y agoIf it finds itself not in the simple case it specialized for, it reverts back to the unspecialized case. In JIT compiler terms, this is called a "guard"
- albertzeyer 3y agoBut isn't this the same as what LOAD_ATTR also does? It first checks the simple case, and if that fails, it goes the complex fallback route. Why does this need a new op? Or why would the standard LOAD_ATTR not do this? Or are there multiple versions of the compiled function, and it modifies the bytecode per each call of the function?
- adrian17 3y agoThe issue is that there are many different "simple cases". The more checks for "fast paths" you add, the more overhead of individually checking for them. Also consider that: - this is more branch predictor friendly (a generic opcode encounters many different cases, while a specialized opcode is expected to almost never fail), - the inline cache has limited space and is stateful (imagine it's a C union of cache structs for different fast paths). If your "generic" LOAD_ATTR found a simple case B, it'd still need to ensure that its inline cache was actually primed for simple case B. This is not the issue with specialized opcodes; the cache is set to correct state when the opcode gets specialized.
- JaDogg 3y agoShameless plug: https://yakshalang.github.io/ https://yakshalang.github.io/ This is my passion project. (Less activity these days as I'm sick)
- mpenick 3y agoI hope you feel better. This is a neat project and I’m looking forward to trying it out.
- abnercoimbre 3y agoI'm impressed. Shameless plug to your plug: you may want to join the Handmade [0] community or attend our conferences [1]. You'd fit right in with us! [0] https://handmade.network https://handmade.network [1] https://handmadecities.com https://handmadecities.com
- JaDogg 3y agoNice. I'm already aware of handmade.network. I was plan on joining but lost track of it. handmadecities also look interesting. Will take a look. Might take until end of month.
- TheMagicHorsey 3y agoPython's real strength at this point is the ecosystem. If you want speed you wouldn't use Python. Having said that, its better for the planet if default Python is more efficient. Everyone has seen Python code doing stuff Python code should not do at scale.
- melenaboija 3y agoThanks for the information, it has never been said before in HN when an improvement to Python happens and I am sure there is still someone out there thinking that Python is fast. BTW from the PEP: “Motivation Python is widely acknowledged as slow”
- TheMagicHorsey 3y agoNo need to get testy. I know its widely known. The problem is, we still see Python in performance sensitive areas at scale anyway. So I said its a good thing. I'm not trying to give anyone a wedgy by saying that. I thought I was just giving some justification for the optimization that OP describes.
- jerf 3y agoThere are a lot of people out there who interpret "Python is a slow language" as an attack, rather than just something that engineers need to know and account for at solution time. There are also a non-trivial number of people out there who read about the constant stream of performance improvements and end up confusing that with absolute high performance; see also the people who think Javascript JITs mean Javascript is generally at C performance, because they've consumed too many microbenchmarks where it was competitive with C on this or that task, and got 500% faster on this and 800% faster on that, and don't know that in totality it's still on the order of 10x slower than C. It is, unfortunately, something that still needs to be pointed out.
- melenaboija 3y agoI am more of the idea that people that truly need performance already know about the differences of Python vs C and my experience with slow Python code is almost always due to lack of basic CS knowledge rather than the language per se. Maybe my comment was too mean, but every time that something related to Python core enhancements appears someone has to say that C is faster and I am not sure if someone interested on how the generated bytecode improves the performance needs to be reminded that C is faster.
- titzer 3y agoI like tinkering with interpreters, and I do it a lot, but this is chump change compared to even a baseline compiler, especially one that makes use of type profiles. There is a pretty big investment in introducing a compiler tier, but I'm not convinced that it's less engineering effort than squeezing more out of an interpreter that fundamentally has dispatch and dynamic typing overhead. There aren't that many bytecode kinds in CPython, so it seems worth doing.
- tekknolagi 3y agoThat's what we did in Skybison before it got wound down: https://github.com/facebookexperimental/skybison https://github.com/facebookexperimental/skybison
- titzer 3y agoAre there publicly-available performance results?
- tekknolagi 3y agoKind of! In my fork I run microbenchmarks on each PR. So you can see on, for example, https://github.com/tekknolagi/skybison/pull/456 https://github.com/tekknolagi/skybison/pull/456, that the change had a 3.6% improvement on the compilation benchmark. If you expand further, you can see a comparison with CPython 3.8. Unfortunately Skybison is still on 3.8.
- mardifoufs 3y agoWhat's the difference between that and cynder? I know they both come from Meta/IG but I wonder if cynder is an evolution from skybison or is it more of a different approach?
- tekknolagi 3y agoCinder and Skybison are different approaches. Skybison was a ground-up rebuild of the object model and interpreter, whereas Cinder is a JIT that lives on top of CPython.
- AbsoluteCabbage 3y agoI prefer C without braces
- low_tech_punk 3y agoWhen will such goodness come to JavaScript engines? I’m curious if there is some fundamental design flaws that made JavaScript difficult to optimize.
- Jasper_ 3y agoJavaScript engines have been using inline caches and specialization for over 10 years now. It's Python that's catching up to JavaScriptCore from 2007.
- formerly_proven 3y agoI mean JavaScript is definitely gross to optimize, if you'd design a language for easy optimization and jitting it would look different. Python is just way worse, especially because of the large and rich CPython C API, which used invasive structures almost everywhere (it has been slowly moving away from that). The CPython people, GvR in particular, always considered it very important for the codebase to remain simple as well, and it is. CPython is a really simple compiler and straightforward bytecode interpreter.
- esaym 3y agoNode.js has been on par with Java for many years now due to the Jit and tons of money google has pumped into that ecosystem: https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/binarytrees.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- shepardrtc 3y ago
- AbsoluteCabbage 3y agoI love me some Python, trust me. But this is just putting lipstick on a pig. The language needs to be rewritten from the ground up. If you’ve ever analyzed the machine code a simple python script spits out you’ll see how helplessly bloated it is.
- leetrout 3y agoDo you have thoughts around the concept of Python becoming a Rust macro based language? I read another comment about that being a potential future for Python but I don't know how feasible it is. I've moved on to Go for things where I need to care about performance but like thinking about things in a similar way Python lets me think about them.
- tredre3 3y ago> If you’ve ever analyzed the machine code a simple python script spits out you’ll see how helplessly bloated it is. How would one do that? I thought python generated bytecode and interpreted that code? Where's the machine code? Do you mean dissembling the python interpreter itself? In which case it would be gcc/llvm spitting out machine code, no?
- __d 3y agoYou can make a Python-ish language that can run fast (or at least, a lot faster) but it cannot be Python: the dynamic nature of the language means many optimizing techniques just aren't available. The unfortunate thing is that the vast majority of Python code in use today doesn't need all those super-dynamic features. So it would run just fine on a cut-down JITed interpreter. But there's always the corner cases. In retrospect, Python 3 was a lost opportunity here: it could have broken more compatibility, enabled multi-core and JITs, and then the 10 year transition pain would actually have been worth it. But that's hindsight, of course.
- robertlagrant 3y agoIt would be cool if they'd put some resource into Pypy as well. Feels like that project is such a great achievement, and with just a bit more resource it would've been an amazing alternative. As it is, it takes a long time to get the new language features out, which always hamstrings it a little.
- adamckay 3y agoThere's a limited number of volunteers with the experience to improve (C)Python, though, so the resources are better spent on improving the CPython implementation to such a point where PyPy is no longer necessary.
- girvo 3y agoSee: HHVM and PHP. HHVM is what lit the fire under the PHP team's collective butts to improve perf, and now most just run the mainline PHP interpreter (well, they did years ago when all this happened and I was still doing PHP)
- qaq 3y agoNow there is a number of payed MS employees lead by Guido van Rossum working on improving (C)Python
- v3ss0n 3y agoWhy don't they just replace python with PyPy, onboard PyPy team and make PyPy compatible with cpython 100 percent? Oh I think NIH syndrome.
- damagednoob 3y agoFrom what I remember, PyPy has some fundamental issues with using libraries with C extensions.
- v3ss0n 3y ago
- freecodyx 3y agoAlways keep in mind that you can’t go faster than what the harware can allow. Sometimes even by what the OS can allow. We can do way better on some specific tasks. But for for that you need to rewrite some algo implementation to better take advantage of all the hardware can allow. ML is a specific task, if you can take advantage of the harware to do better matrix mul. Does not mean you can have the same speed ups overall. Mojo for example
- pipeline_peak 3y agoPorting to WASM and getting C bindings to be compatible on steady well developed VMs will make Python faster.
- jjtheblunt 3y agoHow, specifically?
- pipeline_peak 3y agoBecause C code in web assembly can translate to native machine code and still interop with Python. All this PyObject nonsense does nothing but slowdown the runtime. V8 was built for performance from day 1. CPython is old, slow, and will always be outperformed by V8 regardless of the tweaks this team does. I am not a compiler engineer, I can’t give you a specific, technical answer.
- jjtheblunt 3y agoI was a compiler engineer, even was a committer to V8 itself via Qualcomm's codeaurora legal entity while a Qualcomm engineer, bringing up its backend code generation support to new Snapdragon processors with SIMD extensions about 14 years ago, with a coworker double checking the work on hardware targets. That said, I hope you are right, and it's fun to watch, as time will tell.
- pjmlp 3y agoSo basically following JVM footsteps and CLR in regarding to specialization bytecodes, e.g. invokedynamic would be one of them.
- jwilk 3y agoPart two: https://news.ycombinator.com/edit?id=35919949 https://news.ycombinator.com/edit?id=35919949