9 ms·
RustPython – A Python-3 (CPython >= 3.11.0) Interpreter written in Rust
- mikece 4y agoIs there any chance that a project like RustPython could supplant CPython?
- macspoofing 4y agoTo solve what problem exactly?
- kilgnad 4y agoTo solve the problem that is C.
- wiseowise 4y agoAny constructive comments or we’re /r/ProgrammerHumor now?
- kilgnad 4y agoit wasn't suppose to be funny. C has specific problems. Rust is a solution to those specific problems.
- ameliaquining 4y agoI think the question is how those problems are applicable specifically to CPython's use of C. I agree that it would be better, all else being equal, if CPython were implemented in a memory-safe language (though it would almost certainly need to use unsafe escape hatches in some places), but I think this'd be more viable if it were done by incrementally migrating the existing codebase with the involvement of the current maintainers, rather than as a third-party rewrite from scratch.
- v3ss0n 4y agoThen there is python written in python called pypy
- orangepurple 4y ago> For your big question about a CPython replacement I think that this is our current goal. I do not think people will be using RustPython in their production any time soon but I do think we can be competitive in WASM. Having said that I do not think we should be 100% compatible with CPython. For example we have chosen to implement threading without the GIL. Being compatible with CPython is very helpful as we can use the documentation and tests of CPython. https://github.com/RustPython/RustPython/issues/1940#issuecomment-635818817 https://github.com/RustPython/RustPython/issues/1940#issueco...
- bionade24 4y agoIf they support python packages exposing the C ABI and expose a C ABI for embedding, I can't see why not.
- nilsbunger 4y agoThey're doing threading without the GIL, which means it's not compatible with existing C extensions. CPython is also building a -no-gil compile-time option. If that gets traction, some of the big performance-oriented C extensions like Numpy etc would likely build a non-GIL extension version too. Then, if RustPython's non-GIL implementation is compatible with CPython's, and their C ABI is compatible, those C extensions might work.
- guipsp 4y agogil or no gil is mostly irrelevant for numpy, an extension that does most work with gil released.
- cdavid 4y agoRelying on the cpython gil and releasing the gil are two different thing: 1. releasing the gil means multithreading is opt-in for a given code section in NumPy. Only very specific parts of the code need to be threadsafe. 2. not relying on a gil in cpython runtime means multithreading becomes opt out. Now all the code by default needs to be threadsafe, including the libs you depend on. A lot of C/C++/Fortran scientific code is not thread safe, and the whole scientific python ecosystem depends heavily on those codebases.
- guipsp 4y agoI understand what you are saying, but I don't see how it's relevant? Numpy doesn't rely on the gil, which is why it releases it.
- cdavid 4y agoIt releases the GIL only in very specific sections. Most of the Numpy C code runs under the gil. A quick check on master shows only 10-20 calls using NPY_BEGIN_ALLOW_THREADS (which is an alias to Py_BEGIN_ALLOW_THREADS). A lot of the NumPy code manipulates python runtime objects, and doing so without thread safety would likely break everywhere. A lot of efforts would be needed to gradually make large C extension thread safe.
- ehutch79 4y agoDepends... There are already other implementations. But CPython is the reference implementation. That's part of why we're only now seeing optimizations that reduce code clarity for reading. (i think, dont quote me on this)
- dr_zoidberg 4y agoNo need to quote you, but GvR himself [0]. That's the Faster CPython initiative, which has been ongoing for about 2 or 3 years I think. 3.11 got some nice speedups from it, and 3.12 is on its way to get more. [1] [0] https://github.com/faster-cpython/ideas/blob/main/FasterCPythonDark.pdf https://github.com/faster-cpython/ideas/blob/main/FasterCPyt... [1] https://github.com/faster-cpython/ideas https://github.com/faster-cpython/ideas
- loeg 4y agoOnly if the CPython devs will it so.
- zitterbewegung 4y agoTLDR: No The hardest part of supplanting CPython is the fact that the FFI is already validated on a huge amount of implementations. I believe Cinder and Pyston work out of the box but they are on older versions of python but Pyston wants to be merged back into CPython and so does Cinder (or at least what is relevant). On the other hand JIT in CPython can be achieved by other python packages. Pyston has also extracted the JIT and can be added to 3.7 - 3.10 by installing it. See https://pypi.org/project/pyston/ https://pypi.org/project/pyston/ Pypy is a separate reference implementations that has tried to achieve something different but hasn't supplanted python for years. It has had a JIT and the ability to change garbage collectors but the system stuffers when interacting with C FFIs Pyodide is inthe browser so its something completely different but has no support for CFFIs. IronPython (.net), Jython are inactive. Also GIL removal from CPython has been attempted multiple times but now that CPython is being funded by Microsoft to become faster makes it more unlikely that CPython will be supplanted.
- brucethemoose2 4y agoGraalVM also has a nascent Python runtime, which could be extremely performant. But I think OS devs dont really want to put optimization work into GraalVM, as their hard worked could get sucked up and locked behind the Enterprise Edition.
- zitterbewegung 4y agoMainly resource exhaustion from developers makes the other runtimes fade into the background while resources being put into web deployment (pyodide) and microcontrollers (micropython and circuitpython) are a much better idea.
- noloblo 4y agoWhat's graalvm and where does it fit in with the larger Python ecosystem
- brucethemoose2 4y agoIts a "universal" VM from Oracle that supports many languages and other bells and whistles like AOT compilation of interpreted languages: https://www.graalvm.org/ https://www.graalvm.org/ But its Java support is most prominant, and the its runtime is significantly faster than OpenJDK. Which is why its so promising for Python. I'd argue that OpenJDK is way ahead of the Pyton runtime, and GraalVM is way ahead of OpenJDK. But its also kinda iffy because: - Python support isn't very good now - Oracle is Oracle. Hence they have the better optimizations walled off behind a "Enteprise Edition" registration and license.
- mkoubaa 4y agoWith HPy yes
- klaussilveira 4y agoYou can use just the parser if you want to: https://rustpython.github.io/blog/2020/04/02/thing-explainer-parser.html https://rustpython.github.io/blog/2020/04/02/thing-explainer... Which is great for embedding or customizing your scripting language needs.
- 0xffff2 4y agoThe first link on that page 404s. Anyone know the correct link to the parser code?
- isaacg 4y agohttps://github.com/RustPython/RustPython/tree/main/compiler/parser https://github.com/RustPython/RustPython/tree/main/compiler/...
- dimitrisjim 4y agoAh, thanks. Fixing this now.
- epage 4y agoI believe ruff [0] is using this for their linting / fixing [0]: https://beta.ruff.rs/docs/ https://beta.ruff.rs/docs/
- charliermarsh 4y agoYeah that's right, we use the RustPython parser and lexer internally. Works great! And it's actually seen a bunch of improvements over the past few months -- we contributed back some CPython compatibility improvements and a couple new language features (parenthesized with statements, match statements, *except support).
- DeathArrow 4y agoNow it would be cool to see a Rust compiler written in Python, to close the circle.
- Yoric 4y agoPython has many qualities, but I wouldn't use it to write a compiler. That being said, I've seen much worse languages being used to write compilers, so... who knows?
- baq 4y agoPython has really nice lexer and parser libraries. It isn’t Haskell or ML but it works. It is slow as expected.
- Yoric 4y agoLexing and parsing aren't the scary part. I'm much more nervous about maintaining invariants as you implement compilation passes, especially in the absence of powerful pattern-matching.
- nilsbunger 4y agoAt first I thought this would be about performance, but now I think it's not. Their github.io page mentions interesting use cases though: > RustPython can be embedded into Rust programs to use Python as a scripting language for your application, or it can be compiled to WebAssembly in order to run Python in the browser
- Waterluvian 4y agoThis is what Rust game engines need. Some sort of sensible scripting language to do the game logic in.
- nilsbunger 4y agoLua has been used a lot for that kind of embedding in the past. I think because it's easy to move data between Lua and C? Python would certainly be a big step up but I don't know how hard that kind of integration is.
- Waterluvian 4y agoLua is a great language for game scripting. Perhaps that’s a better option.
- 0xffff2 4y agoWhy is it such a great language? As a developer entirely divorced from the gaming industry aside from occasionally dabbling in modding, lua has always seemed completely inscrutable to me when I have encountered it.
- Tyr42 4y agoThe C API is pretty straightforward and has a stack object to avoid lots of reference counting you would need with Python's c API. It's also a tiny codebase which is easy to vendor and ship.
- 4y ago
- znpy 4y agoI really hope they didn’t also port the gil too
- Alifatisk 4y agoI have some good news for you
- amelius 4y agoThe problem is that if you don't port the GIL, you can't port the C API, and thus you can't support most of the Python library ecosystem.
- mkoubaa 4y agoPorting the GIL is trivial compared to what all it would take to support the C API
- smcl 4y agoAre there really that many Python libraries implemented in C? Genuine question - I thought there were a few very high profile ones, but I always figured there weren't many (like 10% or less, total finger-in-the-air guess)
- randlet 4y ago"there were a few very high profile ones" The absolute number (or fraction) implemented in C is less important than the high profile ones. numpy/scipy alone are used in a huge number of projects so if you can't support those two libraries alone it's pretty much a non starter.
- smcl 4y agoOh I’m not downplaying them, if they were small in number and minor then I’m sure CPython would’ve happily dumped the GIL already.
- bloblaw 4y ago
- nu11ptr 4y agoInterested to know how well their JIT works and in what scenarios they see improvement
- v3ss0n 4y agoFor jitted python, check PyPy.org
- youknowone 4y agoRustPython doesn't have working JIT in the common meaning of JIT yet.
- eigenvalue 4y agoCool project. But I think any project like this, if they want to get real world adoption with the masses, should create a "batteries included" distribution similar to Anaconda that comes bundled with all the most popular libraries, where everything has been tested and known to work. Something like Anaconda without a GIL that just worked and had much better multi-threading support would be amazing.
- beecafe 4y ago[dead]
- capableweb 4y agoAnaconda is a for-profit venture, while RustPython seems to be a loosely collected group of FOSS developers wanting to hack on a Python interpreter in Rust. Building a "batteries included" distribution seems like a noble goal but might be a bit too much to chew at the current stage.
- eigenvalue 4y agoFor sure, but I don't think it has to be as complete or stable/reliable as Anaconda. Just something that bundles the most popular and important libraries, like requests, httpx, pandas, pytorch, sqlalchemy, fastapi, etc. Stuff that most Python devs are going to want or need, but which they probably won't be willing to spend too much time hacking to get to work with a non-standard Python implementation. Without that, I think it gets relegated to some Fibonacci.py type examples that, while cool, aren't of much use to most devs.
- alfalfasprout 4y agoconda-forge isn't. And the conda tooling is not "for profit". It's only the anaconda distribution that is.
- capableweb 4y agoYes, conda and miniconda is FOSS while Anaconda is not. I don't think I said something that is not correct.
- pipeline_peak 4y agoHN needs a new tab for titles that end in written in Rust
- lpapez 4y agoWhy don't you write a 10 line JS Userscript for yourself and make this a reality?
- Arnavion 4y agoNot even 10 lines. https://news.ycombinator.com/item?id=31091542 https://news.ycombinator.com/item?id=31091542
- Dylan16807 4y agoLess than 10 lines, but only does half the job. Where's the new tab that has only Rust stories?
- lpapez 4y agoLeft as an exercise for the inquisitive reader :)
- returningfory2 4y agoOne of the awesome things about Rust is how easy it is to build Rust projects. Looking at the readme, it's a simple `cargo run` to compile and run the project. Versus CPython where you have to worry about Makefiles and configuration and third party libraries in your system.
- codetrotter 4y agoFor a lot of Rust projects this is true. But if they interface with external libraries then some extra steps may be needed. Not to take away from the excitement. I love Rust and cargo is great.
- nrabulinski 4y agoExtra steps i.e. install pkg-config and said library on your system
- codetrotter 4y agoIt’s gonna depend on what kind of system you use and what kind of external things are needed. It can be more involved, it can be less involved. For example https://github.com/servo/servo https://github.com/servo/servo seems a bit involved to set up for development but at the same time not too bad either.
- javier2 4y agoI kind of hate Rust, but I do think cargo looks great!
- xcdzvyn 4y agoThat's what Nix is for ;)
- noloblo 4y agoYes rust has user friendly tooling that's far superior to pip poetry pipenv and venv etc Since rust Python can theoretically access Tokio does it make it avoid the cpython gil issues