8 ms·
A Python interpreter rewritten in Rust, that can run pip
- txdv 6y agowhat are the latest benchmark numbers of rustpython?
- cylde_frog 6y ago.
- rightFootOnly 6y ago“RustPython is a Python interpreter written in Rust. RustPython can be embedded into Rust programs to use Python as a scripting language for your application, or it can be compiled to WebAssembly in order to run Python in the browser.”
- kzrdude 6y agoThey really have a project going, with 2000 PRs merged it's looking good. (Edit: oh they use some code I've written too, that's humbling)
- deleted 6y ago[deleted]
- Twirrim 6y agoWhy not? Someone thought it might be an interesting challenge, and so they did.
- deleted 6y ago[deleted]
- jnwatson 6y agoIt is a reasonably complicated Python script, so is a nice milestone of progress.
- Bostonian 6y agoA single period means is interpreted as "Why?"?
- yudlejoza 6y agoWow. Looks like Rust is winning the catfight against Go, D, Julia, C++20. Need to grab more popcorn.
- dunefox 6y agoAnd why would that be?
- coldtea 6y agoThere's no "catfight". Julia targets a different audience. Go, for the most part, too. C++20 is not going anywhere, and C++ is used more than ever. D hasn't been a contender for 10+ years. It could technology wise, but it never gained traction. All of Rust, Go, Julia, and C++ do well. (Heck, C even continues to do well. Rust is not really eating into it). It remains to be seen how the Mozilla situation (abhorent leadership, firings of Rust team people) will affect Rust.
- ransom_rs 6y agoThis uses [lalrpop](https://github.com/lalrpop/lalrpop https://github.com/lalrpop/lalrpop) under the hood I believe, which is a pretty awesome parser generator library for Rust.
- popotamonga 6y agoNice, long go the days (20 yrs ago) i had to make a JS transpiler with lex+yacc. The pain
- sitkack 6y agohad to? or got to? Sounds great, please tell us more.
- popotamonga 6y agoi wanted to add lambdas to JS, this was back in 2001 i think, so i had this compiler from my compiler class, changed it to parse JS (+ my extra features like the ()=>{} syntax) and generate an AST in XML (instead of emitting bytecode), then i made a python script that took the XML and generated minified JS from it.
- sitkack 6y agoNeat! I did something similar, not quite as big, but I was on a version of Python (Jython) that didn't yet have generator expressions, so I made a generator expression evaluator, gg("x for x in it") that returned a running generator. I was about a week away from shipping an ETL tool to the customer and this allowed me to hit that deadline. I love the AST in XML approach.
- pansa2 6y ago> LALRPOP in fact uses LR(1) by default (though you can opt for LALR(1)), and really I hope to eventually move to something general that can handle all CFGs (like GLL, GLR, LL(*), etc). Seems overkill for a language whose grammar is LL(1)?
- thundergolfer 6y agoSlightly misleading title. "A Python interpreter written from scratch in rust can now run 'pip'" is more accurate. The standard pip does not use Rust.
- dang 6y agoOk, we've changed the title from "Now pip runs on a Python interpreter rewritten from scratch in rust". Hopefully less misleading. Thanks!
- liquidify 6y agoThis is really cool.
- syrusakbary 6y agoThis is awesome! As fan of Rust and Python as I am I can't be more excited for this... great work everyone!!!
- amelius 6y agoDoes it include a concurrent garbage collector? And does it solve the infamous GIL (global interpreter lock) problem?
- corytheboyd 6y agoI’m genuinely curious why people want to work around the GIL in languages like Python and Ruby. I very much do not want my simple interpreted language to inherit the complexity of true concurrency. Why not just use a different language?
- afarrell 6y agoIf a company has 200+ developers already working on a 10-year-old codebase in the language, then the word "just" has a lot of heavy lifting to do in that question.
- duckerude 6y agoThe GIL doesn't really let you escape the horrors of concurrency, at least in CPython. Threads can't execute at the same time, but they can interleave in inconvenient ways.
- pansa2 6y ago> Why not just use a different language? I’ve asked myself this question (in a different context) and have concluded that, in general, developers are incredibly resistant to learning and using new languages. For example: if they weren’t, then server-side JavaScript would never have become popular.
- coldtea 6y ago>I’ve asked myself this question (in a different context) and have concluded that, in general, developers are incredibly resistant to learning and using new languages. You make it sound like a personal psychological failing. In real life, using the same language has business benefits: re-use of the same libraries, no duplicated business logic that needs to be used on both client/server, no need for training to the new language, no need for mental context switch due to the language change when working on the client or server part (aside from the essential domain knowledge of client vs server), less/common tooling infrastructure, and so on...
- deleted 6y ago[deleted]
- ridiculous_fish 6y agoHow are interpreters designed in Rust? What parts can be safe and what can't be? What is the internal representation of a Python object? In particular how do you mutate a Python object while there may be other references to it - is it UnsafeCell, is there some checking? How does the GC work? The GC needs to access and perhaps mutate all Python objects. In C++ a common technique is to make a linked list of scopes, stored in native stack frames; I'm not sure how that can be done in safe Rust.
- steveklabnik 6y agoI was curious too, so I poked around! I had never looked at this codebase before this comment, so, I may have some things wrong. > How are interpreters designed in Rust? What parts can be safe and what can't be? Depends on the design. Different interpreters are different. A quick glance at the source code seems to show unsafe in: * A data structure called "boxvec" * Some calls to unreachable_unchecked, get_unchecked, and the like. Wonder what the overheads were here. * Some lock/mutex implementations * A few unsafe functions in the jit. They use cranelift. Bet there's a bunch in there too. Usually this is the classic thing that needs unsafe in interpreters. * The python object data structures; they do some stuff to have a smaller representation * some FFI code in the standard library, calls directly to libc > What is the internal representation of a Python object? https://github.com/RustPython/RustPython/blob/master/vm/src/pyobjectrc.rs#L12-L16 https://github.com/RustPython/RustPython/blob/master/vm/src/... > How does the GC work? Traditionally, Python uses refcounting + cycle detection. A quick glance at their issue tracker implies that as of last month, they had recounting but no cycle checks yet. > In C++ a common technique is to make a linked list of scopes, stored in native stack frames; I'm not sure how that can be done in safe Rust. Looks like they have a RefCell<Vec<FrameRef>>, instead.