14 ms·
A Python Interpreter Written in Rust
- fxfan 8y agoI don't get the wasm craze in the rust crowd. There is no popular language that cannot be compiled to wasm. It just feels sad to spend time there.
- lukeqsee 8y agoI think the point is Rust is a very good language in its own right. The fact it compiles to WASM is a great feather in the cap. They are able to deliver a demo in the browser for something that would normally require downloading and compiling. I think that’s pretty cool to show for a project people wouldn’t normally be able to try out with such low barrier of entry.
- mjw1007 8y agoIf this ended up making it possible to run Python in the web browser, that might be interesting to a fair number of people. (There are advantages to running the same language on the server and client, and there's plenty of server-side web application Python code out there.)
- soperj 8y ago>There are advantages to running the same language on the server and client I see this said quite often. What are they?
- nsstring96 8y agoSaving in developer salaries is the only tangible one. You could argue there is less context switching for developers between languages, but I’ve never actually heard that from someone who builds the apps.
- Zelmor 8y agoTo make sure the same bootcamp hacks can work on your backend and frontend, neither of which they fully understand.
- mjw1007 8y agoIf you have a traditional server-rendered web application and you want to add a little bit of scripting on the client side, it can be nice to be able to share pieces of domain-specific code (for example, you might want the client and server to have the same notion of what a well-formed stock code looks like). In more sophisticated systems, people sometimes like the client to "optimistically" do the same processing as it expects the server to do, so it can update its display more quickly. Or if you have a mostly client-side application which builds up some fancy widget tree, sometimes people like to have the server do the same rendering as the client would so that it's there on initial page load, or so that search engines can see it.
- zoul 8y agoYou often have the same data types on the server and the client for data exchange. When both sides use the same language, you only write these types once and it’s impossible for the client and server implementations to differ.
- coldtea 8y agoNot having to learn 2 languages and duplicate efforts for starters...
- aaaaaaaaaaab 8y ago>learning TWO languages Oh! The horror!
- tylerl 8y agoYou mean like repl.it or skulpt or ... actually there's a few different option with different features, but doing python in the browser has been a thing for many years.
- half-kh-hacker 8y agoRunning Python, client-side in the browser.
- StavrosK 8y agoThere are already various options, many using Emscripten.
- bbernard 8y agoThere's already a client-side implementation of Python in JavaScript. See: https://brython.info/ https://brython.info/ But it might not give you the sort of compatibility between the client and server code that you're looking for.
- geofft 8y agoI think it's less "the Rust crowd is excited about wasm" and more as "there is a crowd which is excited about doing certain things with languages and software, and Rust and wasm are both exciting means for those ends." Perhaps a good approximation is, they're both tools for using high-level programming techniques with high performance for domains that were previously constrained in terms of what languages you could use.
- nicoburns 8y agoRust is particularly suited to WASM due to the lack of GC and runtime (which tend to be quite large, and have to be downloaded in a web context). And due to it being able to achieve very high performance (which is the whole point of WASM). The only other languages that really compete here are C and C++, and they don't have the great, easily installable library ecosystem that Rust has, and they aren't very accessible to JavaScript developers). Personally I'm more excited about other uses of Rust, but I can see why people are excited about Rust and WASM.
- mhh__ 8y agoYou're talking about web technologies, and needing to download a runtime is the problem? C++ isn't accessible but rust is? Only because they can't be bothered to learn: It complicated but its really not that hard. Plug: D can happily compile to wasm
- nicoburns 8y agoWell yes, the need to download a new runtime for every app you run is a problem if thst runtime is several megabytes. C++ isn't accessible to web developers compared to Rust. It's not just that I can't be bothered to learn, it's that I'm scared of all the security vunerabilities and memory corruption bugs I will write while I'm learning. And wjy put in that effort when I can learn Rust more easily, and get the ongoing benefits of my code being safe and reliable. True, D also competes here. I should probably have put D on the list. Although my understanding is that a large part of the D ecosystem still relies on GC.
- mhh__ 8y agoMost runtimes are fairly small, and most issues could be done away with by either the browser storing common runtimes (Search for x, then checking hashes etc.) or by compiling statically against the parts you actually use. This is still not conclusive, as the runtimes will probably have to be significantly modified (at the ABI/System level, so around the edges) given that they will have to get memory from the browser etc. This leaves much room for WASM specific optimisation, especially given that the actual (let's say) garbage collector implementation is probably quite small compared to the code used to interface to it. Writing C++ defensively (i.e. Do what the guidelines tell you, Preach Andrei and Bjarne etc), and using sanitizers cleans up a huge amount of C++ code.
- xkapastel 8y agoMost of the popular languages are not making a serious effort to compile to WebAssembly, though. It's totally reasonable for Rust to focus on.
- FrankDixon 8y agoHow nice, even a wasm interpreter, that's dandy
- lykr0n 8y agoNeat. Was able to clone the repo, run cargo run, and drop into a python shell. Doesn't seem like can do much right now, but I really like the idea. >>>>> a = [1,2,3] >>>>> a[2:] [3] >>>>> a[1:] [2, 3] >>>>> fh = open('~/.ssh/id_rsa.pub', 'r') thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: RefCell { value: [PyObj instance] }', src/libcore/result.rs:999:5 note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace. What would be really cool if this could one day be like Nuitka- but in rust. Write in python, compile into Rust. Maybe even support inline Rust like cPython supports inline C.
- Ralfp 8y ago> cPython supports inline C First time I am hearing this. Can you share an example?
- lykr0n 8y agohttps://cffi.readthedocs.io/en/latest/ https://cffi.readthedocs.io/en/latest/ & https://docs.python.org/3/extending/extending.html https://docs.python.org/3/extending/extending.html There also is https://github.com/rochacbruno/rust-python-example https://github.com/rochacbruno/rust-python-example which is something I want to look into. As for an example, I'm using Snappy right now, so here you go: https://github.com/andrix/python-snappy/blob/master/snappy/snappy_cffi_builder.py https://github.com/andrix/python-snappy/blob/master/snappy/s... & https://github.com/andrix/python-snappy/blob/master/snappy/snappy_cffi.py https://github.com/andrix/python-snappy/blob/master/snappy/s... It still requires you to compile the C part- which is why sometimes you need GCC when you're doing a pip install.
- stochastic_monk 8y agoI’ve done a lot of interfacing with C and C++. pybind11 [0] has been the easiest and most effective for me to use. It targets C++, but it’s easy enough to wrap C code with it. Cython brings along a lot more bookkeeping code and overhead in my experience. cffi isn’t bad, but it’s not as flexible/expressive. [0] https://github.com/pybind/pybind11 https://github.com/pybind/pybind11
- gonational 8y agoNow this is something I will donate money to… Python with Rust as its foundation sounds like the best idea ever. I’m curious to know whether or not it would be possible (or reasonable) to eventually get the same or better performance as CPython.
- Waterluvian 8y agoCPython is really quite slow by design. The reference implementation is meant to be obvious and pretty easy to interoperate with C. I think a RustPython implementation would be pretty cool. You could definitely take that opportunity to worry about performance more than CPython does while also worrying about interoperability more than PyPy does. Or I'm missing your point and you're suggesting a drop-in replacement for CPython that supports all the same C-based libraries as CPython does.
- gonational 8y agoAwesome! No, that was my point basically, although I could imagine something like this shipping with some basic libraries and package support, and then having a similar ecosystem to the current python ecosystem.
- lykr0n 8y agoFrom my limited experience with Rust, having support for C-based libraries shouldn't be too hard to do- as Rust (like Go) has support for embedding C. I'm not sure how portable the solution would be- I would guess you would need to compile the VM with the modules you want baked in. I think not having support for C modules would hamper long term adoption. I would absolutely love to adopt this for my stuff, but off the top of my head- I use uvloop and confluent-Kafka, both of which are largely written in in C. Moving away from those would be hard-ish.
- weberc2 8y agoCalling into and out of C (whether dynamically or statically linked) isn't the hard part; the hard part is implementing the somewhat arbitrary interface against which C extensions are written in a way that is functionally compatible with CPython without losing the advantages of not being CPython.
- amelius 8y agoCool. Rust with a garbage collector :)
- 4gotunameagain 8y agoSerious question with no irony at all. But why? How is this useful?
- ivanbakel 8y agoIt interprets Python - what about that doesn't appear useful to you?
- pm90 8y agoSerious Question: does everything need to be useful?
- blocked_again 8y agoI think everything in the Universe is useful for some purpose. Can you name anything that is completely useless?
- saghm 8y agoApparently the word "useless" itself, from what you're saying
- blocked_again 8y agoIf word "useless" is actually useless then the word "useless" cannot be useless
- saghm 8y agoOops, Russell's paradox strikes again
- coldtea 8y agoMost of the universe is completely useless. And I mean unbelievably vast quantities of space and matter.
- hathawsh 8y agoThis is wonderful. This could become the best way to move Python projects to Rust: initially just run on the RustPython interpreter, but then optimize low level routines in Rust. In 15 years I wouldn't be surprised if this or something like it surpasses CPython in popularity. Still, no discussion about Python implementations is complete without some mention of the infamous GIL (global interpreter lock). :-) CPython has it, Pypy doesn't (I think) (EDIT: yes it does), Jython doesn't, etc. What is the GIL plan for RustPython?
- lykr0n 8y agoPyPy has the GIL.
- schmichael 8y agoPyPy does have a GIL: http://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why http://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-... Jython, as you state, does not: http://www.jython.org/jythonbook/en/1.0/Concurrency.html http://www.jython.org/jythonbook/en/1.0/Concurrency.html
- znpy 8y agoJython is however stuck with Python 2.x compatibility only as far as I know (I'd be happy to be proven wrong).
- mooreds 8y agoJython 3.x development is "in progress": https://wiki.python.org/jython/JythonFaq/GeneralInfo https://wiki.python.org/jython/JythonFaq/GeneralInfo Last commit was over a year ago: https://github.com/jython/jython3 https://github.com/jython/jython3
- orf 8y agoIf anyone is at FOSDEM this will be presented tomorrow in the rust room. Can't wait!
- tyingq 8y agoIs there anything about Rust that would make it hard to use existing python extensions that are written in C? I see the FFI docs, so it seems like it would be fine, but have no Rust experience.
- biesnecker 8y agoFwiw, nothing about rust (probably) but a whole lot about CPython's implementation. Extensions can and do poke around in object internals and will break if the internals don't work the way the extension expects them to (that is, the way CPython works). This has long been PyPy's big stumbling block (though it seems to have gotten better).
- tasubotadas 8y agoDoes it have a GIL? If not, it's already better than CPython :-D
- kodablah 8y agoNeat. Now keep a hash of the types/info for instructions at runtime, and when a set is called enough times and appears reasonably pure/simple, JIT it with cranelift (akin to a tracing JIT).
- bibyte 8y agoThis is awesome. I hope more features from CPython are ported. It looks a little bare bones right now. Is there any chance it will gain full Python compatibility ?
- FartyMcFarter 8y agoLet's wait for it to support dictionaries first, before we start dreaming about full compatibility. edit - it looks like it supports dictionaries with string keys, but not with integer keys.
- systems 8y agoI wish, I wish, I wish .. something like this is done for Tcl/Tk I like Tcl the language a lot, and I love the idea of two language systems One high level for scripting Tcl One low level for high performance commands and parts Rust You can of course do that today, using Tcl and C But .. well C is no Rust
- jadbox 8y agoWhy do you like Tcl? I worked at a place that used it for their general purpose web servers... and most of their programmers complained that it was like using Fortran for modern web development. That's the only experience I've had with Tcl professionally in the field.
- systems 8y agoI think you really have to like the idea of 2 language system, to appriciate the potential of Tcl Having one team create complex tool using Rust, or Ocaml or Go or C++ And another team more domain oriented, created UIs and Interfaces and Script using Tcl (a language that is simple enough , yet powerful enough) The one size fit all language, dont exist, I think, a two language team, is very good option A (stretched) example, is SQL ..and the DBMS Expert programmers enrich the DBMS using C++ or whatever And domain experts use SQL and Procedureal-SQL to solve business problems Tcl as a universal declarative language, is an idea I like
- earenndil 8y ago> The one size fit all language, dont exist Lisp? (or perhaps lisp is an n language system)
- systems 8y agoI think any one size fit all language, will have to have optional typing, and be both compile and interpreted i dont know of any language, that have both, and was successful Not sure if clojure specs, achieve the same outcome of optional typing, so maybe you have a point :)
- chadrs 8y agoI stopped paying attention to the Rust parsing ecosystem for a while, curious how LALRPOP compares to nom/pest/combine and if something about python's grammar led to the choice.
- losvedir 8y agoTotally different use cases. nom/pest/combine are parser combinators, where you stitch the functions together yourself. LALRPOP is more in the vein of yacc where you specify the grammar and it generates the Rust parsing code for you in a build step.
- surajrmal 8y agopest isn't a parser combinator. It also uses a grammar file.
- losvedir 8y agoOh, wow! Thanks for the correction. I had somehow missed that. I'm working on a rust parser of Elixir myself, and using LALRPOP for it. But looks like I'll have to check out pest, too. LALRPOP has the advantage that it takes a BNF grammar, which the Elixir parser in erlang includes, so it's a somewhat straightforward translation. Not too sure how similar it is to a PEG grammar that pest takes.
- ltratt 8y agoLALRPOP is an LR parser, which is a very different formalism to PEG: it's easy to write a grammar in either that's difficult/impossible to express in the other. If you'll permit the immodesty, another Rust parser is lrpar (https://crates.io/crates/lrpar https://crates.io/crates/lrpar) which is a more direct drop-in replacement for Yacc, but with better error recovery. [Note: I'm biased because I wrote parts of lrpar and the wider framework, grmtools, it's a part of.]
- nostrademons 8y agoCurious what the general story on Rust <=> Python interop is like now (beyond writing a Python interpreter in Rust). I'd checked out rust-cpython a couple years ago, and I see there's now PyO3, but I just looked at some code samples and it seems like they're a long way away from the convenience that you get with CFFI + C code, boost::python, or even SWIG. Anyone have experience writing Python extensions in Rust or embedding a Python interpreter into a Rust program? Could you comment on how hard the process was and how robust the result is?
- yuchi 8y agoI want to create the Ramon’s Law (as a corollary of Atwood’s Law): > Anything that can be Written in Rust, will Eventually be Written in Rust Please go on, cite me, consider it «Attribution, Share-alike»
- sramsay 8y agoMaybe, "Anything that can be Written in Rust Will Eventually be Suggested, and then Celebrated with Great Fanfare on HN as the Obvious Future of Reality." Atwood's Law was a prediction about the future that was based on something that had already happened (zillions of apps and libraries rewritten in JS). I'll quote your law when we have anything like comparable evidence that this is happening with Rust.
- yuchi 8y agoFair enough.
- cs702 8y agoYes. In particular, I like the potential for: * using Rust's borrow-checking to develop new lightweight/shared-memory multiprocessing tools for Python (think "import SharedMemoryPool from multiprocessing") without having to mess with the GIL, so as to maintain compatibility with existing libraries; * using Rust's type inference on Python code for applications in which type safety is highly desirable; and * compiling Python code for speed, targeting all architectures and platforms supported by Rust (e.g., WebAssembly).
- Thaxll 8y agoAnother amateur project that will go no where, look back at this project project 5 years from now, nothing will run on it / be maintain.
- MR4D 8y agoGiven all the great ideas here, maybe it’s time to fork Python. Ok, so maybe a close derivative as opposed to a true fork. Given the language names involved (Rust & Python), I’d like to suggest “Copperhead” as a name for it.
- sametmax 8y agoForking implies keeping the C code. I think a rewrite in rust is more future proof: we benefit from a safer, more modern language to implement it, which comes with cargo, and hence, the potential of an hybrid python/rust toolchain and dev plateform.
- mlevental 8y agocopper doesn't rust (it does patina though)
- bakery2k 8y agoHow would your Python derivative differ from RustPython?
- uhrlocher 8y agohttps://www.theatlantic.com/magazine/archive/2017/04/why-is-silicon-valley-so-awful-to-women/517788/ https://www.theatlantic.com/magazine/archive/2017/04/why-is-... https://www.newsweek.com/2016/06/17/silicon-valley-takeover-468182.html https://www.newsweek.com/2016/06/17/silicon-valley-takeover-... https://medium.com/@preethikasireddy/why-im-leaving-silicon-valley-72919edb3297 https://medium.com/@preethikasireddy/why-im-leaving-silicon-... https://getlighthouse.com/blog/silicon-valley-bad-managers/ https://getlighthouse.com/blog/silicon-valley-bad-managers/ https://www.philanthropy.com/article/A-Star-Performer-Created-a/243159 https://www.philanthropy.com/article/A-Star-Performer-Create... https://www.theglobeandmail.com/report-on-business/international-business/a-valley-too-far-is-the-toxic-culture-of-the-tech-world-beyond-repair/article35610645/ https://www.theglobeandmail.com/report-on-business/internati... https://www.geekwire.com/2018/emily-chang-brotopia-silicon-valley-companies-can-tackle-toxic-culture/ https://www.geekwire.com/2018/emily-chang-brotopia-silicon-v... https://www.entrepreneur.com/article/297481 https://www.entrepreneur.com/article/297481 https://thenextweb.com/insider/2017/03/11/bro-culture-poisoning-silicon-valley/ https://thenextweb.com/insider/2017/03/11/bro-culture-poison... https://www.quora.com/Is-the-culture-in-the-Silicon-Valley-becoming-increasingly-toxic?share=1 https://www.quora.com/Is-the-culture-in-the-Silicon-Valley-b... https://www.vanityfair.com/news/2016/06/ellen-pao-memoir-silicon-valley-toxic-culture https://www.vanityfair.com/news/2016/06/ellen-pao-memoir-sil... https://www.recode.net/2018/2/5/16972096/emily-chang-brotopia-book-bloomberg-technology-culture-silicon-valley-kara-swisher-decode-podcast https://www.recode.net/2018/2/5/16972096/emily-chang-brotopi... https://www.glassdoor.com/Reviews/Silicon-Valley-Community-Foundation-Reviews-E331546.htm https://www.glassdoor.com/Reviews/Silicon-Valley-Community-F... https://www.wired.com/story/everyone-hates-silicon-valley-except-its-imitators/ https://www.wired.com/story/everyone-hates-silicon-valley-ex... https://www.intelligencesquaredus.org/debates/silicon-valley-has-lost-its-soul https://www.intelligencesquaredus.org/debates/silicon-valley... https://yourstory.com/2017/07/silicon-valley-bro-culture/ https://yourstory.com/2017/07/silicon-valley-bro-culture/ https://www.wsj.com/articles/the-quiet-efforts-to-battle-silicon-valleys-bro-culture-1541943938 https://www.wsj.com/articles/the-quiet-efforts-to-battle-sil... https://thebolditalic.com/a-source-of-sorrow-in-silicon-valley-1a81ccd80ac2?gi=bc741a45779e https://thebolditalic.com/a-source-of-sorrow-in-silicon-vall... https://www.bizjournals.com/sanfrancisco/news/2019/01/25/is-silicon-valley-the-real-villain-in-the-theranos.html https://www.bizjournals.com/sanfrancisco/news/2019/01/25/is-... https://www.rawstory.com/2018/07/silicon-valley-become-next-detroit-struggles-survive-toxic-mess-created/ https://www.rawstory.com/2018/07/silicon-valley-become-next-... https://builttoadapt.io/so-you-want-to-build-a-silicon-valley-like-software-developer-culture-focus-on-the-people-809ddf4c426a?gi=af0b0c66fd4f https://builttoadapt.io/so-you-want-to-build-a-silicon-valle... https://www.youtube.com/watch?v=h3-bMDaMhLg https://www.youtube.com/watch?v=h3-bMDaMhLg https://www.theguardian.com/world/2017/mar/01/silicon-valley-sexual-harassment-startups https://www.theguardian.com/world/2017/mar/01/silicon-valley... https://www.indiewire.com/2018/07/silicon-valley-actress-alice-wetterlund-t-j-miller-co-stars-enabled-bullying-1201985168/# https://www.indiewire.com/2018/07/silicon-valley-actress-ali...! https://www.theneweconomy.com/technology/silicons-sexist-brogrammer-culture-is-locking-women-out-of-tech https://www.theneweconomy.com/technology/silicons-sexist-bro... https://news.ycombinator.com/item?id=17051800 https://news.ycombinator.com/item?id=17051800 https://www.losaltosonline.com/news/sections/schools/210-school-features/55819- https://www.losaltosonline.com/news/sections/schools/210-sch...
- sametmax 8y agoIf anybody had any doubt about how welcome the project is... Since this has been posted to HN, the repo got 8 new PR.
- Animats 8y agoIt's a naive interpreter. The source is parsed into a tree and then flattened into byte code. All values are stored in PyObjectPayload structures. Each operation has a function. Here's "hex": fn builtin_hex(vm: &mut VirtualMachine, args: PyFuncArgs) -> PyResult { arg_check!(vm, args, required = [(number, Some(vm.ctx.int_type()))]); let n = objint::get_value(number); let s = if n.is_negative() { format!("-0x{:x}", n.abs()) } else { format!("0x{:x}", n) }; Ok(vm.new_str(s)) } There's a big dispatch table, generated at compile time, and an interpreter loop. Just like you'd expect. It's useful if you happen to need a Python implementation to embed in something and want something cleaner than CPython. Probably slower, though.