14 ms·

At first I thought this would be about performance, but now I think it's not. Their github.io page mentions interesting use cases though: > RustPython can be

At first I thought this would be about performance, but now I think it's not. Their github.io page mentions interesting use cases though: > RustPython can be embedded into Rust programs to use Python as a scripting language for your application, or it can be compiled to WebAssembly in order to run Python in the browser

Waterluvian 4y ago
This is what Rust game engines need. Some sort of sensible scripting language to do the game logic in.
- nilsbunger 4y ago
  Lua has been used a lot for that kind of embedding in the past. I think because it's easy to move data between Lua and C? Python would certainly be a big step up but I don't know how hard that kind of integration is.
  - Waterluvian 4y ago
    Lua is a great language for game scripting. Perhaps that’s a better option.
    - 0xffff2 4y ago
      Why is it such a great language? As a developer entirely divorced from the gaming industry aside from occasionally dabbling in modding, lua has always seemed completely inscrutable to me when I have encountered it.
      - Tyr42 4y ago
        The C API is pretty straightforward and has a stack object to avoid lots of reference counting you would need with Python's c API. It's also a tiny codebase which is easy to vendor and ship.
      - agentultra 4y ago
        Its obscenely easy/consistent memory layout. It makes embedding it into another application straight-forward and easy to manage. Extending the host interface to the embedded scripts is very simple because of this. It's also serialize-able to a degree. Python is also a decent glue language but creating a Python interpreter is a bunch more work and the API is a bit harder to work with.
      - guipsp 4y ago
        LuaJIT is insanely good
      - elfprince13 4y ago
        I think the main attractions are that it has pretty minimal syntax + core libraries, does a good job binding to C FFIs, and the interpreter is fast.
      - schemescape 4y ago
        It’s a tiny but capable language that’s easy to embed, and can be surprisingly fast with LuaJIT.
      - kzrdude 4y ago
        They have a clean API for embedding. All Lua interpreter state is kept in a struct that you alloc. If you alloc two distinct structs, you have two completely independent Lua vms and you can run them on independent threads if you want to no problem, and so on. (Few language runtimes are clean in this way.)
        cozzyd 4y ago
        yes, python is basically impossible to embed in a multithreaded app.
      - outworlder 4y ago
        Inscrutable? The language is _tiny_ (and kept that way on purpose) Aside from stuff like metatables(which may require you to play around for a bit to understand their value), you can pick it up in a couple of hours. I'm not even kidding. So much so that you see people modifying code without even looking at documentation. https://www.lua.org/manual/5.4/ https://www.lua.org/manual/5.4/ https://learnxinyminutes.com/docs/lua/ https://learnxinyminutes.com/docs/lua/
    - v3ss0n 4y ago
      Lua is most hated lang
      - deleted 4y ago
        [deleted]
      - w0m 4y ago
        It seems reasonable in my ~300 line neovim config :D and yes; i know that's a ridiculous example that's not really comparable for anything significant.
      - zemo 4y ago
        do you work in games
      - outworlder 4y ago
        > Lua is most hated lang Impossible. Most developers outside the games industry don't even know it exists. ... and if they knew about it, they would use it much more.
    - bsder 4y ago
      Sorry, Lua is not simply not a great language. The first problem is "Which Lua?" Lua, by itself, is almost completely useless. You first need to compile/install a bunch of things in order to make Lua useful. This has the Perl problem that everybody uses a different dialect of Lua. The second part is that the Lua constructs for programming in the large are very weak. I believe that Adobe had a postmortem about this (Lightroom, I think?) And then there is the language, itself. For me, the 1-based indexing is death in this day and age. Sorry. Zero-based is the dominant ecosystem and not fitting into that is simply not acceptable.
      - tomodachi94 4y ago
        > You first need to compile/install a bunch of things in order to make Lua useful. How is this any different from pretty much any language out there? From a quick eyeball of RustPython's Cargo.toml[1], there are about 70 different dependencies which all need to be compiled. I haven't worked too much with Autoconf, but I am pretty sure CPython has quite a few dependencies. [1]: https://github.com/RustPython/RustPython/blob/main/Cargo.toml https://github.com/RustPython/RustPython/blob/main/Cargo.tom... > The second part is that the Lua constructs for programming in the large are very weak. This is deliberate because it forces you to use the tools you are given instead of reimplementing features that can already be implemented by using other primitives in Lua. > I believe that Adobe had a postmortem about this (Lightroom, I think?) I can't find this article. Has anyone else had any luck?
- dathinab 4y ago
  If experiences I heard from previous projects(1) (successfully) embedding cython are to go by python isn't necessary good choice for this. (1): e.g. awesome, blender
  - gpm 4y ago
    I'm curious what you've heard about them that suggests this?
    - dathinab 4y ago
      As far as I can tell the maintenance overhead to mapping all the available type from/to python as nicely usable python APIs is a major pain and not just because of cythons current implementation but of e.g. conceptually mismatch between "good" python and "good" C++/Rust/whatever implementations. I.e. you could say that you have a paradigm mismatch between the underlying implementation (of whatever you extend) ad the python API which makes the already hard problem of keeping an API stable even harder. Furthermore tooling to work in such contexts (as a API consumer) often is very limited or does not exists, but the API producer seldomly has the time/money to produce such tooling either. Lastly there is the remaining problem of Python by itself being really slow, for it's main use-cases this often doesn't matter as e.g. the scientific computations are run by native extensions python delegates the work, too. But this would mean that you have to also support native extensions support or hope that no extension needs to be really fast (not needing to be really fast is surprisingly often the case) or similar solutions. In turn choosing something which has a good JIT, AOT or similar compilation has benefits.
  - Tade0 4y ago
    I think the worst thing about Python and Blender was not necessarily the choice of language, but the frequent API changes and not having anything close to an IDE-like environment. Not only did old examples often just not work, there was no serious code completion to facilitate creating anything from scratch.
    - pzone 4y ago
- brucethemoose2 4y ago
  Civ4 used Python scripting, and it was a horrendous performance bottleneck :/
  - wyldfire 4y ago
    Python is the most incredible language because everything's dynamic. But it comes at an enormous cost - you can't make many assumptions or optimizations easily. It has tremendous utility for many applications. It's this utility and the wide body of existing software packages which makes it so tempting to use as a scripting engine. But it would be a bad move for any scripting that's in some critical region.
    - brucethemoose2 4y ago
      It would be nice to have an official, more performant statically typed/less dynamic subset of Python. Yeah we have mypy, but its not official and not integrated. And subsets that do have a performance benefit (like TorchScript) are niche and/or not sustainably supported.
      - wyldfire 4y ago
        Well -- other languages have shown up to try and fill this niche.
      - ben-schaaf 4y ago
        Not statically typed, but rpython is a statically analyzable subset of python that is used to write pypy: https://rpython.readthedocs.io/en/latest/rpython.html https://rpython.readthedocs.io/en/latest/rpython.html
        brucethemoose2 4y ago
        That looks very reasonable. Converting a Python codebase would be much easier than porting to a new language. Now all your imports, on the other hand...
  - 4y ago
- seurimas 4y ago
  I think the greatest idea for Rust scripting is just supporting WASM scripts. I've had some success toying with wasmer in bevy.
- fwip 4y ago
  You may like Rhai: https://crates.io/crates/rhai https://crates.io/crates/rhai
singularity2001 4y ago
Yeah the performance comparison doesn't look too great yet: https://rustpython.github.io/benchmarks https://rustpython.github.io/benchmarks Currently it's more about clarity and a nice clean code base, but it can only get faster from here
- andai 4y ago
  Any idea where the slowness is coming from?
  - intrepidhero 4y ago
    I've been playing with making a rust interpreter and reading other people's work. Here's an interesting blog post (not mine) on making a interpreter in rust match the speed of a c implementation. It seems folks often have to resort to unsafe rust to get the performance they want. I don't claim to fully understand that and am looking for more information myself. http://www.dannyvankooten.com/blog/2022/rewriting-interpreter-rust/ http://www.dannyvankooten.com/blog/2022/rewriting-interprete...
    - xiphias2 4y ago
      With webservers we already saw that the way to make new things fast in Rust is to use lots of unsafe Rust when needed, optimize the code, and after that think about the right abstractions that minimize the unsafe code surface.
  - tinco 4y ago
    Well there's no production class garbage collector in Rust yet, so in RustPython they're just using Rc's I think. That's one place to start. The other will probably be the quality of the JIT.
    - Yoric 4y ago
      Does Python have a GC these days?
zem 4y ago
surprised they didn't mention packaging a python project into a static executable, that sounds like a huge use case.
- youknowone 4y ago
  Thank you. That sounds like an important feature.
  - zem 4y ago
    to be clear i don't know if the project supports the feature out of the box; it just seems to me like something it would enable, and that i would have expected to see called out as a use case in the README.
    - Timothycquinn 4y ago
      Thanks for clarifying. Does not look like its there today but hopefully down the road it will.
al_be_back 4y ago
python on the browser is a good use-case, a kinda edge computing scenario where a python API responds with data and python code for client to post-process before display/acting-on.
ameliaquining 4y ago
Both of those things are already possible with CPython. For the former, I'd recommend PyO3; for the latter, pyodide. These projects take CPython as a dependency (though pyodide has to apply a few patches on top) and automate the annoying finicky FFI parts of getting it to talk to Rust or JavaScript.

Waterluvian 4y ago

This is what Rust game engines need. Some sort of sensible scripting language to do the game logic in.

nilsbunger 4y ago
Lua has been used a lot for that kind of embedding in the past. I think because it's easy to move data between Lua and C? Python would certainly be a big step up but I don't know how hard that kind of integration is.
- Waterluvian 4y ago
  Lua is a great language for game scripting. Perhaps that’s a better option.
  - 0xffff2 4y ago
    Why is it such a great language? As a developer entirely divorced from the gaming industry aside from occasionally dabbling in modding, lua has always seemed completely inscrutable to me when I have encountered it.
    - Tyr42 4y ago
      The C API is pretty straightforward and has a stack object to avoid lots of reference counting you would need with Python's c API. It's also a tiny codebase which is easy to vendor and ship.
    - agentultra 4y ago
      Its obscenely easy/consistent memory layout. It makes embedding it into another application straight-forward and easy to manage. Extending the host interface to the embedded scripts is very simple because of this. It's also serialize-able to a degree. Python is also a decent glue language but creating a Python interpreter is a bunch more work and the API is a bit harder to work with.
    - guipsp 4y ago
      LuaJIT is insanely good
    - elfprince13 4y ago
      I think the main attractions are that it has pretty minimal syntax + core libraries, does a good job binding to C FFIs, and the interpreter is fast.
    - schemescape 4y ago
      It’s a tiny but capable language that’s easy to embed, and can be surprisingly fast with LuaJIT.
    - kzrdude 4y ago
      They have a clean API for embedding. All Lua interpreter state is kept in a struct that you alloc. If you alloc two distinct structs, you have two completely independent Lua vms and you can run them on independent threads if you want to no problem, and so on. (Few language runtimes are clean in this way.)
      - cozzyd 4y ago
        yes, python is basically impossible to embed in a multithreaded app.
    - outworlder 4y ago
      Inscrutable? The language is _tiny_ (and kept that way on purpose) Aside from stuff like metatables(which may require you to play around for a bit to understand their value), you can pick it up in a couple of hours. I'm not even kidding. So much so that you see people modifying code without even looking at documentation. https://www.lua.org/manual/5.4/ https://www.lua.org/manual/5.4/ https://learnxinyminutes.com/docs/lua/ https://learnxinyminutes.com/docs/lua/
  - v3ss0n 4y ago
    Lua is most hated lang
    - deleted 4y ago
      [deleted]
    - w0m 4y ago
      It seems reasonable in my ~300 line neovim config :D and yes; i know that's a ridiculous example that's not really comparable for anything significant.
    - zemo 4y ago
      do you work in games
    - outworlder 4y ago
      > Lua is most hated lang Impossible. Most developers outside the games industry don't even know it exists. ... and if they knew about it, they would use it much more.
  - bsder 4y ago
    Sorry, Lua is not simply not a great language. The first problem is "Which Lua?" Lua, by itself, is almost completely useless. You first need to compile/install a bunch of things in order to make Lua useful. This has the Perl problem that everybody uses a different dialect of Lua. The second part is that the Lua constructs for programming in the large are very weak. I believe that Adobe had a postmortem about this (Lightroom, I think?) And then there is the language, itself. For me, the 1-based indexing is death in this day and age. Sorry. Zero-based is the dominant ecosystem and not fitting into that is simply not acceptable.
    - tomodachi94 4y ago
      > You first need to compile/install a bunch of things in order to make Lua useful. How is this any different from pretty much any language out there? From a quick eyeball of RustPython's Cargo.toml[1], there are about 70 different dependencies which all need to be compiled. I haven't worked too much with Autoconf, but I am pretty sure CPython has quite a few dependencies. [1]: https://github.com/RustPython/RustPython/blob/main/Cargo.toml https://github.com/RustPython/RustPython/blob/main/Cargo.tom... > The second part is that the Lua constructs for programming in the large are very weak. This is deliberate because it forces you to use the tools you are given instead of reimplementing features that can already be implemented by using other primitives in Lua. > I believe that Adobe had a postmortem about this (Lightroom, I think?) I can't find this article. Has anyone else had any luck?
dathinab 4y ago
If experiences I heard from previous projects(1) (successfully) embedding cython are to go by python isn't necessary good choice for this. (1): e.g. awesome, blender
- gpm 4y ago
  I'm curious what you've heard about them that suggests this?
  - dathinab 4y ago
    As far as I can tell the maintenance overhead to mapping all the available type from/to python as nicely usable python APIs is a major pain and not just because of cythons current implementation but of e.g. conceptually mismatch between "good" python and "good" C++/Rust/whatever implementations. I.e. you could say that you have a paradigm mismatch between the underlying implementation (of whatever you extend) ad the python API which makes the already hard problem of keeping an API stable even harder. Furthermore tooling to work in such contexts (as a API consumer) often is very limited or does not exists, but the API producer seldomly has the time/money to produce such tooling either. Lastly there is the remaining problem of Python by itself being really slow, for it's main use-cases this often doesn't matter as e.g. the scientific computations are run by native extensions python delegates the work, too. But this would mean that you have to also support native extensions support or hope that no extension needs to be really fast (not needing to be really fast is surprisingly often the case) or similar solutions. In turn choosing something which has a good JIT, AOT or similar compilation has benefits.
- Tade0 4y ago
  I think the worst thing about Python and Blender was not necessarily the choice of language, but the frequent API changes and not having anything close to an IDE-like environment. Not only did old examples often just not work, there was no serious code completion to facilitate creating anything from scratch.
  - pzone 4y ago
brucethemoose2 4y ago
Civ4 used Python scripting, and it was a horrendous performance bottleneck :/
- wyldfire 4y ago
  Python is the most incredible language because everything's dynamic. But it comes at an enormous cost - you can't make many assumptions or optimizations easily. It has tremendous utility for many applications. It's this utility and the wide body of existing software packages which makes it so tempting to use as a scripting engine. But it would be a bad move for any scripting that's in some critical region.
  - brucethemoose2 4y ago
    It would be nice to have an official, more performant statically typed/less dynamic subset of Python. Yeah we have mypy, but its not official and not integrated. And subsets that do have a performance benefit (like TorchScript) are niche and/or not sustainably supported.
    - wyldfire 4y ago
      Well -- other languages have shown up to try and fill this niche.
    - ben-schaaf 4y ago
      Not statically typed, but rpython is a statically analyzable subset of python that is used to write pypy: https://rpython.readthedocs.io/en/latest/rpython.html https://rpython.readthedocs.io/en/latest/rpython.html
      - brucethemoose2 4y ago
        That looks very reasonable. Converting a Python codebase would be much easier than porting to a new language. Now all your imports, on the other hand...
- 4y ago
seurimas 4y ago
I think the greatest idea for Rust scripting is just supporting WASM scripts. I've had some success toying with wasmer in bevy.
fwip 4y ago
You may like Rhai: https://crates.io/crates/rhai https://crates.io/crates/rhai

singularity2001 4y ago

Yeah the performance comparison doesn't look too great yet: https://rustpython.github.io/benchmarks https://rustpython.github.io/benchmarks Currently it's more about clarity and a nice clean code base, but it can only get faster from here

andai 4y ago
Any idea where the slowness is coming from?
- intrepidhero 4y ago
  I've been playing with making a rust interpreter and reading other people's work. Here's an interesting blog post (not mine) on making a interpreter in rust match the speed of a c implementation. It seems folks often have to resort to unsafe rust to get the performance they want. I don't claim to fully understand that and am looking for more information myself. http://www.dannyvankooten.com/blog/2022/rewriting-interpreter-rust/ http://www.dannyvankooten.com/blog/2022/rewriting-interprete...
  - xiphias2 4y ago
    With webservers we already saw that the way to make new things fast in Rust is to use lots of unsafe Rust when needed, optimize the code, and after that think about the right abstractions that minimize the unsafe code surface.
- tinco 4y ago
  Well there's no production class garbage collector in Rust yet, so in RustPython they're just using Rc's I think. That's one place to start. The other will probably be the quality of the JIT.
  - Yoric 4y ago
    Does Python have a GC these days?

zem 4y ago

surprised they didn't mention packaging a python project into a static executable, that sounds like a huge use case.

youknowone 4y ago
Thank you. That sounds like an important feature.
- zem 4y ago
  to be clear i don't know if the project supports the feature out of the box; it just seems to me like something it would enable, and that i would have expected to see called out as a use case in the README.
  - Timothycquinn 4y ago
    Thanks for clarifying. Does not look like its there today but hopefully down the road it will.

al_be_back 4y ago

python on the browser is a good use-case, a kinda edge computing scenario where a python API responds with data and python code for client to post-process before display/acting-on.

ameliaquining 4y ago

Both of those things are already possible with CPython. For the former, I'd recommend PyO3; for the latter, pyodide. These projects take CPython as a dependency (though pyodide has to apply a few patches on top) and automate the annoying finicky FFI parts of getting it to talk to Rust or JavaScript.