29 ms·

MesaPy: A Memory-Safe Python Implementation based on PyPy (2018)

119 points by ospider 7y ago

ivoras 7y ago
Too bad there's nothing about threaded / parallel execution.
- bjoli 7y ago
  It probably has the same limitations as PyPy.
- locoman88 7y ago
  Use Go!
mattip 7y ago
How does this handle “pip install” and third party code? It seems the memory safety guarantees depend on reimplementing things in Rust (for stdlib modules), which would be interesting in and of itself, as well as formal verification.
edraferi 7y ago
Interesting project. Who’s sponsoring this?
- est31 7y ago
  The mesalock project is run by Baidu's X-lab.
metalliqaz 7y ago
So did they actually find any memory leaks in the C python libs or are they just assuming they exist because it's written in C?
- Conan_Kudo 7y ago
  It's pretty much the latter, as far as I can tell?
  - misterdoubt 7y ago
    C bad. Rust good. Rah rah rah.
- MGzBezycyjEQrk5 7y ago
  This is part of Baidu X-Lab's efforts to make MesaLock Linux (https://github.com/mesalock-linux https://github.com/mesalock-linux). They are creating memory safe versions of tools for their distribution. Baidu has even made a memory safe version of TPM (https://medium.com/baiduxlab/mesatee-security-formal-verification-in-action-bc179c0a9141 https://medium.com/baiduxlab/mesatee-security-formal-verific...).
no_wizard 7y ago
Looks like it’s only python 2.7? Is there any plans to move this to 3.7+?
- bastawhiz 7y ago
  PyPy is still not fully Py3K ready. I'd expect MesaPy to only have built on a stable version of PyPy.
  - misterdoubt 7y ago
    It is, insofar as any PyPy is ever "fully ready." The latest PyPy3 release targets 3.5.3.
    - mattip 7y ago
      PyPy 3.6.1 was released and is very usable, the next release will be 3.6.9
      - misterdoubt 7y ago
        3.6 releases are "beta quality," which I agree is very usable but not "fully ready." as per grandparent.
_nhynes 7y ago
I gave this a spin a while back but, unfortunately didn’t get very far since the software and dependencies have grown long in the byte. Since then, I’ve found RustPython [0] which is progressing toward feature parity with CPython but entirely written in Rust (!). A side benefit is that it compiles to Web Assembly, so if you could sandbox it without too much extra overhead. [0] https://github.com/RustPython/RustPython https://github.com/RustPython/RustPython
- oefrha 7y ago
  Since a lot of CPython’s power comes from its C API (think for example NumPy), this is unfortunately pretty far from feature parity with CPython, unless I’m missing something.
  - mohaine 7y ago
    This. Native python is very slow so almost any compute heavy python work will offload work to external native code. While the a RustPython would help on the python side, this wouldn't help the external native calls which end up doing most of the work.
- westurner 7y ago
  > Since then, I’ve found RustPython [0] which is progressing toward feature parity with CPython but entirely written in Rust (!). A side benefit is that it compiles to Web Assembly, so if you could sandbox it without too much extra overhead. It's now possible to run JupyterLab entirely within a browser with jyve (JupyterLab + pyodide) https://github.com/iodide-project/pyodide/issues/431 https://github.com/iodide-project/pyodide/issues/431 Pyodide: > Pyodide brings the Python runtime to the browser via WebAssembly, along with the Python scientific stack including NumPy, Pandas, Matplotlib, parts of SciPy, and NetworkX. The packages directory lists over 35 packages which are currently available. Is the RustPython WASM build more performant or otherwise preferable to brython or pyodide?

119 points by ospider 7y ago

ivoras 7y ago

Too bad there's nothing about threaded / parallel execution.

bjoli 7y ago
It probably has the same limitations as PyPy.
locoman88 7y ago
Use Go!

mattip 7y ago

How does this handle “pip install” and third party code? It seems the memory safety guarantees depend on reimplementing things in Rust (for stdlib modules), which would be interesting in and of itself, as well as formal verification.

edraferi 7y ago

Interesting project. Who’s sponsoring this?

est31 7y ago
The mesalock project is run by Baidu's X-lab.

metalliqaz 7y ago

So did they actually find any memory leaks in the C python libs or are they just assuming they exist because it's written in C?

Conan_Kudo 7y ago
It's pretty much the latter, as far as I can tell?
- misterdoubt 7y ago
  C bad. Rust good. Rah rah rah.
MGzBezycyjEQrk5 7y ago
This is part of Baidu X-Lab's efforts to make MesaLock Linux (https://github.com/mesalock-linux https://github.com/mesalock-linux). They are creating memory safe versions of tools for their distribution. Baidu has even made a memory safe version of TPM (https://medium.com/baiduxlab/mesatee-security-formal-verification-in-action-bc179c0a9141 https://medium.com/baiduxlab/mesatee-security-formal-verific...).

no_wizard 7y ago

Looks like it’s only python 2.7? Is there any plans to move this to 3.7+?

bastawhiz 7y ago
PyPy is still not fully Py3K ready. I'd expect MesaPy to only have built on a stable version of PyPy.
- misterdoubt 7y ago
  It is, insofar as any PyPy is ever "fully ready." The latest PyPy3 release targets 3.5.3.
  - mattip 7y ago
    PyPy 3.6.1 was released and is very usable, the next release will be 3.6.9
    - misterdoubt 7y ago
      3.6 releases are "beta quality," which I agree is very usable but not "fully ready." as per grandparent.

_nhynes 7y ago

I gave this a spin a while back but, unfortunately didn’t get very far since the software and dependencies have grown long in the byte. Since then, I’ve found RustPython [0] which is progressing toward feature parity with CPython but entirely written in Rust (!). A side benefit is that it compiles to Web Assembly, so if you could sandbox it without too much extra overhead. [0] https://github.com/RustPython/RustPython https://github.com/RustPython/RustPython

oefrha 7y ago
Since a lot of CPython’s power comes from its C API (think for example NumPy), this is unfortunately pretty far from feature parity with CPython, unless I’m missing something.
- mohaine 7y ago
  This. Native python is very slow so almost any compute heavy python work will offload work to external native code. While the a RustPython would help on the python side, this wouldn't help the external native calls which end up doing most of the work.
westurner 7y ago
> Since then, I’ve found RustPython [0] which is progressing toward feature parity with CPython but entirely written in Rust (!). A side benefit is that it compiles to Web Assembly, so if you could sandbox it without too much extra overhead. It's now possible to run JupyterLab entirely within a browser with jyve (JupyterLab + pyodide) https://github.com/iodide-project/pyodide/issues/431 https://github.com/iodide-project/pyodide/issues/431 Pyodide: > Pyodide brings the Python runtime to the browser via WebAssembly, along with the Python scientific stack including NumPy, Pandas, Matplotlib, parts of SciPy, and NetworkX. The packages directory lists over 35 packages which are currently available. Is the RustPython WASM build more performant or otherwise preferable to brython or pyodide?