5 ms·

I am always keeping and eye on mypyc, typed_python (llvm Python compiler)[0] and nuitka[1] I guess that because Python is extremely dynamic, we may never have

I am always keeping and eye on mypyc, typed_python (llvm Python compiler)[0] and nuitka[1] I guess that because Python is extremely dynamic, we may never have a full everything-works compiler, but I’m excited about the possibility of this becoming some kind of intermediate step where different parts of the program get compiled when possible. [0] https://github.com/APrioriInvestments/typed_python https://github.com/APrioriInvestments/typed_python [1] https://github.com/Nuitka/Nuitka https://github.com/Nuitka/Nuitka

CyberRabbi 6y ago
I feel like it should be the agenda of the typed python syntax to allow writing annotated python code that can be compiled into a form that is as fast as equivalent c code.
intrepidhero 6y ago
Cython is also mentioned downthread. typed_python is new to me. I'll check it out. I'm too am keeping an eye on this space. I think that compiling or transpiling python may be the solution to both the major problems I have with python: performance and distribution. Exciting times.
crazypython 6y ago
Nuitka only removes interpreter overhead. (just 30%) It's still quite slow. To get real performance improvements, we'd need memory optimizations such as a modern JIT's hidden classes and shapes, which store data directly on the object, instead of inside a dictionary. https://mathiasbynens.be/notes/shapes-ics https://mathiasbynens.be/notes/shapes-ics
anaphor 6y ago
We can have a compiler that does everything. It's just a matter of whether you have to stick the python interpreter in the compiled binary or not, or how much of it you have to use and whether you can only use the parts required. This is how a lot of Scheme compilers work, even though you still have `eval` and similar things.
dehrmann 6y ago
Have you considered Kotlin and Graal? It's obviously not Python, but Kotlin feels syntactically like Python meets Java, and since it compiles to byte code, you can do AoT compilation in Graal. Edit: apparently GraalPython is a thing.
- crazypython 6y ago
  Syntactically, sure. But D is semantically a better combination of Python and Java. With `mixin`, you can `eval` arbitrary strings at compile time. You can call methods and use variables that are dynamically generated, like Python's `__getitem__` with D's `opDispatch`. You can generate code based on names and member variables using D's traits. You can use Python-like refcounting with `RefCounted!`. You can use Python's generators/iterators using D's lazy ranges, which are just as fast as manual for loops.[0] You can bind to Python with little effort using PyD. Just like Python, D has great C interop. D compiles quickly and has much nicer syntax than C or C++. [0]: https://forum.dlang.org/post/xbejngbluilrulstmmhu@forum.dlang.org https://forum.dlang.org/post/xbejngbluilrulstmmhu@forum.dlan...
  - pjmlp 6y ago
    There a new book about D for Python programmers. "D, the Best Programming Language, for Former Python Developers: (Learn D Programming for Python Developers)" https://www.amazon.com/Programming-Language-Former-Python-Developers-ebook/dp/B08MD7ZB2X https://www.amazon.com/Programming-Language-Former-Python-De...
- amelius 6y ago
  The main benefit of Python is the ecosystem.
  - dehrmann 6y ago
    For ML and scientific work, yes. Aside from that, Java has a very good ecosystem. I personally think it's better than Python's, but they're both good.
  - 6y ago
cycomanic 6y ago
I'd add Pythran to that list. It's a python to cpp compiler for numerical python. It achieves impressive speed ups often with very little adjustment of the numerical code. It's highly undervalued IMO, you get speed similar or better than highly optimized cython or c code with very little or no adjustments. I compared it to cython, numba and Julia for DSO, which I wrote about here: https://jochenschroeder.com/blog/articles/DSP_with_Python2/ https://jochenschroeder.com/blog/articles/DSP_with_Python2/
- nerpderp82 6y ago
  If you have a Python2 codebase, Shedskin also gives excellent speedups for numerical codes, the only thing that didn't see as good of a speed boost was string operations. Although that might be fixed. https://shedskin.github.io/ https://shedskin.github.io/
rkeene2 6y ago
If Tcl can be compiled (to a large degree, and without type annotations) to machine code (AOT) using TclQuadCode there's every hope for Python !
pjmlp 6y ago
SELF, Lisp and Smalltalk are just as dynamic. There isn't any Python dynamic feature that those languages lack, yet they solved out the problem a couple of decades ago, and SELF JIT is the genesis of HotSpot. What Python lacks is JIT love and being too much attached to CPython.

CyberRabbi 6y ago

I feel like it should be the agenda of the typed python syntax to allow writing annotated python code that can be compiled into a form that is as fast as equivalent c code.

intrepidhero 6y ago

Cython is also mentioned downthread. typed_python is new to me. I'll check it out. I'm too am keeping an eye on this space. I think that compiling or transpiling python may be the solution to both the major problems I have with python: performance and distribution. Exciting times.

crazypython 6y ago

Nuitka only removes interpreter overhead. (just 30%) It's still quite slow. To get real performance improvements, we'd need memory optimizations such as a modern JIT's hidden classes and shapes, which store data directly on the object, instead of inside a dictionary. https://mathiasbynens.be/notes/shapes-ics https://mathiasbynens.be/notes/shapes-ics

anaphor 6y ago

We can have a compiler that does everything. It's just a matter of whether you have to stick the python interpreter in the compiled binary or not, or how much of it you have to use and whether you can only use the parts required. This is how a lot of Scheme compilers work, even though you still have `eval` and similar things.

dehrmann 6y ago

Have you considered Kotlin and Graal? It's obviously not Python, but Kotlin feels syntactically like Python meets Java, and since it compiles to byte code, you can do AoT compilation in Graal. Edit: apparently GraalPython is a thing.

crazypython 6y ago
Syntactically, sure. But D is semantically a better combination of Python and Java. With `mixin`, you can `eval` arbitrary strings at compile time. You can call methods and use variables that are dynamically generated, like Python's `__getitem__` with D's `opDispatch`. You can generate code based on names and member variables using D's traits. You can use Python-like refcounting with `RefCounted!`. You can use Python's generators/iterators using D's lazy ranges, which are just as fast as manual for loops.[0] You can bind to Python with little effort using PyD. Just like Python, D has great C interop. D compiles quickly and has much nicer syntax than C or C++. [0]: https://forum.dlang.org/post/xbejngbluilrulstmmhu@forum.dlang.org https://forum.dlang.org/post/xbejngbluilrulstmmhu@forum.dlan...
- pjmlp 6y ago
  There a new book about D for Python programmers. "D, the Best Programming Language, for Former Python Developers: (Learn D Programming for Python Developers)" https://www.amazon.com/Programming-Language-Former-Python-Developers-ebook/dp/B08MD7ZB2X https://www.amazon.com/Programming-Language-Former-Python-De...
amelius 6y ago
The main benefit of Python is the ecosystem.
- dehrmann 6y ago
  For ML and scientific work, yes. Aside from that, Java has a very good ecosystem. I personally think it's better than Python's, but they're both good.
- 6y ago

cycomanic 6y ago

I'd add Pythran to that list. It's a python to cpp compiler for numerical python. It achieves impressive speed ups often with very little adjustment of the numerical code. It's highly undervalued IMO, you get speed similar or better than highly optimized cython or c code with very little or no adjustments. I compared it to cython, numba and Julia for DSO, which I wrote about here: https://jochenschroeder.com/blog/articles/DSP_with_Python2/ https://jochenschroeder.com/blog/articles/DSP_with_Python2/

nerpderp82 6y ago
If you have a Python2 codebase, Shedskin also gives excellent speedups for numerical codes, the only thing that didn't see as good of a speed boost was string operations. Although that might be fixed. https://shedskin.github.io/ https://shedskin.github.io/

rkeene2 6y ago

If Tcl can be compiled (to a large degree, and without type annotations) to machine code (AOT) using TclQuadCode there's every hope for Python !

pjmlp 6y ago

SELF, Lisp and Smalltalk are just as dynamic. There isn't any Python dynamic feature that those languages lack, yet they solved out the problem a couple of decades ago, and SELF JIT is the genesis of HotSpot. What Python lacks is JIT love and being too much attached to CPython.