12 ms·

Pyjion – A Python JIT Compiler

257 points by BerislavLopac 5y ago

deleted 5y ago
[deleted]
asicsp 5y ago
GitHub repo: https://github.com/tonybaloney/pyjion https://github.com/tonybaloney/pyjion HN discussion on the repo from which this is forked: "Pyjion – A JIT for Python based upon CoreCLR" https://news.ycombinator.com/item?id=10984514 https://news.ycombinator.com/item?id=10984514 (Jan 28, 2016 | 23 comments)
y7 5y ago
This is the first time I've learned of this. How it compares to PyPy [1]: > PyPy is an implementation of Python with its own JIT. The biggest difference compared to Pyjion is that PyPy doesn't support all C extension modules without modification unless they use CFFI or work with the select subset of CPython's C API that PyPy does support. Pyjion also aims to support many JIT compilers while PyPy only supports their custom JIT compiler. Apparently it's originally a Microsoft project, and it requires .NET. The project already exists for a few years, but it looks like they've gained net performance improvements over CPython only the past few months [2]. [1] https://github.com/tonybaloney/Pyjion https://github.com/tonybaloney/Pyjion [2] https://github.com/tonybaloney/Pyjion/issues/198 https://github.com/tonybaloney/Pyjion/issues/198
- johnnycerberus 5y ago
  Isn't this what the GraalVM [1] guys are also trying to do? Seems like today the competition is between who is more polyglot than the other, JVM, CLR or WASM. [1] https://github.com/oracle/graalpython https://github.com/oracle/graalpython
  - chrisseaton 5y ago
    A big difference is the Pyjion generates .NET compiler IR manually, while GraalVM generates Graal compiler IR automatically through partial evaluation of a declarative specification of an interpreter specialised to the program (the first Futamura Projection.) In my opinion that's far more powerful in terms of removing abstraction. But Pyjion also seems like a very cool project.
    - rowanG077 5y ago
      Damn I didn't know any of the Futamura Projections where actually implemented in practice.
      - chrisseaton 5y ago
        https://chrisseaton.com/truffleruby/pldi17-truffle/pldi17-truffle.pdf https://chrisseaton.com/truffleruby/pldi17-truffle/pldi17-tr...
      - jhgb 5y ago
        Sounds to me like PyPy has been using Futamura projections for a decade or so.
        rowanG077 5y ago
        Can you link me to some sources? I'm not familiar with PyPy but I thought it's a normal tracing JIT. In fact quick googling shows a blog post explicitly saying PyPy does NOT use PE: https://www.pypy.org/posts/2018/09/the-first-15-years-of-pypy-3412615975376972020.html#why-did-we-abandon-partial-evaluation https://www.pypy.org/posts/2018/09/the-first-15-years-of-pyp.... Besides not every PE instance is a Futurama Projection.
        svieira 5y ago
        PyPy is two projects. 1. RPython + the PyPy _compiler_ which is a compiler for JIT compilers (like GraalVM as I understand it) 2. An implementation of the Python language _using_ RPython to produce a JIT for Python scripts. There are other languages _using_ RPython + PyPy compiler to produce JIT compilers for languages other than Python too. https://doc.pypy.org/en/latest/architecture.html#layers https://doc.pypy.org/en/latest/architecture.html#layers
        rowanG077 5y ago
        I see. This doesn't look like a Futaruma Projection to me. What PyPy does is run a python program under their own RPython based interpreter. Then it uses a tracing JIT to JIT the RPython based interpreter. There is no PE going on. No residual specialized program is created. The first Futaruma Projection would be if PyPy would specialize the interpreter based on the python program yielding an executable.
  - pjmlp 5y ago
    WASM isn't on the same league until it offers GC support.
qalmakka 5y ago
So this is basically a newer, cooler IronPython?
- BiteCode_dev 5y ago
  No, it's cpython in a trenchcoat, not written in .net, not ridding the .net vm. But it uses the JIT from .net.
- pabloem 5y ago
  From FAQ[1]: > IronPython is an implementation of Python that is implemented using .NET. While IronPython tries to be usable from within .NET, Pyjion does not have a compatibility story with .NET. This also means IronPython cannot use C extension modules while Pyjion can. [1] https://github.com/tonybaloney/pyjion#ironpython https://github.com/tonybaloney/pyjion#ironpython
  - qalmakka 5y ago
    So it basically uses .NET in place of something like, LLVM, in order to generate native code. Clever.
- sakesun 5y ago
  One of the Pyjion contributor worked on IronPython before.
BBC-vs-neolibs 5y ago
Why is there so little mention of https://cython.org/ https://cython.org/ ? It's very fast, it compiles to C code, then compiles the C code with a normal compiler.
- chrisseaton 5y ago
  My understanding is that Cython does not run unmodified standard Python code.
  - jonatron 5y ago
    It can run most unmodified Python code, but it needs types for the big speedups.
    - chrisseaton 5y ago
      For example here's a Python project modified to work well on Cython - I think that's pretty substantially modified. https://github.com/jameskmurphy/nes/tree/main/nes/cycore https://github.com/jameskmurphy/nes/tree/main/nes/cycore It's more like a DSL for an FFI.
      - jonatron 5y ago
        And here's a project that's mostly Python, and optionally uses Cython https://github.com/falconry/falcon https://github.com/falconry/falcon
    - DasIch 5y ago
      That's misleading as normal type annotations don't give you any performance benefit at all and you may need to do a lot more than just add type annotations. I think it's far more accurate to say that Cython is a programming language of its own that is a hybrid of Python and C++, that happens to produce CPython extension modules when compiled. The performance benefits are really achieved by incrementally changing your Python code to something that looks a lot more like C(++). This is also reflected in the Cython documentation which literally mentions the "Cython language". Cython is great as an alternative to writing C extension modules for performance reasons or to creating bindings to libraries written in C or C++. It's not so great just to make Python applications faster as it's not fully compatible[1]. [1]: https://cython.readthedocs.io/en/latest/src/userguide/limitations.html https://cython.readthedocs.io/en/latest/src/userguide/limita...
cool-RR 5y ago
Is this a modern version of Psyco? http://psyco.sourceforge.net/ http://psyco.sourceforge.net/
- pabloem 5y ago
  From FAQ[1]: > Psyco was a module that monkeypatched CPython to add a custom JIT compiler. Pyjion wants to introduce a proper C API for adding a JIT compiler to CPython instead of monkeypatching it. It should be noted the creator of Psyco went on to be one of the co-founders of PyPy. [1] https://github.com/tonybaloney/pyjion#psyco https://github.com/tonybaloney/pyjion#psyco
antman 5y ago
Can the dotnet dependencies be installed in Python?
peterhil 5y ago
Does it have GIL? IronPython and Jython does not. https://wiki.python.org/moin/GlobalInterpreterLock https://wiki.python.org/moin/GlobalInterpreterLock How about Numpy performance?
- jreese 5y ago
  It's not an alternate runtime, so the GIL is still there and Numpy performance should be unchanged, though any pure-Python modules from Numpy could potentially benefit from the JIT speedup.
- int_19h 5y ago
  It is not a complete rewrite. It basically just replaces the bytecode interpreter (which is a documented extensibility point) in regular CPython.
vletal 5y ago
> Some benchmarks have a slow max/mean value because the time includes JIT-compiling large libraries like Pandas. So why do not they warm the JIT up first? Moreover the graph does not contain any notion of mean/max values at all. Any idea where I can see a more comprehensive benchmark?
Rochus 5y ago
The overall speed-up factor (calculated from the numbers in the Benchmarks diagram, geomean of factors) is 1.6, which is much less than the factor 4 achieved with PyPy (or the even higher factor claimed by Graal Python). CLI is not very well suited for dynamic languages; as far as I remember the results presented here correspond with Iron Python, which is to be expected based on the proposed concept. EDIT: Yes, indeed pretty much the same factor, see e.g. https://web.archive.org/web/20090324020143/http://www.codeplex.com/IronPython/Wiki/View.aspx?title=IP20VsCPy25Perf&referringTitle=IronPython%20Performance https://web.archive.org/web/20090324020143/http://www.codepl...
- ksec 5y ago
  Assuming zero compatibility problems and a simple drop in, I say 1.6 is pretty damn good from the start. Ruby' drop in YJIT barely managed ~20% speed up.
  - joelbluminator 5y ago
    It's at 28.4% so let's make it barely managed ~30% :) https://speed.yjit.org/ https://speed.yjit.org/ I'd give these guys some more time, it's a long project. They're working hard.
  - Rochus 5y ago
    > Ruby' drop in YJIT barely managed ~20% speed up. I don't know what they calculated, but if I use the numbers given in the table of https://speed.yjit.org/benchmarks/bench-2021-11-04-071006 https://speed.yjit.org/benchmarks/bench-2021-11-04-071006 I get a factor 1.9 which is much better than the reported 27%.
- Twirrim 5y ago
  From the details on https://pypi.org/project/pyjion/ https://pypi.org/project/pyjion/ > Goal #1 is explicitly to add a C API to CPython to support JIT compilers. Given the plural compilers, hopefully this means it's going to support multiple JITs, which would be interesting. Choose the right JIT for your particular workload.
zatarc 5y ago
Pyjion requires: CPython 3.10 and .NET 6 .NET 6 Release: 19 hours ago (https://github.com/dotnet/core/blob/main/release-notes/6.0/6.0.0/6.0.0.md?WT.mc_id=dotnet-35129-website https://github.com/dotnet/core/blob/main/release-notes/6.0/6...) ... ok.
- throwawayninja 5y ago
  We found the Arch Linux developer XD "All hail rolling releases, why haven't you updated yet? I published it 15 minutes ago!"
  - m00dy 5y ago
    haha :)
- caseymarquis 5y ago
  .NET 6 has been in prerelease for a while, and it's the next LTS release. Makes sense that it would be used as soon as it was available. Upgrading from 5 to 6 should be pretty trivial.
  - zatarc 5y ago
    I've tried it on Fedora: https://docs.microsoft.com/en-us/dotnet/core/install/linux-fedora https://docs.microsoft.com/en-us/dotnet/core/install/linux-f... > The latest version of .NET that's available in the default package repositories for Fedora is .NET 5. Installing .NET 6 through the default package repositories is coming soon. For now, you'll need to install .NET 6 in one of the following ways: > Install the .NET SDK or the .NET Runtime with Snap. > Install the .NET SDK or the .NET Runtime with a script. > Install the .NET SDK or the .NET Runtime manually. Nope, thank you
    - progmetaldev 5y ago
      .NET 6 was just released this morning, so I'd assume it will take at least a little bit of time to get into the default package repositories.
rahen 5y ago
I don't really see the benefit over Nuitka, or even a JIT like Numba. Can someone help?
- nerdponx 5y ago
  Neither of those is a drop-in replacement for CPython. It would be better to compare this project to, in addition to CPython: PyPy, GraalPython, Pyston, Cinder, Jython, and IronPython. As well as the now-inactive Psyco project, and possibly also Stackless Python.
  - int_19h 5y ago
    It should be noted that Pyjion in particular is CPython. It's not even a fork - it's a native CPython module that provides a replacement for the stock bytecode interpreter (for which CPython has an API). Everything else is the same, so it's compatible with pretty much any random native module compiled for CPython.
swuecho 5y ago
Known Limitations With statements Pyjion does not currently support with blocks. This is on the roadmap (and related to try..except). ===== I thin with block is used all of the places, so this limitation make it not usable to a lot of projects
haberman 5y ago
Python is in a situation where it is enormously popular, due to a winning aesthetic in syntax/ergonomics that appeals to both newbies and experienced programmers. And due to this success, there is a lot of demand and interest in speeding the language up and getting rid of the GIL. But paradoxically, this veneer of simplicity hides an incredible amount of complexity. You have to read hundreds of lines of CPython to understand the full semantics of a statement like "a + b". The semantics are complicated/compromised by many optimizations and implementation details of CPython. This is a fantastic talk that goes into these details: https://youtu.be/qCGofLIzX6g https://youtu.be/qCGofLIzX6g It is not good for anybody that the semantics are this quirky. But it's especially not good for people trying to optimize the language, who basically have to implement quirk-for-quirk identical semantics to what CPython has ended up with after 30 years of evolution and optimization. I wish Python 3.0 had included a more formalized and cleaned up set of semantics. These things can't be simplified without a breaking change to the language (even if only a small number of programs truly depend on these quirks). I would say Python should fix this in 4.0, but the 2->3 transition was so painful and long that I'm not sure the ecosystem can take it.
- marmaduke 5y ago
  Have you seen the Python docs which have pages on the execution model and data model, with notes in implementation details? https://docs.python.org/3/reference/executionmodel.html https://docs.python.org/3/reference/executionmodel.html https://docs.python.org/3/reference/datamodel.html https://docs.python.org/3/reference/datamodel.html
  - haberman 5y ago
    In your second link it says: > For instance, to evaluate the expression x + y, where x is an instance of a class that has an __add__() method, x.__add__(y) is called. The talk I linked to (https://youtu.be/qCGofLIzX6g https://youtu.be/qCGofLIzX6g) is a deep dive on how there is much more to the story than the simplified statement above. What I wish for is not better docs, but rather simpler language in which the statement above would actually be an accurate specification of the behavior implemented by the interpreter.
jreese 5y ago
> Pyjion does not currently support with blocks. > Pyjion does not currently support async..await (YIELD_FROM) statements. Those are some major limitations for modern Python code. https://pyjion.readthedocs.io/en/latest/limitations.html https://pyjion.readthedocs.io/en/latest/limitations.html
- josefx 5y ago
  The point for with blocks claims that this is related to exceptions. How can it handle CPython reference counting during exceptions but fail at cleaning up with blocks?
dang 5y ago
Seems there was just one past thread: Pyjion – A JIT for Python based upon CoreCLR - https://news.ycombinator.com/item?id=10984514 https://news.ycombinator.com/item?id=10984514 - Jan 2016 (23 comments)

257 points by BerislavLopac 5y ago

deleted 5y ago

[deleted]

GitHub repo: https://github.com/tonybaloney/pyjion https://github.com/tonybaloney/pyjion HN discussion on the repo from which this is forked: "Pyjion – A JIT for Python based upon CoreCLR" https://news.ycombinator.com/item?id=10984514 https://news.ycombinator.com/item?id=10984514 (Jan 28, 2016 | 23 comments)

y7 5y ago

This is the first time I've learned of this. How it compares to PyPy [1]: > PyPy is an implementation of Python with its own JIT. The biggest difference compared to Pyjion is that PyPy doesn't support all C extension modules without modification unless they use CFFI or work with the select subset of CPython's C API that PyPy does support. Pyjion also aims to support many JIT compilers while PyPy only supports their custom JIT compiler. Apparently it's originally a Microsoft project, and it requires .NET. The project already exists for a few years, but it looks like they've gained net performance improvements over CPython only the past few months [2]. [1] https://github.com/tonybaloney/Pyjion https://github.com/tonybaloney/Pyjion [2] https://github.com/tonybaloney/Pyjion/issues/198 https://github.com/tonybaloney/Pyjion/issues/198

johnnycerberus 5y ago
Isn't this what the GraalVM [1] guys are also trying to do? Seems like today the competition is between who is more polyglot than the other, JVM, CLR or WASM. [1] https://github.com/oracle/graalpython https://github.com/oracle/graalpython
- chrisseaton 5y ago
  A big difference is the Pyjion generates .NET compiler IR manually, while GraalVM generates Graal compiler IR automatically through partial evaluation of a declarative specification of an interpreter specialised to the program (the first Futamura Projection.) In my opinion that's far more powerful in terms of removing abstraction. But Pyjion also seems like a very cool project.
  - rowanG077 5y ago
    Damn I didn't know any of the Futamura Projections where actually implemented in practice.
    - chrisseaton 5y ago
      https://chrisseaton.com/truffleruby/pldi17-truffle/pldi17-truffle.pdf https://chrisseaton.com/truffleruby/pldi17-truffle/pldi17-tr...
    - jhgb 5y ago
      Sounds to me like PyPy has been using Futamura projections for a decade or so.
      - rowanG077 5y ago
        Can you link me to some sources? I'm not familiar with PyPy but I thought it's a normal tracing JIT. In fact quick googling shows a blog post explicitly saying PyPy does NOT use PE: https://www.pypy.org/posts/2018/09/the-first-15-years-of-pypy-3412615975376972020.html#why-did-we-abandon-partial-evaluation https://www.pypy.org/posts/2018/09/the-first-15-years-of-pyp.... Besides not every PE instance is a Futurama Projection.
        svieira 5y ago
        PyPy is two projects. 1. RPython + the PyPy _compiler_ which is a compiler for JIT compilers (like GraalVM as I understand it) 2. An implementation of the Python language _using_ RPython to produce a JIT for Python scripts. There are other languages _using_ RPython + PyPy compiler to produce JIT compilers for languages other than Python too. https://doc.pypy.org/en/latest/architecture.html#layers https://doc.pypy.org/en/latest/architecture.html#layers
        rowanG077 5y ago
        I see. This doesn't look like a Futaruma Projection to me. What PyPy does is run a python program under their own RPython based interpreter. Then it uses a tracing JIT to JIT the RPython based interpreter. There is no PE going on. No residual specialized program is created. The first Futaruma Projection would be if PyPy would specialize the interpreter based on the python program yielding an executable.
- pjmlp 5y ago
  WASM isn't on the same league until it offers GC support.

qalmakka 5y ago

So this is basically a newer, cooler IronPython?

BiteCode_dev 5y ago
No, it's cpython in a trenchcoat, not written in .net, not ridding the .net vm. But it uses the JIT from .net.
pabloem 5y ago
From FAQ[1]: > IronPython is an implementation of Python that is implemented using .NET. While IronPython tries to be usable from within .NET, Pyjion does not have a compatibility story with .NET. This also means IronPython cannot use C extension modules while Pyjion can. [1] https://github.com/tonybaloney/pyjion#ironpython https://github.com/tonybaloney/pyjion#ironpython
- qalmakka 5y ago
  So it basically uses .NET in place of something like, LLVM, in order to generate native code. Clever.
sakesun 5y ago
One of the Pyjion contributor worked on IronPython before.

BBC-vs-neolibs 5y ago

Why is there so little mention of https://cython.org/ https://cython.org/ ? It's very fast, it compiles to C code, then compiles the C code with a normal compiler.

chrisseaton 5y ago
My understanding is that Cython does not run unmodified standard Python code.
- jonatron 5y ago
  It can run most unmodified Python code, but it needs types for the big speedups.
  - chrisseaton 5y ago
    For example here's a Python project modified to work well on Cython - I think that's pretty substantially modified. https://github.com/jameskmurphy/nes/tree/main/nes/cycore https://github.com/jameskmurphy/nes/tree/main/nes/cycore It's more like a DSL for an FFI.
    - jonatron 5y ago
      And here's a project that's mostly Python, and optionally uses Cython https://github.com/falconry/falcon https://github.com/falconry/falcon
  - DasIch 5y ago
    That's misleading as normal type annotations don't give you any performance benefit at all and you may need to do a lot more than just add type annotations. I think it's far more accurate to say that Cython is a programming language of its own that is a hybrid of Python and C++, that happens to produce CPython extension modules when compiled. The performance benefits are really achieved by incrementally changing your Python code to something that looks a lot more like C(++). This is also reflected in the Cython documentation which literally mentions the "Cython language". Cython is great as an alternative to writing C extension modules for performance reasons or to creating bindings to libraries written in C or C++. It's not so great just to make Python applications faster as it's not fully compatible[1]. [1]: https://cython.readthedocs.io/en/latest/src/userguide/limitations.html https://cython.readthedocs.io/en/latest/src/userguide/limita...

cool-RR 5y ago

Is this a modern version of Psyco? http://psyco.sourceforge.net/ http://psyco.sourceforge.net/

pabloem 5y ago
From FAQ[1]: > Psyco was a module that monkeypatched CPython to add a custom JIT compiler. Pyjion wants to introduce a proper C API for adding a JIT compiler to CPython instead of monkeypatching it. It should be noted the creator of Psyco went on to be one of the co-founders of PyPy. [1] https://github.com/tonybaloney/pyjion#psyco https://github.com/tonybaloney/pyjion#psyco

antman 5y ago

Can the dotnet dependencies be installed in Python?

peterhil 5y ago

Does it have GIL? IronPython and Jython does not. https://wiki.python.org/moin/GlobalInterpreterLock https://wiki.python.org/moin/GlobalInterpreterLock How about Numpy performance?

jreese 5y ago
It's not an alternate runtime, so the GIL is still there and Numpy performance should be unchanged, though any pure-Python modules from Numpy could potentially benefit from the JIT speedup.
int_19h 5y ago
It is not a complete rewrite. It basically just replaces the bytecode interpreter (which is a documented extensibility point) in regular CPython.

vletal 5y ago

> Some benchmarks have a slow max/mean value because the time includes JIT-compiling large libraries like Pandas. So why do not they warm the JIT up first? Moreover the graph does not contain any notion of mean/max values at all. Any idea where I can see a more comprehensive benchmark?

Rochus 5y ago

The overall speed-up factor (calculated from the numbers in the Benchmarks diagram, geomean of factors) is 1.6, which is much less than the factor 4 achieved with PyPy (or the even higher factor claimed by Graal Python). CLI is not very well suited for dynamic languages; as far as I remember the results presented here correspond with Iron Python, which is to be expected based on the proposed concept. EDIT: Yes, indeed pretty much the same factor, see e.g. https://web.archive.org/web/20090324020143/http://www.codeplex.com/IronPython/Wiki/View.aspx?title=IP20VsCPy25Perf&referringTitle=IronPython%20Performance https://web.archive.org/web/20090324020143/http://www.codepl...

ksec 5y ago
Assuming zero compatibility problems and a simple drop in, I say 1.6 is pretty damn good from the start. Ruby' drop in YJIT barely managed ~20% speed up.
- joelbluminator 5y ago
  It's at 28.4% so let's make it barely managed ~30% :) https://speed.yjit.org/ https://speed.yjit.org/ I'd give these guys some more time, it's a long project. They're working hard.
- Rochus 5y ago
  > Ruby' drop in YJIT barely managed ~20% speed up. I don't know what they calculated, but if I use the numbers given in the table of https://speed.yjit.org/benchmarks/bench-2021-11-04-071006 https://speed.yjit.org/benchmarks/bench-2021-11-04-071006 I get a factor 1.9 which is much better than the reported 27%.
Twirrim 5y ago
From the details on https://pypi.org/project/pyjion/ https://pypi.org/project/pyjion/ > Goal #1 is explicitly to add a C API to CPython to support JIT compilers. Given the plural compilers, hopefully this means it's going to support multiple JITs, which would be interesting. Choose the right JIT for your particular workload.

zatarc 5y ago

Pyjion requires: CPython 3.10 and .NET 6 .NET 6 Release: 19 hours ago (https://github.com/dotnet/core/blob/main/release-notes/6.0/6.0.0/6.0.0.md?WT.mc_id=dotnet-35129-website https://github.com/dotnet/core/blob/main/release-notes/6.0/6...) ... ok.

throwawayninja 5y ago
We found the Arch Linux developer XD "All hail rolling releases, why haven't you updated yet? I published it 15 minutes ago!"
- m00dy 5y ago
  haha :)
caseymarquis 5y ago
.NET 6 has been in prerelease for a while, and it's the next LTS release. Makes sense that it would be used as soon as it was available. Upgrading from 5 to 6 should be pretty trivial.
- zatarc 5y ago
  I've tried it on Fedora: https://docs.microsoft.com/en-us/dotnet/core/install/linux-fedora https://docs.microsoft.com/en-us/dotnet/core/install/linux-f... > The latest version of .NET that's available in the default package repositories for Fedora is .NET 5. Installing .NET 6 through the default package repositories is coming soon. For now, you'll need to install .NET 6 in one of the following ways: > Install the .NET SDK or the .NET Runtime with Snap. > Install the .NET SDK or the .NET Runtime with a script. > Install the .NET SDK or the .NET Runtime manually. Nope, thank you
  - progmetaldev 5y ago
    .NET 6 was just released this morning, so I'd assume it will take at least a little bit of time to get into the default package repositories.

rahen 5y ago

I don't really see the benefit over Nuitka, or even a JIT like Numba. Can someone help?

nerdponx 5y ago
Neither of those is a drop-in replacement for CPython. It would be better to compare this project to, in addition to CPython: PyPy, GraalPython, Pyston, Cinder, Jython, and IronPython. As well as the now-inactive Psyco project, and possibly also Stackless Python.
- int_19h 5y ago
  It should be noted that Pyjion in particular is CPython. It's not even a fork - it's a native CPython module that provides a replacement for the stock bytecode interpreter (for which CPython has an API). Everything else is the same, so it's compatible with pretty much any random native module compiled for CPython.

swuecho 5y ago

Known Limitations With statements Pyjion does not currently support with blocks. This is on the roadmap (and related to try..except). ===== I thin with block is used all of the places, so this limitation make it not usable to a lot of projects

haberman 5y ago

Python is in a situation where it is enormously popular, due to a winning aesthetic in syntax/ergonomics that appeals to both newbies and experienced programmers. And due to this success, there is a lot of demand and interest in speeding the language up and getting rid of the GIL. But paradoxically, this veneer of simplicity hides an incredible amount of complexity. You have to read hundreds of lines of CPython to understand the full semantics of a statement like "a + b". The semantics are complicated/compromised by many optimizations and implementation details of CPython. This is a fantastic talk that goes into these details: https://youtu.be/qCGofLIzX6g https://youtu.be/qCGofLIzX6g It is not good for anybody that the semantics are this quirky. But it's especially not good for people trying to optimize the language, who basically have to implement quirk-for-quirk identical semantics to what CPython has ended up with after 30 years of evolution and optimization. I wish Python 3.0 had included a more formalized and cleaned up set of semantics. These things can't be simplified without a breaking change to the language (even if only a small number of programs truly depend on these quirks). I would say Python should fix this in 4.0, but the 2->3 transition was so painful and long that I'm not sure the ecosystem can take it.

marmaduke 5y ago
Have you seen the Python docs which have pages on the execution model and data model, with notes in implementation details? https://docs.python.org/3/reference/executionmodel.html https://docs.python.org/3/reference/executionmodel.html https://docs.python.org/3/reference/datamodel.html https://docs.python.org/3/reference/datamodel.html
- haberman 5y ago
  In your second link it says: > For instance, to evaluate the expression x + y, where x is an instance of a class that has an __add__() method, x.__add__(y) is called. The talk I linked to (https://youtu.be/qCGofLIzX6g https://youtu.be/qCGofLIzX6g) is a deep dive on how there is much more to the story than the simplified statement above. What I wish for is not better docs, but rather simpler language in which the statement above would actually be an accurate specification of the behavior implemented by the interpreter.

jreese 5y ago

> Pyjion does not currently support with blocks. > Pyjion does not currently support async..await (YIELD_FROM) statements. Those are some major limitations for modern Python code. https://pyjion.readthedocs.io/en/latest/limitations.html https://pyjion.readthedocs.io/en/latest/limitations.html

josefx 5y ago
The point for with blocks claims that this is related to exceptions. How can it handle CPython reference counting during exceptions but fail at cleaning up with blocks?

dang 5y ago

Seems there was just one past thread: Pyjion – A JIT for Python based upon CoreCLR - https://news.ycombinator.com/item?id=10984514 https://news.ycombinator.com/item?id=10984514 - Jan 2016 (23 comments)