7 ms·

Python Frontend for GCC

72 points by srean 16y ago

deleted 16y ago
[deleted]
rbanffy 16y ago
I wish Philip and Lance the best of luck. Making this work is a very ambitious goal and we'll all learn a lot from it. And hopefully, get a bunch of new and shiny toys.
srean 16y ago
This started of as a Google Summer of code project last year but is very active http://code.redbrain.co.uk/cgit.cgi/gcc-dev/log/?h=python-dev http://code.redbrain.co.uk/cgit.cgi/gcc-dev/log/?h=python-de... What I find quite interesting is that they are going for full compatibility with Python and not some Pythonic language of somewhat restricted dynamism like rpython, shed-skin, unpython or wirbel, well to an extent cython too. So they will require quite an elaborate runtime library. It will be good if they support optional type annotations like unpython, in the interest of runtime efficiency. The good thing is that unpython programs continue to be valid python code even with those annotations. And if they bake in true tail recursion that would be awesome, not very unrealistic, given that gcc supports tail call optimization for a number of scenarios.
- sambeau 16y ago
  That link appear to show GCC activity for other projects too. For instance, clicking on 'ian' gives you the details of a gccgo checkin. http://code.redbrain.co.uk/cgit.cgi/gcc-dev/commit/?h=python-dev&id=0dc2f918b17f3bf66aec64d8f9b0630f4677fbbb http://code.redbrain.co.uk/cgit.cgi/gcc-dev/commit/?h=python...
- TillE 16y ago
  > So they will require quite an elaborate runtime library. No kidding. The most realistic methods for compiling Python are probably something like Nuitka, which basically turns Python code into CPython API calls, bypassing the step of parsing bytecode at runtime. So it started out 100% compatible with CPython but only marginally faster, and then they started adding some optimizations. http://kayhayen24x7.homelinux.org/blog/nuitka-a-python-compiler/ http://kayhayen24x7.homelinux.org/blog/nuitka-a-python-compi...
sambeau 16y ago
Ian Lance Taylor is the creator and maintainer of gccgo the Go front-end for gcc. http://golang.org/doc/gccgo_install.html http://golang.org/doc/gccgo_install.html He also has an interesting blog: http://www.airs.com/blog/ http://www.airs.com/blog/
- danieldk 16y ago
  Not only that, he is also the developer of the gold linker (and binutils maintainer from 1996 to 1999): https://lwn.net/Articles/274859/ https://lwn.net/Articles/274859/
IgorPartola 16y ago
Does anyone have an idea of how difficult this will be? Is this going to be another Unladen Swallow project, where the research is really cool, but it never can get to be production ready, or should I be getting excited for being able to write production code against this toolchain in a few years?
- kingkilr 16y ago
  a) Unladen Swallow wasn't research, in fact it was explicitly "just engineering". b) I seriously doubt this goes anywhere, Python is simply not amenable to productive static analysis.
  - rayiner 16y ago
    Static whole program type analysis is pretty much a lost cause for any production compiler.
fleitz 16y ago
Is there another group doing this with LLVM?
- maximilianburke 16y ago
  There was the Unladen Swallow project but it was different in a few areas. The Python front end seeks to create an ahead of time compiler for GCC, whereas Unladen Swallow used LLVM's just-in-time compilation framework. Ahead of time compilation could be quite beneficial to applications like smartphones or video games that could benefit from a faster Python environment but don't allow JITs like PyPy or Unladen Swallow. (License permitting.) The Python front end is also being actively developed right now whereas US seems to have petered out.
bcl 16y ago
Interesting project, and I wish them luck. But Python is really not suitable as a compiled language. Its ability to change what any object is, on the fly, really throws a wrench into things. I think efforts like PyPy are much more likely to produce improvements in execution speed.
- riobard 16y ago
  "Its ability to change what any object is, on the fly" No, it doesn't. It changes the reference. If things are referred in the right way, it is possible to statistically compile them. PyPy relies on this info too, but JIT compiling.
  - haberman 16y ago
    class Dog: def speak(self): print "Woof!" class Cat: def speak(self): print "Meow!" obj = Dog() obj.speak() obj.__class__ = Cat obj.speak() Prints: Woof! Meow! Seems to validate what the OP was saying. EDIT: I don't understand the downvotes. The OP spoke of Python's "ability to change what any object is, on the fly," but was refuted and downvoted. Doesn't the above code sample illustrate exactly that? The object's class can be changed on the fly, therefore changing what the object is.
    - ohyes 16y ago
      Obj when constructed contains a reference to the Dog class. Assignment updates the reference to be the Cat class. None of this has relevance to the ability to compile to machine code. It does however, have implications for the efficiency of said code. (Object has to dereference the speak function for its class, and then call it, rather than being able to call the class's speak function directly, as you might in Java. It is also harder to eliminate type checks with such a scheme).
      - haberman 16y ago
        > None of this has relevance to the ability to compile to machine code. It does however, have implications for the efficiency of said code. This is a tautology. You can always "compile" a script to machine code by just making all the generated code call back into the interpreter. The only point of compiling to machine code is if it can be more efficient than interpreting the code. > Object has to dereference the speak function for its class, and then call it That's exactly what the interpreter would have to do. So what are you gaining by ahead-of-time compiling to machine code, again?

72 points by srean 16y ago

deleted 16y ago

[deleted]

I wish Philip and Lance the best of luck. Making this work is a very ambitious goal and we'll all learn a lot from it. And hopefully, get a bunch of new and shiny toys.

srean 16y ago

This started of as a Google Summer of code project last year but is very active http://code.redbrain.co.uk/cgit.cgi/gcc-dev/log/?h=python-dev http://code.redbrain.co.uk/cgit.cgi/gcc-dev/log/?h=python-de... What I find quite interesting is that they are going for full compatibility with Python and not some Pythonic language of somewhat restricted dynamism like rpython, shed-skin, unpython or wirbel, well to an extent cython too. So they will require quite an elaborate runtime library. It will be good if they support optional type annotations like unpython, in the interest of runtime efficiency. The good thing is that unpython programs continue to be valid python code even with those annotations. And if they bake in true tail recursion that would be awesome, not very unrealistic, given that gcc supports tail call optimization for a number of scenarios.

sambeau 16y ago
That link appear to show GCC activity for other projects too. For instance, clicking on 'ian' gives you the details of a gccgo checkin. http://code.redbrain.co.uk/cgit.cgi/gcc-dev/commit/?h=python-dev&id=0dc2f918b17f3bf66aec64d8f9b0630f4677fbbb http://code.redbrain.co.uk/cgit.cgi/gcc-dev/commit/?h=python...
TillE 16y ago
> So they will require quite an elaborate runtime library. No kidding. The most realistic methods for compiling Python are probably something like Nuitka, which basically turns Python code into CPython API calls, bypassing the step of parsing bytecode at runtime. So it started out 100% compatible with CPython but only marginally faster, and then they started adding some optimizations. http://kayhayen24x7.homelinux.org/blog/nuitka-a-python-compiler/ http://kayhayen24x7.homelinux.org/blog/nuitka-a-python-compi...

sambeau 16y ago

Ian Lance Taylor is the creator and maintainer of gccgo the Go front-end for gcc. http://golang.org/doc/gccgo_install.html http://golang.org/doc/gccgo_install.html He also has an interesting blog: http://www.airs.com/blog/ http://www.airs.com/blog/

danieldk 16y ago
Not only that, he is also the developer of the gold linker (and binutils maintainer from 1996 to 1999): https://lwn.net/Articles/274859/ https://lwn.net/Articles/274859/

IgorPartola 16y ago

Does anyone have an idea of how difficult this will be? Is this going to be another Unladen Swallow project, where the research is really cool, but it never can get to be production ready, or should I be getting excited for being able to write production code against this toolchain in a few years?

kingkilr 16y ago
a) Unladen Swallow wasn't research, in fact it was explicitly "just engineering". b) I seriously doubt this goes anywhere, Python is simply not amenable to productive static analysis.
- rayiner 16y ago
  Static whole program type analysis is pretty much a lost cause for any production compiler.

fleitz 16y ago

Is there another group doing this with LLVM?

maximilianburke 16y ago
There was the Unladen Swallow project but it was different in a few areas. The Python front end seeks to create an ahead of time compiler for GCC, whereas Unladen Swallow used LLVM's just-in-time compilation framework. Ahead of time compilation could be quite beneficial to applications like smartphones or video games that could benefit from a faster Python environment but don't allow JITs like PyPy or Unladen Swallow. (License permitting.) The Python front end is also being actively developed right now whereas US seems to have petered out.

bcl 16y ago

Interesting project, and I wish them luck. But Python is really not suitable as a compiled language. Its ability to change what any object is, on the fly, really throws a wrench into things. I think efforts like PyPy are much more likely to produce improvements in execution speed.

riobard 16y ago
"Its ability to change what any object is, on the fly" No, it doesn't. It changes the reference. If things are referred in the right way, it is possible to statistically compile them. PyPy relies on this info too, but JIT compiling.
- haberman 16y ago
  class Dog: def speak(self): print "Woof!" class Cat: def speak(self): print "Meow!" obj = Dog() obj.speak() obj.__class__ = Cat obj.speak() Prints: Woof! Meow! Seems to validate what the OP was saying. EDIT: I don't understand the downvotes. The OP spoke of Python's "ability to change what any object is, on the fly," but was refuted and downvoted. Doesn't the above code sample illustrate exactly that? The object's class can be changed on the fly, therefore changing what the object is.
  - ohyes 16y ago
    Obj when constructed contains a reference to the Dog class. Assignment updates the reference to be the Cat class. None of this has relevance to the ability to compile to machine code. It does however, have implications for the efficiency of said code. (Object has to dereference the speak function for its class, and then call it, rather than being able to call the class's speak function directly, as you might in Java. It is also harder to eliminate type checks with such a scheme).
    - haberman 16y ago
      > None of this has relevance to the ability to compile to machine code. It does however, have implications for the efficiency of said code. This is a tautology. You can always "compile" a script to machine code by just making all the generated code call back into the interpreter. The only point of compiling to machine code is if it can be more efficient than interpreting the code. > Object has to dereference the speak function for its class, and then call it That's exactly what the interpreter would have to do. So what are you gaining by ahead-of-time compiling to machine code, again?