23 ms·

Nuitka: a Python compiler

267 points by woadwarrior01 12y ago

dannyroberts1 12y ago
This is awesome! I tested it on some code that uses some not-compeltely-trivial python features like metaclasses, etc. and it worked like a charm. I couldn't get it to work against this obfuscated hello world though: http://benkurtovic.com/2014/06/01/obfuscating-hello-world.html http://benkurtovic.com/2014/06/01/obfuscating-hello-world.ht... :)
- mkesper 12y ago
  That one's relying on CPython implementation details, the author says so in his detailed description.
untothebreach 12y ago
This is probably a better link, for those of us (like me) who had never heard of Nuitka: http://nuitka.net/pages/overview.html http://nuitka.net/pages/overview.html
- ntrepid8 12y ago
  Cached: http://webcache.googleusercontent.com/search?q=cache:0v-pht19f18J:nuitka.net/pages/overview.html http://webcache.googleusercontent.com/search?q=cache:0v-pht1...
bhouston 12y ago
Mirror: http://web.archive.org/web/20141218012211/http://nuitka.net/ http://web.archive.org/web/20141218012211/http://nuitka.net/ BTW how much faster is this than PyPy or CPython? That is the question that I think is most on people's minds.
- lucashn 12y ago
  Seems to be 2x faster than CPython, but as I understood it is working on optimizations now.
  - mkesper 12y ago
    As others have pointed out, these claims have to be taken very carefully and the main value (at least for now) seems to be the easy packaging and deploying.
simonpantzare 12y ago
It seems to be, by and large, the work of a single person on his spare time. Very impressive!
jarcane 12y ago
It is somewhat faster than CPython already, but currently it doesn't make all the optimizations possible, but a 258% factor on pystone is a good start (number is from version 0.3.11). Holy cow. And here I'd set aside Python in part because of the difficulty in getting performant executables (I was getting visible lag in a turn-based SDL game ...)
- bhouston 12y ago
  But how does it compare to PyPy?
  - jarcane 12y ago
    To be honest, I found that in my use case (game dev, supported by bindings to a C library), PyPy was actually slower than CPython. I'd be curious to dig up my code and compare performance to a Nuitka-compiled executable.
    - detaro 12y ago
      out of curiosity, which bindings did you use?
      - jarcane 12y ago
        libtcod. It was a roguelike.
        ptx 12y ago
        Those Python bindings seem to use ctypes, which is indeed slower[1] on PyPy. They recommend[2] using cffi instead. [1] http://pypy.org/performance.html http://pypy.org/performance.html [2] http://pypy.readthedocs.org/en/latest/extending.html http://pypy.readthedocs.org/en/latest/extending.html
  - illumen 12y ago
    For games, other soft realtime, and run once code, pypy is slower. The newer GC is better, but the JIT still causes pauses. The slow warm up of the jit, no type hint saving, and slower interpreter is why it is not good on run once code. Of course if your python extensions are calling optimized assembly routines, or hardware, then it will be faster than pypy code written in python. Also, using a JIT is not possible on some platforms (iOS, or if the CPU architecture isn't supported by pypy). Not to say that pypy isn't better for certain tasks of course.
ntrepid8 12y ago
This project looks completely misguided. The talk focused on the trivialities of mapping Python to C++ rather than on the interesting problems to be encountered when trying to optimize Python while maintaining its extremely dynamic semantics. Also the benchmarking effort is laughable; pystone is not to be taken seriously (only exercises a tiny part of the language) and pybench does microbenchmarks, which are optimized away. You should try the "real-world" benchmarks from the PyPy and Unladen Swallow projects. And what is the size of the generated code? (E.g. how big would the binary for the entire standard library be?) In your blog, please use less boring subjects than "version x.y.z released". ~ Guido van Rossum Find Guido's quote in the first comment. https://ep2013.europython.eu/conference/talks/nuitka-the-python-compiler https://ep2013.europython.eu/conference/talks/nuitka-the-pyt...
- fmoralesc 12y ago
  Well, that is harsh and somewhat unwarranted ("In your blog, please use..." Who gave him that authority?). I like Hayden's comment on this all: > The good thing [GvR] did to Python is to make his opinion be just that. I can do Nuitka without and beyond his control.[1] [1]: https://ep2013.europython.eu/conference/talks/nuitka-the-python-compiler#comment-2815 https://ep2013.europython.eu/conference/talks/nuitka-the-pyt...
  - deleted 12y ago
    [deleted]
- orf 12y ago
  I like Guido but that comment seems too harsh and commenting on how boring his blog titles are is completely unnecessary. The talk was very interesting and is very neat for only a spare time project.
- vegabook 12y ago
  So this would be the same Guido van Rossum who made a completely misguided analysis of the language he had birthed, tried to make it "grow up" in version 3.x, brow-beated us all about the dubious benefit of this "new" cruft-encrusted language, causing strategic drift for the entire ecosystem? The best thing that could happen to Python would be a benevolent, and mercifully silent, retirement of GvR. And I speak as a long term Pythonista. Kudos to anybody who is trying to move Python into a performance envelope similar to Lua and Javascript, not to mention Golang, Julia, or Clojure.
zongitsrinzler 12y ago
I don't know much about compilers and runtimes but how come there are SO MANY of them for Python. Is there something about Python that makes it so easy to write runtimes for it?
- bane 12y ago
  I think it's because lots of people write lots of code in python, but then end up with performance problems at some point and try to tackle those. Have a nice regular language makes it relative easy to rewrite a runtime to scratch your itch. I suspect we would have seen this more with languages like Perl, except Perl is virtually impossible to parse.
- bkeroack 12y ago
  I think it's largely because while a lot of people really love the language, they struggle with the performance and scaling issues that arise with the standard CPython implementation.
- dalke 12y ago
  I think it's a matter of numbers. There are a lot of people who use Python. (https://blog.pythonanywhere.com/67/ https://blog.pythonanywhere.com/67/ estimates in the low millions.) Some of them like to work on alternate implementations of Python. I think Python is average in this respect. By comparison, here's an incomplete list of C compilers: http://en.wikipedia.org/wiki/List_of_compilers#C_compilers http://en.wikipedia.org/wiki/List_of_compilers#C_compilers and you can also look at the list of Pascal compilers.
- mhd 12y ago
  Popularity and a syntax that can be parsed, plus being widely taught. Pretty much the same reason why we had plenty of Pascal compilers back in the days. And quite unlike why we have that many JavaScript transpilers. A few of the compilers might be running on the AST, and thus have no complete alternative implementeation. A good module support for that helps, although it's still not close to VM-based languages like Java.
- njharman 12y ago
  Python is the best "glue" language. Large 3rd party libs, easily readable/writable (even by non-programmers), ld (it predates Java), established, great user community. So, people in different realms (MSoft IronPython, JVM/Enterprise Jython, restofus CPython, scientists/performance Cython/PyPy,Stackless,Psyco, academics/experimenters many more) all want to use it and make versions that integrate with their tools.
Beltiras 12y ago
Git repo is unresponsive. I think we killed the poor guys box. EDIT: https://github.com/kayhayen/Nuitka https://github.com/kayhayen/Nuitka
insertion 12y ago
I noticed that PyPy does not show any real speed improvement over CPython in these benchmarks: https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/ If you filter by Python, you can compare some results running on both CPython and PyPy. I would be curious to know what it is about these benchmarks that makes PyPy perform poorly. I would also be interested to see how Nuitka performs. At the moment I'm also very excited about Pyston from Dropbox: https://github.com/dropbox/pyston https://github.com/dropbox/pyston
- Beltiras 12y ago
  I'm supremely sceptical about those benchmarks. Go take a look at the code being tested. I would welcome a serious look at developmental time versus resource used, using code that is probable in production. I read the code used to test Django. It's not reasonable code.
  - saym 12y ago
    It's an open sourced benchmarking comparison. If you can improve them, shoot them a pull request! https://github.com/TechEmpower/FrameworkBenchmarks https://github.com/TechEmpower/FrameworkBenchmarks
- ant6n 12y ago
  I found that PyPy sometimes has unexpected slowdowns. When we were porting from Python to PyPy on some offline processing tools, the most crazy one was building strings via += and sum(arrays,[]), which is much slower than cpython.
  - Fede_V 12y ago
    There was a good blog post this by Armin. Basically, if you have to concatenate strings, don't do so via += but use "".join([]).
    - ant6n 12y ago
      I find that unexpected. Java has had string builder optimization for a long time, and CPython is much much faster in this respect. It's not always easy to use "".join when using a string, so you end up having to build a separate array of strings in some cases. And building arrays isn't always that fast either. And [].join doesn't exist, so summing arrays is always kinda slow. Anyway, all that is to say: I really like PyPy, and we use it a lot, but those _unexpected_ crazy slowdowns are unfortunate.
Beltiras 12y ago
I program Django websites and work in an environment where we like to deploy often. I'm not sure we could tolerate the extra wait in compile time for deployment. At least we would need to take a serious look at the speed/resource gain before considering it.
- andybak 12y ago
  It's been said before but it's fairly unlikely that Python is the bottleneck for typical web-apps.
- robertfw 12y ago
  Deployment in a controlled environment is not the ideal use case here - the benefits come from when you are shipping to non-technical users.
tfeldmann 12y ago
Great project. Compiling a simple PySide application worked fine on the first try. If you bundle all dependencies you get a binary about the same size as a binary produced by py2exe. I will do some speed comparisons next.
misaelm 12y ago
I built a small app that retrieves data from a remote server, builds a summary and then pushes it to a local html file. Number of people that tried it when I told them that to make it work they had to download python, pip, paramiko and install the pycrypto binary? Zero. Number of people that tried it after I just gave them a zip file with an executable? No longer zero. I tried different libraries but Nuitka was the only one that made everything work seamlessly. I owe this guy a beer. Edit: I no longer owe Kay a beer. I found the donation link on his website.
- aidenn0 12y ago
  I recently ended up rewriting large parts of a python project in common-lisp due to this exact issue; in this case it was one stage of a pipeline that I prototyped with an existing python library that could output to xml, but delivery of python applications on windows is painful.
- president 12y ago
  If you packaged it correctly, you could get by just requiring Python and pip to be installed. But still, the zip and single executable is still faster.
- sanderjd 12y ago
  I think this is one of the reasons Go was a breath of fresh air to a lot of Python and Ruby programmers. The ability to quickly generate a static binary is a great feature.
- fernly 12y ago
  This issue has been addressed multiple times with considerable success by pyinstaller, py2app, cxfreeze. You can get your single-zip-file distribution package without nuitka, although it may have other advantages.
bane 12y ago
I'm very interested in some benchmarks on this and various other compilers/packagers/runtimes for Python. Sidenote: A very long time ago I had some great luck with Perl's packagers: PAR, PerlApp, etc. but they all basically just bundle in the basic Perl runtime with the guts of your program so there's no real benchmark difference. I suspect the same is not true of efforts like this for Python.
shadowmint 12y ago
For all that people may think this isnt a good idea, this is a very similar idea to what Unity3d is persuing with IL2CPP (http://blogs.unity3d.com/2014/05/20/the-future-of-scripting-in-unity/ http://blogs.unity3d.com/2014/05/20/the-future-of-scripting-...). This seems immidately relevant to address pythons biggest problems; its slow, and its hard to distribute. Relevant quote: All code generation is done to C++ rather than architecture specific machine code. The cost of porting and maintenance of architecture specific code generation is now more amortised
est 12y ago
Python is a fun language, it has as many web frameworks as its runtime/interpreter.
agentultra 12y ago
I would love to make distributing games built on Python easier. Jessica McKeller suggested this should be a priority[0] to make the language more accessible by getting kids involved in making games and being able to share them, easily, with their friends. These days Javascript is kicking our butt in this area. [0] https://www.youtube.com/watch?v=d1a4Jbjc-vU https://www.youtube.com/watch?v=d1a4Jbjc-vU I've been writing little helper libraries on top of Python 3.4 + pysdl2 as I've been working on games and demos for my (unfortunately cancelled) Pycon talk. One area I've only played with, unsuccessfully, is getting packaging going on Nuitka or some other compiler. If anyone wants to get together to make it awesome get in touch.
kedean 12y ago
My immediate question, which I don't see an answer to on the site or Github page: is it self-hosting yet?
avinassh 12y ago
Checked contributors list on Github[0] and most of the code is written by a single person, Kay. Isn't that amazing? A project of this large and value, done by single person in his own time. I wish this project gets more popularity. Bought a beer to Kay :-) [0] - https://github.com/kayhayen/Nuitka/graphs/contributors https://github.com/kayhayen/Nuitka/graphs/contributors
pbiggar 12y ago
I think this is a really solid approach. It's the same thing I was trying to do for PHP with phc (http://phpcompiler.org/ http://phpcompiler.org/) - compiling down to C using the built in stdlib and C API, then using full program optimization. Long term, it won't be as fast as a full JIT, as the Facebook HPHP team showed, but Python doesn't have a JIT of the same caliber, so this is probably useful for tons of people. Dunno if the author is around, but they might find some of the stuff from my PhD relevant, especially how the static analysis worked and some of the challenges of compiling using the C API: http://paulbiggar.com/research/#phd-dissertation http://paulbiggar.com/research/#phd-dissertation
3am 12y ago
Why are so people so fast to trust a new compiler? Have you audited that compiler's source and building the compiler from it with a known good compiler? Are you inspecting the resulting .pyc, and in this case, the resulting PEs? It's a super easy way to inject a compromise into a package that will probably get widely distributed. full disclosure: link is down for me, so haven't read the article. Been a comment that has been building up for a while for me, and not specific to Nuitka. Same goes for new frameworks/languages/etc.
- coldtea 12y ago
  >Have you audited that compiler's source and building the compiler from it with a known good compiler? Are you inspecting the resulting .pyc, and in this case, the resulting PEs? It's a super easy way to inject a compromise into a package that will probably get widely distributed. Because nobody is that paranoid?
  - hughes 12y ago
    And if they are, they probably aren't in the business of distributing precompiled binaries.
- andreasvc 12y ago
  Why limit yourself to new compilers? Every new version of an old compiler could be suspect, every line of code in general. There's no point in being this paranoid, nobody has time for all the audits that might be theoretically desirable. Almost everything you do on a computer relies on trusting an untold number of components.
- waxjar 12y ago
  I think it unlikely someone would design something complicated like a new compiler / programming language, open source it and attach their real name to the project just to hide an exploit in it. There's much lower hanging fruit.
HerrMonnezza 12y ago
On a tangentially related note: here is a comparison of various Python runtimes (interpreters and compilers including Nuitka and PyPy) on a fairly complex scientific code: http://arxiv.org/abs/1404.6388 http://arxiv.org/abs/1404.6388 The shootout, however, is from August 2013 so getting a bit old by now.
PythonicAlpha 12y ago
I really like the idea, that it compiles Python but is (99,9%) compatible to CPython -- since so you can still use the many C extensions (I don't want to do my projects any more without my own Python extensions written in C).

267 points by woadwarrior01 12y ago

This is awesome! I tested it on some code that uses some not-compeltely-trivial python features like metaclasses, etc. and it worked like a charm. I couldn't get it to work against this obfuscated hello world though: http://benkurtovic.com/2014/06/01/obfuscating-hello-world.html http://benkurtovic.com/2014/06/01/obfuscating-hello-world.ht... :)

mkesper 12y ago
That one's relying on CPython implementation details, the author says so in his detailed description.

untothebreach 12y ago

This is probably a better link, for those of us (like me) who had never heard of Nuitka: http://nuitka.net/pages/overview.html http://nuitka.net/pages/overview.html

ntrepid8 12y ago
Cached: http://webcache.googleusercontent.com/search?q=cache:0v-pht19f18J:nuitka.net/pages/overview.html http://webcache.googleusercontent.com/search?q=cache:0v-pht1...

bhouston 12y ago

Mirror: http://web.archive.org/web/20141218012211/http://nuitka.net/ http://web.archive.org/web/20141218012211/http://nuitka.net/ BTW how much faster is this than PyPy or CPython? That is the question that I think is most on people's minds.

lucashn 12y ago
Seems to be 2x faster than CPython, but as I understood it is working on optimizations now.
- mkesper 12y ago
  As others have pointed out, these claims have to be taken very carefully and the main value (at least for now) seems to be the easy packaging and deploying.

simonpantzare 12y ago

It seems to be, by and large, the work of a single person on his spare time. Very impressive!

jarcane 12y ago

It is somewhat faster than CPython already, but currently it doesn't make all the optimizations possible, but a 258% factor on pystone is a good start (number is from version 0.3.11). Holy cow. And here I'd set aside Python in part because of the difficulty in getting performant executables (I was getting visible lag in a turn-based SDL game ...)

bhouston 12y ago
But how does it compare to PyPy?
- jarcane 12y ago
  To be honest, I found that in my use case (game dev, supported by bindings to a C library), PyPy was actually slower than CPython. I'd be curious to dig up my code and compare performance to a Nuitka-compiled executable.
  - detaro 12y ago
    out of curiosity, which bindings did you use?
    - jarcane 12y ago
      libtcod. It was a roguelike.
      - ptx 12y ago
        Those Python bindings seem to use ctypes, which is indeed slower[1] on PyPy. They recommend[2] using cffi instead. [1] http://pypy.org/performance.html http://pypy.org/performance.html [2] http://pypy.readthedocs.org/en/latest/extending.html http://pypy.readthedocs.org/en/latest/extending.html
- illumen 12y ago
  For games, other soft realtime, and run once code, pypy is slower. The newer GC is better, but the JIT still causes pauses. The slow warm up of the jit, no type hint saving, and slower interpreter is why it is not good on run once code. Of course if your python extensions are calling optimized assembly routines, or hardware, then it will be faster than pypy code written in python. Also, using a JIT is not possible on some platforms (iOS, or if the CPU architecture isn't supported by pypy). Not to say that pypy isn't better for certain tasks of course.

ntrepid8 12y ago

This project looks completely misguided. The talk focused on the trivialities of mapping Python to C++ rather than on the interesting problems to be encountered when trying to optimize Python while maintaining its extremely dynamic semantics. Also the benchmarking effort is laughable; pystone is not to be taken seriously (only exercises a tiny part of the language) and pybench does microbenchmarks, which are optimized away. You should try the "real-world" benchmarks from the PyPy and Unladen Swallow projects. And what is the size of the generated code? (E.g. how big would the binary for the entire standard library be?) In your blog, please use less boring subjects than "version x.y.z released". ~ Guido van Rossum Find Guido's quote in the first comment. https://ep2013.europython.eu/conference/talks/nuitka-the-python-compiler https://ep2013.europython.eu/conference/talks/nuitka-the-pyt...

fmoralesc 12y ago
Well, that is harsh and somewhat unwarranted ("In your blog, please use..." Who gave him that authority?). I like Hayden's comment on this all: > The good thing [GvR] did to Python is to make his opinion be just that. I can do Nuitka without and beyond his control.[1] [1]: https://ep2013.europython.eu/conference/talks/nuitka-the-python-compiler#comment-2815 https://ep2013.europython.eu/conference/talks/nuitka-the-pyt...
- deleted 12y ago
  [deleted]
orf 12y ago
I like Guido but that comment seems too harsh and commenting on how boring his blog titles are is completely unnecessary. The talk was very interesting and is very neat for only a spare time project.
vegabook 12y ago
So this would be the same Guido van Rossum who made a completely misguided analysis of the language he had birthed, tried to make it "grow up" in version 3.x, brow-beated us all about the dubious benefit of this "new" cruft-encrusted language, causing strategic drift for the entire ecosystem? The best thing that could happen to Python would be a benevolent, and mercifully silent, retirement of GvR. And I speak as a long term Pythonista. Kudos to anybody who is trying to move Python into a performance envelope similar to Lua and Javascript, not to mention Golang, Julia, or Clojure.

zongitsrinzler 12y ago

I don't know much about compilers and runtimes but how come there are SO MANY of them for Python. Is there something about Python that makes it so easy to write runtimes for it?

bane 12y ago
I think it's because lots of people write lots of code in python, but then end up with performance problems at some point and try to tackle those. Have a nice regular language makes it relative easy to rewrite a runtime to scratch your itch. I suspect we would have seen this more with languages like Perl, except Perl is virtually impossible to parse.
bkeroack 12y ago
I think it's largely because while a lot of people really love the language, they struggle with the performance and scaling issues that arise with the standard CPython implementation.
dalke 12y ago
I think it's a matter of numbers. There are a lot of people who use Python. (https://blog.pythonanywhere.com/67/ https://blog.pythonanywhere.com/67/ estimates in the low millions.) Some of them like to work on alternate implementations of Python. I think Python is average in this respect. By comparison, here's an incomplete list of C compilers: http://en.wikipedia.org/wiki/List_of_compilers#C_compilers http://en.wikipedia.org/wiki/List_of_compilers#C_compilers and you can also look at the list of Pascal compilers.
mhd 12y ago
Popularity and a syntax that can be parsed, plus being widely taught. Pretty much the same reason why we had plenty of Pascal compilers back in the days. And quite unlike why we have that many JavaScript transpilers. A few of the compilers might be running on the AST, and thus have no complete alternative implementeation. A good module support for that helps, although it's still not close to VM-based languages like Java.
njharman 12y ago
Python is the best "glue" language. Large 3rd party libs, easily readable/writable (even by non-programmers), ld (it predates Java), established, great user community. So, people in different realms (MSoft IronPython, JVM/Enterprise Jython, restofus CPython, scientists/performance Cython/PyPy,Stackless,Psyco, academics/experimenters many more) all want to use it and make versions that integrate with their tools.

Beltiras 12y ago

Git repo is unresponsive. I think we killed the poor guys box. EDIT: https://github.com/kayhayen/Nuitka https://github.com/kayhayen/Nuitka

insertion 12y ago

I noticed that PyPy does not show any real speed improvement over CPython in these benchmarks: https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/ If you filter by Python, you can compare some results running on both CPython and PyPy. I would be curious to know what it is about these benchmarks that makes PyPy perform poorly. I would also be interested to see how Nuitka performs. At the moment I'm also very excited about Pyston from Dropbox: https://github.com/dropbox/pyston https://github.com/dropbox/pyston

Beltiras 12y ago
I'm supremely sceptical about those benchmarks. Go take a look at the code being tested. I would welcome a serious look at developmental time versus resource used, using code that is probable in production. I read the code used to test Django. It's not reasonable code.
- saym 12y ago
  It's an open sourced benchmarking comparison. If you can improve them, shoot them a pull request! https://github.com/TechEmpower/FrameworkBenchmarks https://github.com/TechEmpower/FrameworkBenchmarks
ant6n 12y ago
I found that PyPy sometimes has unexpected slowdowns. When we were porting from Python to PyPy on some offline processing tools, the most crazy one was building strings via += and sum(arrays,[]), which is much slower than cpython.
- Fede_V 12y ago
  There was a good blog post this by Armin. Basically, if you have to concatenate strings, don't do so via += but use "".join([]).
  - ant6n 12y ago
    I find that unexpected. Java has had string builder optimization for a long time, and CPython is much much faster in this respect. It's not always easy to use "".join when using a string, so you end up having to build a separate array of strings in some cases. And building arrays isn't always that fast either. And [].join doesn't exist, so summing arrays is always kinda slow. Anyway, all that is to say: I really like PyPy, and we use it a lot, but those _unexpected_ crazy slowdowns are unfortunate.

Beltiras 12y ago

I program Django websites and work in an environment where we like to deploy often. I'm not sure we could tolerate the extra wait in compile time for deployment. At least we would need to take a serious look at the speed/resource gain before considering it.

andybak 12y ago
It's been said before but it's fairly unlikely that Python is the bottleneck for typical web-apps.
robertfw 12y ago
Deployment in a controlled environment is not the ideal use case here - the benefits come from when you are shipping to non-technical users.

tfeldmann 12y ago

Great project. Compiling a simple PySide application worked fine on the first try. If you bundle all dependencies you get a binary about the same size as a binary produced by py2exe. I will do some speed comparisons next.

misaelm 12y ago

I built a small app that retrieves data from a remote server, builds a summary and then pushes it to a local html file. Number of people that tried it when I told them that to make it work they had to download python, pip, paramiko and install the pycrypto binary? Zero. Number of people that tried it after I just gave them a zip file with an executable? No longer zero. I tried different libraries but Nuitka was the only one that made everything work seamlessly. I owe this guy a beer. Edit: I no longer owe Kay a beer. I found the donation link on his website.

aidenn0 12y ago
I recently ended up rewriting large parts of a python project in common-lisp due to this exact issue; in this case it was one stage of a pipeline that I prototyped with an existing python library that could output to xml, but delivery of python applications on windows is painful.
president 12y ago
If you packaged it correctly, you could get by just requiring Python and pip to be installed. But still, the zip and single executable is still faster.
sanderjd 12y ago
I think this is one of the reasons Go was a breath of fresh air to a lot of Python and Ruby programmers. The ability to quickly generate a static binary is a great feature.
fernly 12y ago
This issue has been addressed multiple times with considerable success by pyinstaller, py2app, cxfreeze. You can get your single-zip-file distribution package without nuitka, although it may have other advantages.

bane 12y ago

I'm very interested in some benchmarks on this and various other compilers/packagers/runtimes for Python. Sidenote: A very long time ago I had some great luck with Perl's packagers: PAR, PerlApp, etc. but they all basically just bundle in the basic Perl runtime with the guts of your program so there's no real benchmark difference. I suspect the same is not true of efforts like this for Python.

shadowmint 12y ago

For all that people may think this isnt a good idea, this is a very similar idea to what Unity3d is persuing with IL2CPP (http://blogs.unity3d.com/2014/05/20/the-future-of-scripting-in-unity/ http://blogs.unity3d.com/2014/05/20/the-future-of-scripting-...). This seems immidately relevant to address pythons biggest problems; its slow, and its hard to distribute. Relevant quote: All code generation is done to C++ rather than architecture specific machine code. The cost of porting and maintenance of architecture specific code generation is now more amortised

est 12y ago

Python is a fun language, it has as many web frameworks as its runtime/interpreter.

agentultra 12y ago

I would love to make distributing games built on Python easier. Jessica McKeller suggested this should be a priority[0] to make the language more accessible by getting kids involved in making games and being able to share them, easily, with their friends. These days Javascript is kicking our butt in this area. [0] https://www.youtube.com/watch?v=d1a4Jbjc-vU https://www.youtube.com/watch?v=d1a4Jbjc-vU I've been writing little helper libraries on top of Python 3.4 + pysdl2 as I've been working on games and demos for my (unfortunately cancelled) Pycon talk. One area I've only played with, unsuccessfully, is getting packaging going on Nuitka or some other compiler. If anyone wants to get together to make it awesome get in touch.

kedean 12y ago

My immediate question, which I don't see an answer to on the site or Github page: is it self-hosting yet?

avinassh 12y ago

Checked contributors list on Github[0] and most of the code is written by a single person, Kay. Isn't that amazing? A project of this large and value, done by single person in his own time. I wish this project gets more popularity. Bought a beer to Kay :-) [0] - https://github.com/kayhayen/Nuitka/graphs/contributors https://github.com/kayhayen/Nuitka/graphs/contributors

pbiggar 12y ago

I think this is a really solid approach. It's the same thing I was trying to do for PHP with phc (http://phpcompiler.org/ http://phpcompiler.org/) - compiling down to C using the built in stdlib and C API, then using full program optimization. Long term, it won't be as fast as a full JIT, as the Facebook HPHP team showed, but Python doesn't have a JIT of the same caliber, so this is probably useful for tons of people. Dunno if the author is around, but they might find some of the stuff from my PhD relevant, especially how the static analysis worked and some of the challenges of compiling using the C API: http://paulbiggar.com/research/#phd-dissertation http://paulbiggar.com/research/#phd-dissertation

3am 12y ago

Why are so people so fast to trust a new compiler? Have you audited that compiler's source and building the compiler from it with a known good compiler? Are you inspecting the resulting .pyc, and in this case, the resulting PEs? It's a super easy way to inject a compromise into a package that will probably get widely distributed. full disclosure: link is down for me, so haven't read the article. Been a comment that has been building up for a while for me, and not specific to Nuitka. Same goes for new frameworks/languages/etc.

coldtea 12y ago
>Have you audited that compiler's source and building the compiler from it with a known good compiler? Are you inspecting the resulting .pyc, and in this case, the resulting PEs? It's a super easy way to inject a compromise into a package that will probably get widely distributed. Because nobody is that paranoid?
- hughes 12y ago
  And if they are, they probably aren't in the business of distributing precompiled binaries.
andreasvc 12y ago
Why limit yourself to new compilers? Every new version of an old compiler could be suspect, every line of code in general. There's no point in being this paranoid, nobody has time for all the audits that might be theoretically desirable. Almost everything you do on a computer relies on trusting an untold number of components.
waxjar 12y ago
I think it unlikely someone would design something complicated like a new compiler / programming language, open source it and attach their real name to the project just to hide an exploit in it. There's much lower hanging fruit.

HerrMonnezza 12y ago

On a tangentially related note: here is a comparison of various Python runtimes (interpreters and compilers including Nuitka and PyPy) on a fairly complex scientific code: http://arxiv.org/abs/1404.6388 http://arxiv.org/abs/1404.6388 The shootout, however, is from August 2013 so getting a bit old by now.

PythonicAlpha 12y ago

I really like the idea, that it compiles Python but is (99,9%) compatible to CPython -- since so you can still use the many C extensions (I don't want to do my projects any more without my own Python extensions written in C).