12 ms·

Nuitka Progress in 2015 – Python Compiler

166 points by rachbelaid 11y ago

pcx 11y ago
This looks pretty cool, know of any production-grade software using it? I remember reading about something like this being used for Docker Compose, can't recollect what exactly that was.
amelius 11y ago
Trying to understand this project. Is this more than a python compiler? Edit: Ok, now I see that the "Overview" button is hidden on mobile.
- deleted 11y ago
  [deleted]
- mseebach 11y ago
  So, from my reading of the "Overview" page, it compiles Python to C++ and then use native types as much as possible to make things run really fast. http://nuitka.net/pages/overview.html http://nuitka.net/pages/overview.html
_yy 11y ago
> The stable release has full support for Python 3.5, including the new async and await functions. So recent releases can pronounce it as fully supported which was quite a feat. Wow. That is amazing! Pyston only supports Python 2 and PyPy is working on it. Nuitka looks very promising.
objectified 11y ago
IMHO, this is a very important project as it makes up for one of the biggest shortcomings in the Python ecosystem - distribution of software. Distribution is clearly one of the reasons that Go is so popular. It would be great if we could take advantage of Python as a language and Python as an ecosystem while still being able to deploy as if it was a walk in the park.
- fulafel 11y ago
  Last time I needed single-executable distribution with Python (admittedly a while ago), there were easy to use "freeze" tools for this that worked really well. A cursory web search suggests that cx_Freeze is the current popular tool for it. Has the situation tehnically deteriorated or are people unaware of these Python tools now?
  - Loic 11y ago
    PyInstaller[0] is working really well. I am using it to deliver Python application in big companies setup as a single ".exe" without the need to install anything on the target machine. The only drawback is that you have the decompression time when starting the software, but in my case, the customers have not noticed it yet. Edit: The advantages of Nuitka are that as you compile your code to C++, you make it faster and also protect it against easy reverse engineering. This could be interesting to compile the critical parts of your application with Nuitka and pack the rest with PyInstaller. [0]: http://www.pyinstaller.org/ http://www.pyinstaller.org/
  - scarygliders 11y ago
    cx_Freeze is still very effective for distribution - I use it all the time to produce a standalone version of my software and then I use Inno Setup to create an installer package for my customers. As Loic states, Nuitka works differently from cx_Freeze in that Nuitka takes your Python source, compiles that to C++, then compiles the C++, whereas cx_Freeeze creates Python bytecode which is subsequently run by the included Python interpreter. The result should be that the same Python code should run much faster in the form created by Nuitka, than the result created by cx_Freeze.
    - falcolas 11y ago
      > The result should be that the same Python code should run much faster in the form created by Nuitka How do you figure? Python's slowness is not due to it's lack of compilation, it's due to its dynamic nature and all of the runtime lookups. Past efforts to compile python down to bytecode have not resulted in speedups. Unladen Swallow is one such failed example. PyPy gets around this by analyzing the actual running code and is only able to speed up a subset of all of Python.
truncate 11y ago
Looking at their future plan, can we say they are aiming to build something like RPython, but instead of a subset of Python support full Python language?
michaelmior 11y ago
> SSA (Single State Assignment Form) Is this something different from static single assignment? I'm assuming not, but I've never heard SSA expanded in this form.
- UniQP 11y ago
  You are right, this should be static single assignment.
wibr 11y ago
This looks very similar to Cython, which can also compile .py files without modification and is more mature at the moment. It seems that Nuitka wants to do a few things differently, though. The biggest difference is probably that Nuitka wants to use type inference + hints instead of explicit declarations, which make Cython code incompatible with CPython but give you more control and C interoperability. Edit: Thanks, no more interference!
- qb45 11y ago
  I've never heard about type inTERference, but I'm sure that if something like that existed, it would have little to do with type inference which you probably meant :) interference - obstruction, collision inference - deduction, derivation
  - mikeash 11y ago
    I imagine that type interference might be what an advocate of dynamically typed languages sees in statically typed languages.
- p4wnc6 11y ago
  Cython supports type inference, in the form of `infer_types` which can be used either as a compiler directive or a decorator (which means you can pick and choose, and control, when and where type inference is allowed to occur).
- sametmax 11y ago
  No. Actually nuikta can take your core "as-is". It doesn't need any work of your part at all. And it embeds dependancies as well, without any change needed for them either. That's the game changer : take your Python program, call nuikta, get a standalone exe.
ptype 11y ago
Does anyone know if this works with numpy?
- witty_username 11y ago
  It works with numpy. (I tried it and it seems to be 100% compatible, I haven't noticed any differences)
scarygliders 11y ago
It's been great watching Nuitka progress over the past couple of years. I suggested the tl;dr section to Kay in the Overview ;) I've also donated to the guy. I'd help out in the development, but to be honest a lot of what he's doing goes way over my head and I don't have the time to sit down and try to understand it all. I encourage others better than me to help in Nuitka's development :)
gtrubetskoy 11y ago
It's fantastic to see someone taking on the hard problems, things that take years to solve, especially if you're on your own. This is the essence of Open Source - take your time to do something right. Great stuff, good luck with it, we need more grand undertakings like this! (This comment is mainly based on this talk: https://www.youtube.com/watch?v=a8RRbT4BTEw https://www.youtube.com/watch?v=a8RRbT4BTEw)
entrean 11y ago
I believe this project is filling the single largest gap that Python has for development of widely-distributed desktop applications. This is pure awesomeness.
AlphaSite 11y ago
I wonder how the performance of this compares to ZiPy.
disillusioned1 11y ago
The Requirements section of the user manual says that in addition to a C++ compiler, you need Python2 at compile time even if you're using Python3, due to a requirement of the Scons tool.

166 points by rachbelaid 11y ago

pcx 11y ago

This looks pretty cool, know of any production-grade software using it? I remember reading about something like this being used for Docker Compose, can't recollect what exactly that was.

amelius 11y ago

Trying to understand this project. Is this more than a python compiler? Edit: Ok, now I see that the "Overview" button is hidden on mobile.

deleted 11y ago
[deleted]
mseebach 11y ago
So, from my reading of the "Overview" page, it compiles Python to C++ and then use native types as much as possible to make things run really fast. http://nuitka.net/pages/overview.html http://nuitka.net/pages/overview.html

_yy 11y ago

> The stable release has full support for Python 3.5, including the new async and await functions. So recent releases can pronounce it as fully supported which was quite a feat. Wow. That is amazing! Pyston only supports Python 2 and PyPy is working on it. Nuitka looks very promising.

objectified 11y ago

IMHO, this is a very important project as it makes up for one of the biggest shortcomings in the Python ecosystem - distribution of software. Distribution is clearly one of the reasons that Go is so popular. It would be great if we could take advantage of Python as a language and Python as an ecosystem while still being able to deploy as if it was a walk in the park.

fulafel 11y ago
Last time I needed single-executable distribution with Python (admittedly a while ago), there were easy to use "freeze" tools for this that worked really well. A cursory web search suggests that cx_Freeze is the current popular tool for it. Has the situation tehnically deteriorated or are people unaware of these Python tools now?
- Loic 11y ago
  PyInstaller[0] is working really well. I am using it to deliver Python application in big companies setup as a single ".exe" without the need to install anything on the target machine. The only drawback is that you have the decompression time when starting the software, but in my case, the customers have not noticed it yet. Edit: The advantages of Nuitka are that as you compile your code to C++, you make it faster and also protect it against easy reverse engineering. This could be interesting to compile the critical parts of your application with Nuitka and pack the rest with PyInstaller. [0]: http://www.pyinstaller.org/ http://www.pyinstaller.org/
- scarygliders 11y ago
  cx_Freeze is still very effective for distribution - I use it all the time to produce a standalone version of my software and then I use Inno Setup to create an installer package for my customers. As Loic states, Nuitka works differently from cx_Freeze in that Nuitka takes your Python source, compiles that to C++, then compiles the C++, whereas cx_Freeeze creates Python bytecode which is subsequently run by the included Python interpreter. The result should be that the same Python code should run much faster in the form created by Nuitka, than the result created by cx_Freeze.
  - falcolas 11y ago
    > The result should be that the same Python code should run much faster in the form created by Nuitka How do you figure? Python's slowness is not due to it's lack of compilation, it's due to its dynamic nature and all of the runtime lookups. Past efforts to compile python down to bytecode have not resulted in speedups. Unladen Swallow is one such failed example. PyPy gets around this by analyzing the actual running code and is only able to speed up a subset of all of Python.

truncate 11y ago

Looking at their future plan, can we say they are aiming to build something like RPython, but instead of a subset of Python support full Python language?

michaelmior 11y ago

> SSA (Single State Assignment Form) Is this something different from static single assignment? I'm assuming not, but I've never heard SSA expanded in this form.

UniQP 11y ago
You are right, this should be static single assignment.

wibr 11y ago

This looks very similar to Cython, which can also compile .py files without modification and is more mature at the moment. It seems that Nuitka wants to do a few things differently, though. The biggest difference is probably that Nuitka wants to use type inference + hints instead of explicit declarations, which make Cython code incompatible with CPython but give you more control and C interoperability. Edit: Thanks, no more interference!

qb45 11y ago
I've never heard about type inTERference, but I'm sure that if something like that existed, it would have little to do with type inference which you probably meant :) interference - obstruction, collision inference - deduction, derivation
- mikeash 11y ago
  I imagine that type interference might be what an advocate of dynamically typed languages sees in statically typed languages.
p4wnc6 11y ago
Cython supports type inference, in the form of `infer_types` which can be used either as a compiler directive or a decorator (which means you can pick and choose, and control, when and where type inference is allowed to occur).
sametmax 11y ago
No. Actually nuikta can take your core "as-is". It doesn't need any work of your part at all. And it embeds dependancies as well, without any change needed for them either. That's the game changer : take your Python program, call nuikta, get a standalone exe.

ptype 11y ago

Does anyone know if this works with numpy?

witty_username 11y ago
It works with numpy. (I tried it and it seems to be 100% compatible, I haven't noticed any differences)

scarygliders 11y ago

It's been great watching Nuitka progress over the past couple of years. I suggested the tl;dr section to Kay in the Overview ;) I've also donated to the guy. I'd help out in the development, but to be honest a lot of what he's doing goes way over my head and I don't have the time to sit down and try to understand it all. I encourage others better than me to help in Nuitka's development :)

gtrubetskoy 11y ago

It's fantastic to see someone taking on the hard problems, things that take years to solve, especially if you're on your own. This is the essence of Open Source - take your time to do something right. Great stuff, good luck with it, we need more grand undertakings like this! (This comment is mainly based on this talk: https://www.youtube.com/watch?v=a8RRbT4BTEw https://www.youtube.com/watch?v=a8RRbT4BTEw)

entrean 11y ago

I believe this project is filling the single largest gap that Python has for development of widely-distributed desktop applications. This is pure awesomeness.

AlphaSite 11y ago

I wonder how the performance of this compares to ZiPy.

disillusioned1 11y ago

The Requirements section of the user manual says that in addition to a C++ compiler, you need Python2 at compile time even if you're using Python3, due to a requirement of the Scons tool.