5 ms·

How is this more compatible than just python? Since it requires libpython so you anyway need to have python installed on the target machine? Also is this actua

How is this more compatible than just python? Since it requires libpython so you anyway need to have python installed on the target machine? Also is this actually faster than just python? I tried it out with some bad expensive looping using lists with python3.9 and it was 100ms slower (1.3sec vs 1.4sec). Just in general why should I be using Nuitka?

nightcracker 5y ago
I think the idea behind Nuitka is to compile Python into an executable, rather than interpret a script using a fully generic interpreter, but without (or with minimal) limitations, falling back to libpython's implementation if necessary. > In the future Nuitka will be able to use type inferencing based on whole program analysis. It will apply that information in order to perform as many calculations as possible in C, using C native types, without accessing libpython. I believe the difference with PyPy is that PyPy tries to do this using just-in-time compilation and nuitka uses ahead-of-time compilation.
BiteCode_dev 5y ago
nuitka takes your python code and all its deps, turns it to c code, and compiles that into a standalone executable that is both faster and doesn't need a separate python vm to be installed.
- nextlevelwizard 5y ago
  Expect it does require libpython and at least on Ubuntu 20.04 it links against the dev version of the library. So not only can you not run the resulting binary on any random Ubuntu 20.04 you also need to install same version of the python and the "-dev" version. I did try this. With the "stock" python3 (3.8.something) I couldn't get nuitka to compile my code (or the example from the docs). With python3.9 I could start the compilation, but I didn't have libpython-dev shared object and finally with python3.9-dev I was able to make the binary. However after transfering it to another Ubuntu 20.04 machine I couldn't run it since it was linked against libpython3.9-dev. Just as side note. The same python code (i.e. the .py file) would run on both machines out of the box.
  - paiute 5y ago
    it has an option to build with libpython. so your comment is incorrect.
    - nextlevelwizard 5y ago
      I guess there is a good reason for it to not be default behavior
  - BiteCode_dev 5y ago
    If you pass the proper option, it will compile everything, including libpython, resulting in a stand alone executable. The only dep would be libc, and if you compile against an old version, it should be forward compatible.
eatonphil 5y ago
> Also is this actually faster than just python? I tried it out with some bad expensive looping using lists with python3.9 and it was 100ms slower (1.3sec vs 1.4sec). Mind sharing that code? I'm sure someone would like to be able to look into this.
- elmolino89 5y ago
  I also got a slower than rub by cpython a nuitka produced binary. Granted, it is a complicated beast with several external python libraries, numpy, others sprinkled with cython, etc. Here is the original repo I have tried to speed up using: python -m nuitka --clang --follow-imports main.py repo: https://github.com/MRCIEU/gwas2vcf https://github.com/MRCIEU/gwas2vcf If someone can make the program run faster by whatever means, it will make a bunch of people quite happy.
- nextlevelwizard 5y ago
  I don't have it anymore, but it was something stupid like: import time start = time.time() list = [] for i in range(10000000): list.append(i) sum = 0 for item in list: sum = sum + i print(time.time() - start) I am not superduper python expert, but I know that arrays/lists are the stupidest thing you can use so that's why I did stupid things with them. So before someone posts more optimized version the code was suppose to be shit since I assumed that the compiler would optimize it. I expected the binary to essentially be 1. start = time.time() 2. pre-populated list since it should be constant e.g. list = [1,2,3,...] or completely remove it since it is not used anywhere 3. pre-filled sum since it too is constant or same as above, removed since it is not used anywhere 4. getting second time.time() and printting the difference But there was no compilation time optimization. Granted this wasn't promised, but I just assumed it was in there since it is compiling.
  - emmelaich 5y ago
    Maybe it was the size of the list. Have you tried xrange?
    - 5y ago

nightcracker 5y ago

I think the idea behind Nuitka is to compile Python into an executable, rather than interpret a script using a fully generic interpreter, but without (or with minimal) limitations, falling back to libpython's implementation if necessary. > In the future Nuitka will be able to use type inferencing based on whole program analysis. It will apply that information in order to perform as many calculations as possible in C, using C native types, without accessing libpython. I believe the difference with PyPy is that PyPy tries to do this using just-in-time compilation and nuitka uses ahead-of-time compilation.

BiteCode_dev 5y ago

nuitka takes your python code and all its deps, turns it to c code, and compiles that into a standalone executable that is both faster and doesn't need a separate python vm to be installed.

nextlevelwizard 5y ago
Expect it does require libpython and at least on Ubuntu 20.04 it links against the dev version of the library. So not only can you not run the resulting binary on any random Ubuntu 20.04 you also need to install same version of the python and the "-dev" version. I did try this. With the "stock" python3 (3.8.something) I couldn't get nuitka to compile my code (or the example from the docs). With python3.9 I could start the compilation, but I didn't have libpython-dev shared object and finally with python3.9-dev I was able to make the binary. However after transfering it to another Ubuntu 20.04 machine I couldn't run it since it was linked against libpython3.9-dev. Just as side note. The same python code (i.e. the .py file) would run on both machines out of the box.
- paiute 5y ago
  it has an option to build with libpython. so your comment is incorrect.
  - nextlevelwizard 5y ago
    I guess there is a good reason for it to not be default behavior
- BiteCode_dev 5y ago
  If you pass the proper option, it will compile everything, including libpython, resulting in a stand alone executable. The only dep would be libc, and if you compile against an old version, it should be forward compatible.

eatonphil 5y ago

> Also is this actually faster than just python? I tried it out with some bad expensive looping using lists with python3.9 and it was 100ms slower (1.3sec vs 1.4sec). Mind sharing that code? I'm sure someone would like to be able to look into this.

elmolino89 5y ago
I also got a slower than rub by cpython a nuitka produced binary. Granted, it is a complicated beast with several external python libraries, numpy, others sprinkled with cython, etc. Here is the original repo I have tried to speed up using: python -m nuitka --clang --follow-imports main.py repo: https://github.com/MRCIEU/gwas2vcf https://github.com/MRCIEU/gwas2vcf If someone can make the program run faster by whatever means, it will make a bunch of people quite happy.
nextlevelwizard 5y ago
I don't have it anymore, but it was something stupid like: import time start = time.time() list = [] for i in range(10000000): list.append(i) sum = 0 for item in list: sum = sum + i print(time.time() - start) I am not superduper python expert, but I know that arrays/lists are the stupidest thing you can use so that's why I did stupid things with them. So before someone posts more optimized version the code was suppose to be shit since I assumed that the compiler would optimize it. I expected the binary to essentially be 1. start = time.time() 2. pre-populated list since it should be constant e.g. list = [1,2,3,...] or completely remove it since it is not used anywhere 3. pre-filled sum since it too is constant or same as above, removed since it is not used anywhere 4. getting second time.time() and printting the difference But there was no compilation time optimization. Granted this wasn't promised, but I just assumed it was in there since it is compiling.
- emmelaich 5y ago
  Maybe it was the size of the list. Have you tried xrange?
  - 5y ago