6 ms·

This is true for _libpython_ (the shared library version), which is the default on some distros (RedHat, Fedora, Arch), but many others (Debian, Ubuntu) use sta

This is true for _libpython_ (the shared library version), which is the default on some distros (RedHat, Fedora, Arch), but many others (Debian, Ubuntu) use statically linked Python and never paid this performance tax.

dheera 5y ago
I think using Pypy instead of CPython will give you several times the performance boost as any of this.
- dathinab 5y ago
  Likely, but it doesn't work with all applications.
- dharmab 5y ago
  Pypy is not a drop-in replacement for CPython. It does not support many libraries that rely on C extensions, and targets a slightly older version of the language.
- nwmcsween 5y ago
  Assuming pypy does what it says it does: Python -> RPython -> C -> LLVM Clang -> LLVM IR -> 'JIT', you're still paying a large tax compared to something like RPython -> Some IR -> 'JIT'
  - joshuamorton 5y ago
    I think you've misinterpreted something. The pypy interpreter is written in rpython, but executed code itself never gets translated to rpython.
- coldtea 5y ago
  Now you have 10 problems. A reference to the classic line, oft attributed to JWZ: "Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems". The point being, switching from CPython to PyPy does not simply give you the performance boost and that's it. It comes with its own tradeoffs, including: (a) trading speed for much more memory consumption (b) slower C FFI - important for all kinds of Python workflows (e.g. Pandas, Numpy, and so on) (c) behind mainline CPython releases, and with subtle incompatibilities (d) slower startup times (due to the JITting involved) (e) different garbage collection model (and less predictable) (f) less support (from companies, distros, etc), fewer ports, less manpower to port quickly to new platforms (e.g. Apple's M1).
usr1106 5y ago
So what do Redhat, Fedora, Arch gain by using the shared libpython? Who except python is using the library?
- mjw1007 5y ago
  Software that embeds Python uses the library (even on Debian and Ubuntu: they end up building the whole Python source base twice). For example Vim (if so configured), or Postgres PL/Python, or the old Apache mod_python.
  - usr1106 5y ago
    Interesting. Performancewise not having to call a separate executable is of course a good thing. Securitywise process boundaries are better.

dheera 5y ago

I think using Pypy instead of CPython will give you several times the performance boost as any of this.

dathinab 5y ago
Likely, but it doesn't work with all applications.
dharmab 5y ago
Pypy is not a drop-in replacement for CPython. It does not support many libraries that rely on C extensions, and targets a slightly older version of the language.
nwmcsween 5y ago
Assuming pypy does what it says it does: Python -> RPython -> C -> LLVM Clang -> LLVM IR -> 'JIT', you're still paying a large tax compared to something like RPython -> Some IR -> 'JIT'
- joshuamorton 5y ago
  I think you've misinterpreted something. The pypy interpreter is written in rpython, but executed code itself never gets translated to rpython.
coldtea 5y ago
Now you have 10 problems. A reference to the classic line, oft attributed to JWZ: "Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems". The point being, switching from CPython to PyPy does not simply give you the performance boost and that's it. It comes with its own tradeoffs, including: (a) trading speed for much more memory consumption (b) slower C FFI - important for all kinds of Python workflows (e.g. Pandas, Numpy, and so on) (c) behind mainline CPython releases, and with subtle incompatibilities (d) slower startup times (due to the JITting involved) (e) different garbage collection model (and less predictable) (f) less support (from companies, distros, etc), fewer ports, less manpower to port quickly to new platforms (e.g. Apple's M1).

usr1106 5y ago

So what do Redhat, Fedora, Arch gain by using the shared libpython? Who except python is using the library?

mjw1007 5y ago
Software that embeds Python uses the library (even on Debian and Ubuntu: they end up building the whole Python source base twice). For example Vim (if so configured), or Postgres PL/Python, or the old Apache mod_python.
- usr1106 5y ago
  Interesting. Performancewise not having to call a separate executable is of course a good thing. Securitywise process boundaries are better.