4 ms·

I hope that the "pinning" work in the GC code is a first step for better CPython extension support. Classic C interop requires that some memory pointed from the

by jerven 12y ago

I hope that the "pinning" work in the GC code is a first step for better CPython extension support. Classic C interop requires that some memory pointed from the C side in the Python heap stores the same thing. i.e. is not moved. Pinning is one way to do this. Otherwise the same idea that Jruby/Graal did with co interpreting C code can work for PyPy as well (http://www.chrisseaton.com/rubytruffle/cext/ http://www.chrisseaton.com/rubytruffle/cext/). Although that would need someone to step up and implement a RPython C interpreter, which to be honest is not a trivial exercise.

masklinn 12y ago
That's not really a solution when using precompiled native extensions. Not much of a solution either when the native extension (precompiled or not) is a wrapper against a separate native library (which may or may not be coded in C). Basically, it only works with "native accelerators", libraries which provide a pure {language} implementation as well as an optional self-contained C version of a subset of the library. Which is a case where you'd hope a high-quality JITed implementation would have a good speed on the "pure" implementation (and would probably prefer work be done on improving the JIT to make the pure version fast)
- chrisseaton 12y ago
  It doesn't work against precompiled native extensions, but it does work against wrappers against separate native libraries, like a database driver or libpng or something like that. The C interpreter can interface with native code, and the JIT has special support for making those calls very fast [1] (it's not like JNI, for example). So the only thing it doesn't work for is precompiled native extensions where you don't have these source code - so maybe a proprietary C extension. Do you get many of those in practice? [1] http://dl.acm.org/citation.cfm?id=2500832 http://dl.acm.org/citation.cfm?id=2500832
  - masklinn 12y ago
    > So the only thing it doesn't work for is precompiled native extensions where you don't have these source code Or you can't build it because you're on windows, or the extension was installed precompiled by the package manager (on Python, that's exactly one of the use case for wheels, I don't think wheels ship C code ever), or the extension is coded in non-C (which is very much explicitly supported by CPython: https://docs.python.org/3/extending/extending.html#writing-extensions-in-c https://docs.python.org/3/extending/extending.html#writing-e...)
    - maxerickson 12y ago
      Microsoft has done work to make the Windows situation at least incrementally better: https://mail.python.org/pipermail/distutils-sig/2014-September/024885.html https://mail.python.org/pipermail/distutils-sig/2014-Septemb... (I'm crassly attaching a bit of information to your post more than trying to respond to anything in it)

by jerven 12y ago

masklinn 12y ago

That's not really a solution when using precompiled native extensions. Not much of a solution either when the native extension (precompiled or not) is a wrapper against a separate native library (which may or may not be coded in C). Basically, it only works with "native accelerators", libraries which provide a pure {language} implementation as well as an optional self-contained C version of a subset of the library. Which is a case where you'd hope a high-quality JITed implementation would have a good speed on the "pure" implementation (and would probably prefer work be done on improving the JIT to make the pure version fast)

chrisseaton 12y ago
It doesn't work against precompiled native extensions, but it does work against wrappers against separate native libraries, like a database driver or libpng or something like that. The C interpreter can interface with native code, and the JIT has special support for making those calls very fast [1] (it's not like JNI, for example). So the only thing it doesn't work for is precompiled native extensions where you don't have these source code - so maybe a proprietary C extension. Do you get many of those in practice? [1] http://dl.acm.org/citation.cfm?id=2500832 http://dl.acm.org/citation.cfm?id=2500832
- masklinn 12y ago
  > So the only thing it doesn't work for is precompiled native extensions where you don't have these source code Or you can't build it because you're on windows, or the extension was installed precompiled by the package manager (on Python, that's exactly one of the use case for wheels, I don't think wheels ship C code ever), or the extension is coded in non-C (which is very much explicitly supported by CPython: https://docs.python.org/3/extending/extending.html#writing-extensions-in-c https://docs.python.org/3/extending/extending.html#writing-e...)
  - maxerickson 12y ago
    Microsoft has done work to make the Windows situation at least incrementally better: https://mail.python.org/pipermail/distutils-sig/2014-September/024885.html https://mail.python.org/pipermail/distutils-sig/2014-Septemb... (I'm crassly attaching a bit of information to your post more than trying to respond to anything in it)