3 ms·

> For interfacing with C, I think cython is better than cffi. I think we'll have to agree to disagree on this. To quote the CFFI project description [0]: the g

by _dps 13y ago

> For interfacing with C, I think cython is better than cffi. I think we'll have to agree to disagree on this. To quote the CFFI project description [0]: the goal is to be able to seamlessly call C from Python without having to learn a third language/API. Cython, Ctypes, and CPython's C API all fail to meet this criterion, and in my opinion do so by a large margin. Let's also not forget the fact that Cython requires a compilation step not only for the bound C code but also for the portion of the "Python" code that interfaces with it. Compare that (or writing CPython C API manipulation code) with the following: >>> from cffi import FFI >>> ffi = FFI() >>> ffi.cdef("int printf(const char *format, ...);") >>> C = ffi.dlopen(None) # loads the entire C namespace >>> arg = ffi.new("char[]", "world") # char arg[] = "world"; >>> C.printf("hi there, %s!\n", arg) # call printf hi there, world! For me CFFI wins without question. Of course, it's even simpler in LuaJIT (but the CFFI people are getting closer every day): local ffi = require "ffi" ffi.cdef [[ int printf(const char*, ...); ]] ffi.C.printf("hi there, %s!\n", "world") Now, it would be misleading for me not to point out the limitations of an approach like CFFI/LJ-FFI vs CPython API: you may have to contort yourself to make C calls that safely manipulate the scripting language VM (e.g. instantiate new Python objects from within a C FFI call). In my experience this has not been a serious problem because it's easy to write struct-to-object mappers in idiomatic Python or Lua rather than having to muck about with PyObject* and friends. Of course, in LuaJIT it's even easier with FFI metatypes (which let you attach dispatch tables to FFI native types, so your structs can behave just like Lua tables with method dispatch, inheritance, etc.). Anyway, leaving personal taste aside for a moment, let me rephrase the original question that sparked my interest in your opinion: in a world with a mature CFFI option providing the above binding capabilities do you think that easy C extensions are still a worthwhile tradeoff for the GIL? (FWIW I find the point to be quite compelling for why Python had a GIL to begin with in the late 90s, but I'm suspecting it's less of an intrinsic design tradeoff and more historical cruft as the PyPy and CFFI people roll out their work). [0] http://cffi.readthedocs.org/en/release-0.6/ http://cffi.readthedocs.org/en/release-0.6/

cdavid 13y ago
I don't think the points you highlighted above matter fundamentally: they don't handle memory management for you, and that's the difficulty, especially at the language boundary. What is interesting for me in Lua is the stack-based argument passing and the GC: it avoids leaking the reference counting and the binary representation of objects (I think, I am rather clueless when it comes to Lua). Wrapping things like crypto, file format and co is relatively trivial, because not much crosses the language barrier. Now, when you need to handle non native object life cycle, that's another matter, and I don't see how cffi makes it any easier than cython
- _dps 13y ago
  This thread has probably gone too long, and I thank you for your time and comments so far. I will close by saying that, while I agree with you that CFFI doesn't make the problem of writing memory-safe C extensions to the Python VM any easier, I also believe that almost no one (aside from people like Numpy developers) actually need to write a Python extension. They just write extensions (or use Cython, or Ctypes) because that's "the way" to call C from Python. In my personal experience 90+% of the Python+C problems in the world are just about calling an existing piece of C code and maybe mapping some structs or arrays into Python types; these workflows rarely involve instantiating complex objects on the Python side. For my work habits neither CPython, Ctypes, nor Cython are optimized for this extremely common use case. CFFI solves these problems for me better than any of those alternatives, and if someone offered me a Python 4.0 with CFFI and no GIL at the cost of harder-to-use extension mechanisms I'd jump on it; I suspect that a significant portion of the people who do Python/C interop work would feel similarly. I also do realize that such a change could make projects like Numpy harder to build.

by _dps 13y ago

cdavid 13y ago

I don't think the points you highlighted above matter fundamentally: they don't handle memory management for you, and that's the difficulty, especially at the language boundary. What is interesting for me in Lua is the stack-based argument passing and the GC: it avoids leaking the reference counting and the binary representation of objects (I think, I am rather clueless when it comes to Lua). Wrapping things like crypto, file format and co is relatively trivial, because not much crosses the language barrier. Now, when you need to handle non native object life cycle, that's another matter, and I don't see how cffi makes it any easier than cython

_dps 13y ago
This thread has probably gone too long, and I thank you for your time and comments so far. I will close by saying that, while I agree with you that CFFI doesn't make the problem of writing memory-safe C extensions to the Python VM any easier, I also believe that almost no one (aside from people like Numpy developers) actually need to write a Python extension. They just write extensions (or use Cython, or Ctypes) because that's "the way" to call C from Python. In my personal experience 90+% of the Python+C problems in the world are just about calling an existing piece of C code and maybe mapping some structs or arrays into Python types; these workflows rarely involve instantiating complex objects on the Python side. For my work habits neither CPython, Ctypes, nor Cython are optimized for this extremely common use case. CFFI solves these problems for me better than any of those alternatives, and if someone offered me a Python 4.0 with CFFI and no GIL at the cost of harder-to-use extension mechanisms I'd jump on it; I suspect that a significant portion of the people who do Python/C interop work would feel similarly. I also do realize that such a change could make projects like Numpy harder to build.