6 ms·

I've been using boost::python, and the experience is what I'd call cargo-cult programming. You copy and paste stuff off the tutorial, and when you get a compile

I've been using boost::python, and the experience is what I'd call cargo-cult programming. You copy and paste stuff off the tutorial, and when you get a compiler error, you just try other stuff at random, because the inner workings of the library are inscrutable. I vastly prefer cython to this. Is Pybind11 any better?

autopoiesis 9y ago
Pybind is derived from boost::python, so it's not much different. I wrote a large Python binding in the past for a moving C++ library target, using boost::python, and keeping the C++ binding code up to date was a nightmare akin to maintaining a fork of any fast-moving project. Try cppyy [1]. It's very nice, though quite fresh. It's used extensively (as far as I can tell) at CERN, and derives from their cling C++ JIT interpreter. Plus it does lots of nice automatic transformations to make a C++ API more pythonic. [1] http://cppyy.readthedocs.io/en/latest/index.html http://cppyy.readthedocs.io/en/latest/index.html
- deleted 9y ago
  [deleted]
- wjakob 9y ago
  Not quite -- while pybind11 was originally inspired by boost::python, it is an entirely different project that is intended to address many of its problems.
  - autopoiesis 9y ago
    Indeed; I was using "derived" somewhat loosely. But I think there is certainly a visible lineage there. Indeed, pybind11 is much nicer than boost::python (and I did evaluate it for the future of the project I mentioned above). However, the very nice feature of cppyy is that it does much (most?) of what pybind11 does, but it can also be completely on-the-fly, in the sense that it relies on the cling JIT interpreter. This means that there is absolutely no need to maintain a compiled C++ part for your bindings, and so the problem of keeping the interface up to date is greatly mitigated: the equivalent changes to match the interface when using cppyy are _much_ smaller. Often, one maintains a "C++ interface" layer in one's Python bindings (which could be created by pybind11, boost::python, or SWIG, for example), with a pure Python shim layer on top of that. cppyy allows you to do away with this two-layer structure entirely; all you need is the shim, if you need anything at all.
    - aldanor 9y ago
      The catch with cling and the derived libraries is in that you have to download a whole bunch [1] of CERN stuff and then build a customized LLVM as part of the build process. That's a bit too heavy for the nice reflection- and REPL-like features that you gain. [1] https://github.com/antocuni/cppyy-backend/blob/master/create_src_directory.py Boost.Python is better in that regard since you "just" have to build boost; on some platforms, you can just snatch that via a package manager; that being said, you still need to build it. SWIG, aside from being ugly, requires an extra build step. > This means that there is absolutely no need to maintain a compiled C++ part for your bindings, and so the problem of keeping the interface up to date is greatly mitigated I tend to disagree. I would never consider the raw (swig or cling) 1-to-1 bindings of C++ code satisfactory for end-user use in Python. Ideally (in my subjective opinion and previous experience) Python-side bindings would closely mirror the C++ API, to the point where downstream code in either language looks very similar, but they don't reference any C++ stuff, be it vectors, or maps, or template arguments, or anything else. This implies you would have to maintain a set of higher level bindings on top of swig/cling ones anyway -- and these are the ones that'll break as the code evolves and that you'll have to maintain manually. As such, I'd rather maintain one set of bindings than two.
      - autopoiesis 9y ago
        You're right about the CERN stuff, though recent efforts seem to have been made to split at least some of that out. I hope that it continues. I think, aside from anything else, that cling is a really cool project, and if it could be easily available more widely, that would be great. I was being quite literal when I wrote "no need to maintain a compiled C++ part": of course, you probably do want to maintain /some/ extra layer! And, in that sense, I do think that cppyy lets you maintain just one set of bindings (not two); and, as in pybind11, the ultimate aim is to transparently translate any "vectors, or maps, or template arguments" into idiomatic Python: this is why cppyy has a 'Pythonization' API. Perhaps there are just two slightly different niches: cppyy is good when you need more a interactive interface to C++, for prototyping or exploration (for example), because of its JIT nature; and pybind11 is good for building something more static in the longer term, and where you don't mind the cost of keeping the compiled part up to date with the relevant C++ API. It's certainly an interesting space at the moment, and I do hope both projects keep the momentum up and keep innovating!
zwieback 9y ago
I'm curious - are you more of a Python or C++ programmer? C++ compiler errors can be famously cryptic so I'm wondering if these types of binding-generators are mostly suitable for C++ programmers that want to expose their libs to Python and don't mind decoding template errors in their own code. I've done both C++ and Python and never attempted to generate bindings, in the near future I may be faced with this though.
wjakob 9y ago
pybind11 shares some of the motivation of boost::python, but it is designed to be considerably easier to use. Out of the box, it is aware of a large set of C++ types that are automatically translated to their Python equivalents and vice versa (STL data structures, C++17 types like optional<> and variant<>, std::function<> for callbacks, and even sparse and dense matrices for numerical linear algebra). @aldanor recently gave a nice overview talk of what pybind11 can do: https://speakerdeck.com/aldanor/pybind11-seamless-operability-between-c-plus-plus-11-and-python https://speakerdeck.com/aldanor/pybind11-seamless-operabilit... Also, take a look at our documentation (http://pybind11.readthedocs.io http://pybind11.readthedocs.io) -- it contains detailed examples of many tricky use cases.
arximboldi 9y ago
I have used boost::python a lot and later tried pybind11. A lot of people prise it because they consider boost bloated, but in my experiments boost::python is actually 2X faster at runtime (not at compile time). I think I'd rather use boost::python.

autopoiesis 9y ago

Pybind is derived from boost::python, so it's not much different. I wrote a large Python binding in the past for a moving C++ library target, using boost::python, and keeping the C++ binding code up to date was a nightmare akin to maintaining a fork of any fast-moving project. Try cppyy [1]. It's very nice, though quite fresh. It's used extensively (as far as I can tell) at CERN, and derives from their cling C++ JIT interpreter. Plus it does lots of nice automatic transformations to make a C++ API more pythonic. [1] http://cppyy.readthedocs.io/en/latest/index.html http://cppyy.readthedocs.io/en/latest/index.html

deleted 9y ago
[deleted]
wjakob 9y ago
Not quite -- while pybind11 was originally inspired by boost::python, it is an entirely different project that is intended to address many of its problems.
- autopoiesis 9y ago
  Indeed; I was using "derived" somewhat loosely. But I think there is certainly a visible lineage there. Indeed, pybind11 is much nicer than boost::python (and I did evaluate it for the future of the project I mentioned above). However, the very nice feature of cppyy is that it does much (most?) of what pybind11 does, but it can also be completely on-the-fly, in the sense that it relies on the cling JIT interpreter. This means that there is absolutely no need to maintain a compiled C++ part for your bindings, and so the problem of keeping the interface up to date is greatly mitigated: the equivalent changes to match the interface when using cppyy are _much_ smaller. Often, one maintains a "C++ interface" layer in one's Python bindings (which could be created by pybind11, boost::python, or SWIG, for example), with a pure Python shim layer on top of that. cppyy allows you to do away with this two-layer structure entirely; all you need is the shim, if you need anything at all.
  - aldanor 9y ago
    The catch with cling and the derived libraries is in that you have to download a whole bunch [1] of CERN stuff and then build a customized LLVM as part of the build process. That's a bit too heavy for the nice reflection- and REPL-like features that you gain. [1] https://github.com/antocuni/cppyy-backend/blob/master/create_src_directory.py Boost.Python is better in that regard since you "just" have to build boost; on some platforms, you can just snatch that via a package manager; that being said, you still need to build it. SWIG, aside from being ugly, requires an extra build step. > This means that there is absolutely no need to maintain a compiled C++ part for your bindings, and so the problem of keeping the interface up to date is greatly mitigated I tend to disagree. I would never consider the raw (swig or cling) 1-to-1 bindings of C++ code satisfactory for end-user use in Python. Ideally (in my subjective opinion and previous experience) Python-side bindings would closely mirror the C++ API, to the point where downstream code in either language looks very similar, but they don't reference any C++ stuff, be it vectors, or maps, or template arguments, or anything else. This implies you would have to maintain a set of higher level bindings on top of swig/cling ones anyway -- and these are the ones that'll break as the code evolves and that you'll have to maintain manually. As such, I'd rather maintain one set of bindings than two.
    - autopoiesis 9y ago
      You're right about the CERN stuff, though recent efforts seem to have been made to split at least some of that out. I hope that it continues. I think, aside from anything else, that cling is a really cool project, and if it could be easily available more widely, that would be great. I was being quite literal when I wrote "no need to maintain a compiled C++ part": of course, you probably do want to maintain /some/ extra layer! And, in that sense, I do think that cppyy lets you maintain just one set of bindings (not two); and, as in pybind11, the ultimate aim is to transparently translate any "vectors, or maps, or template arguments" into idiomatic Python: this is why cppyy has a 'Pythonization' API. Perhaps there are just two slightly different niches: cppyy is good when you need more a interactive interface to C++, for prototyping or exploration (for example), because of its JIT nature; and pybind11 is good for building something more static in the longer term, and where you don't mind the cost of keeping the compiled part up to date with the relevant C++ API. It's certainly an interesting space at the moment, and I do hope both projects keep the momentum up and keep innovating!

zwieback 9y ago

I'm curious - are you more of a Python or C++ programmer? C++ compiler errors can be famously cryptic so I'm wondering if these types of binding-generators are mostly suitable for C++ programmers that want to expose their libs to Python and don't mind decoding template errors in their own code. I've done both C++ and Python and never attempted to generate bindings, in the near future I may be faced with this though.

wjakob 9y ago

pybind11 shares some of the motivation of boost::python, but it is designed to be considerably easier to use. Out of the box, it is aware of a large set of C++ types that are automatically translated to their Python equivalents and vice versa (STL data structures, C++17 types like optional<> and variant<>, std::function<> for callbacks, and even sparse and dense matrices for numerical linear algebra). @aldanor recently gave a nice overview talk of what pybind11 can do: https://speakerdeck.com/aldanor/pybind11-seamless-operability-between-c-plus-plus-11-and-python https://speakerdeck.com/aldanor/pybind11-seamless-operabilit... Also, take a look at our documentation (http://pybind11.readthedocs.io http://pybind11.readthedocs.io) -- it contains detailed examples of many tricky use cases.

arximboldi 9y ago

I have used boost::python a lot and later tried pybind11. A lot of people prise it because they consider boost bloated, but in my experiments boost::python is actually 2X faster at runtime (not at compile time). I think I'd rather use boost::python.