4 ms·

A big use case for python is glue for high performance c/c++ code. How is racket at this? Also, is there a cython alternative for racket? I would also argue tha

by lkirk 7y ago

A big use case for python is glue for high performance c/c++ code. How is racket at this? Also, is there a cython alternative for racket? I would also argue that, though racket may have more batteries included, there are definitely not as many externally developed libraries. Especially important are numpy/scipy/pandas/pysam, the list could go on... That's not to say that I'm against racket, I really like the language and its level of design and documentation.

paroneayea 7y ago
There's a nice FFI for C stuff, but writing glue for C++ is more work.
- deleted 7y ago
  [deleted]
- alexhutcheson 7y ago
  FFI into C++ is a lot of work in any language. A lot of projects just define a plain C wrapper for the C++ code they want to call, and call that C code from whatever their glue language is. However, Python does have CLIF[1], which is the nicest solution for calling into C++ that I'm aware of. [1] https://github.com/google/clif https://github.com/google/clif
  - jdc 7y ago
    Have you seen cppyy? https://cppyy.readthedocs.io/en/latest/ https://cppyy.readthedocs.io/en/latest/
    - alexhutcheson 7y ago
      Not in enough detail to have an informed opinion.
  - winter_blue 7y ago
    There's also Boost.Python, which boasts "seamless interoperability between C++ and Python": https://www.boost.org/doc/libs/1_70_0/libs/python/doc/html/index.html https://www.boost.org/doc/libs/1_70_0/libs/python/doc/html/i... (In general, the Boost C++ libraries are well-renowned in the C++ world.)
    - lkirk 7y ago
      I think the biggest thing, for me is Cython. I've not seen anything quite like it in other languages. It allows you to compile python code to c, with gradual typing. It also allows you to write c code inline w/ your python or interface with other C/C++ libraries. https://cython.org/ https://cython.org/ Other languages will be pressed to beat its utility (esp in the scientific computing world)
    - jefft255 7y ago
      In my experience it’s far from what I’d call seamless but it’s definitely good enough
    - jononor 7y ago
      pybind11 is a modern version of the Boost Python approach to Python bindings. Header only and can be installed trivially from PyPi. I like it well enough that I even use it for binding C code. Especially because it has nice numpy array support (albeit a little underdocumented).
  - pjmlp 7y ago
    On Windows it is much better, thanks to COM and now UWP, which improves COM a lot regarding what is exposed across the ABI. Yes, bare bones COM is full of boilerplate, however there are more productive ways to use it.
  - reallydude 7y ago
    Nicest for Python, I assume. Many languages have FFI. Lua, PHP, etc. https://en.wikipedia.org/wiki/Foreign_function_interface https://en.wikipedia.org/wiki/Foreign_function_interface I don't understand the choice of Python for things like gluing together C programs. Seems like a performant mistake, at the very least.
    - todd8 7y ago
      It’s the 80:20 rule, which naturally doesn’t always holds. Often, however, a small part of the code is responsible for almost all of the performance issues. By picking the performance sensitive areas of the program to code in C one can often code the rest of the program in a slower more convenient language.
  - MereInterest 7y ago
    How does CLIF compare to PyBind11[1]? [1] https://github.com/pybind/pybind11 https://github.com/pybind/pybind11
pavpanchekha 7y ago
I've used the Racket FFI for some important tasks, and it's very easy to use. However, it's not as fully baked as the Python one (I've found bugs due to GC interactions) and it's weirdly slow (a few too many levels of wrapping). That said, I still use Racket extensively and enjoy it.
- neilv 7y ago
  You found bugs in the FFI mechanism itself, or in FFI bindings that someone had written for a particular library?
  - pavpanchekha 7y ago
    In the interaction between FFI, GC, and hash tables, so yes, in the FFI itself, not in particular bindings: https://github.com/racket/racket/issues/2702 https://github.com/racket/racket/issues/2702 https://github.com/racket/racket/issues/2263 https://github.com/racket/racket/issues/2263 There were workarounds (in Racket you can instruct the GC not to move your FFI objects) and I believe the underlying issues have been fixed. I must add that I got immediate and comprehensive support from the core developers, who not only fixed the bug but also suggested the workaround (so I could continue to support all our target versions of Racket).
    - neilv 7y ago
      Thank you for your great bug report on this, and I'm glad to see it was in good hands.
- yomly 7y ago
  This is the write up I was looking for as this is the use case I've had in mind for Racket of late after reading Felleisen's LOP paper. Would love to know more about your experiences with doing Racket FFI. How hard were the bugs around GC to discover and workaround? What were you building in Racket?
  - pavpanchekha 7y ago
0xcde4c3db 7y ago
While it's nowhere near as mature as Racket, those interested in this use case might also be interested in Clasp, a Common Lisp implementation built on LLVM specifically to enable the combination of Lisp-style dynamic development approaches with C++ libraries. https://github.com/clasp-developers/clasp https://github.com/clasp-developers/clasp https://www.youtube.com/watch?v=mbdXeRBbgDM https://www.youtube.com/watch?v=mbdXeRBbgDM

by lkirk 7y ago

paroneayea 7y ago

There's a nice FFI for C stuff, but writing glue for C++ is more work.

deleted 7y ago
[deleted]
alexhutcheson 7y ago
FFI into C++ is a lot of work in any language. A lot of projects just define a plain C wrapper for the C++ code they want to call, and call that C code from whatever their glue language is. However, Python does have CLIF[1], which is the nicest solution for calling into C++ that I'm aware of. [1] https://github.com/google/clif https://github.com/google/clif
- jdc 7y ago
  Have you seen cppyy? https://cppyy.readthedocs.io/en/latest/ https://cppyy.readthedocs.io/en/latest/
  - alexhutcheson 7y ago
    Not in enough detail to have an informed opinion.
- winter_blue 7y ago
  There's also Boost.Python, which boasts "seamless interoperability between C++ and Python": https://www.boost.org/doc/libs/1_70_0/libs/python/doc/html/index.html https://www.boost.org/doc/libs/1_70_0/libs/python/doc/html/i... (In general, the Boost C++ libraries are well-renowned in the C++ world.)
  - lkirk 7y ago
    I think the biggest thing, for me is Cython. I've not seen anything quite like it in other languages. It allows you to compile python code to c, with gradual typing. It also allows you to write c code inline w/ your python or interface with other C/C++ libraries. https://cython.org/ https://cython.org/ Other languages will be pressed to beat its utility (esp in the scientific computing world)
  - jefft255 7y ago
    In my experience it’s far from what I’d call seamless but it’s definitely good enough
  - jononor 7y ago
    pybind11 is a modern version of the Boost Python approach to Python bindings. Header only and can be installed trivially from PyPi. I like it well enough that I even use it for binding C code. Especially because it has nice numpy array support (albeit a little underdocumented).
- pjmlp 7y ago
  On Windows it is much better, thanks to COM and now UWP, which improves COM a lot regarding what is exposed across the ABI. Yes, bare bones COM is full of boilerplate, however there are more productive ways to use it.
- reallydude 7y ago
  Nicest for Python, I assume. Many languages have FFI. Lua, PHP, etc. https://en.wikipedia.org/wiki/Foreign_function_interface https://en.wikipedia.org/wiki/Foreign_function_interface I don't understand the choice of Python for things like gluing together C programs. Seems like a performant mistake, at the very least.
  - todd8 7y ago
    It’s the 80:20 rule, which naturally doesn’t always holds. Often, however, a small part of the code is responsible for almost all of the performance issues. By picking the performance sensitive areas of the program to code in C one can often code the rest of the program in a slower more convenient language.
- MereInterest 7y ago
  How does CLIF compare to PyBind11[1]? [1] https://github.com/pybind/pybind11 https://github.com/pybind/pybind11

pavpanchekha 7y ago

I've used the Racket FFI for some important tasks, and it's very easy to use. However, it's not as fully baked as the Python one (I've found bugs due to GC interactions) and it's weirdly slow (a few too many levels of wrapping). That said, I still use Racket extensively and enjoy it.

neilv 7y ago
You found bugs in the FFI mechanism itself, or in FFI bindings that someone had written for a particular library?
- pavpanchekha 7y ago
  In the interaction between FFI, GC, and hash tables, so yes, in the FFI itself, not in particular bindings: https://github.com/racket/racket/issues/2702 https://github.com/racket/racket/issues/2702 https://github.com/racket/racket/issues/2263 https://github.com/racket/racket/issues/2263 There were workarounds (in Racket you can instruct the GC not to move your FFI objects) and I believe the underlying issues have been fixed. I must add that I got immediate and comprehensive support from the core developers, who not only fixed the bug but also suggested the workaround (so I could continue to support all our target versions of Racket).
  - neilv 7y ago
    Thank you for your great bug report on this, and I'm glad to see it was in good hands.
yomly 7y ago
This is the write up I was looking for as this is the use case I've had in mind for Racket of late after reading Felleisen's LOP paper. Would love to know more about your experiences with doing Racket FFI. How hard were the bugs around GC to discover and workaround? What were you building in Racket?
- pavpanchekha 7y ago

0xcde4c3db 7y ago

While it's nowhere near as mature as Racket, those interested in this use case might also be interested in Clasp, a Common Lisp implementation built on LLVM specifically to enable the combination of Lisp-style dynamic development approaches with C++ libraries. https://github.com/clasp-developers/clasp https://github.com/clasp-developers/clasp https://www.youtube.com/watch?v=mbdXeRBbgDM https://www.youtube.com/watch?v=mbdXeRBbgDM