5 ms·

As a longtime programmer of Python, C & C++ (and a longtime appreciator of your FQA Lite, by the way), my preference until recently was also to use CPython & fa

by jboy 11y ago

As a longtime programmer of Python, C & C++ (and a longtime appreciator of your FQA Lite, by the way), my preference until recently was also to use CPython & fall back on extension modules for speed. But I grew weary of writing all the Python C-API boilerplate -- especially checking parameter types and managing reference counts properly. So I created (and recently open-sourced) Pymod, a Nim+Python project that auto-generates all the Python C-API boilerplate & auto-compiles a Python C extension module, to wrap the functions in a Nim module: https://github.com/jboy/nim-pymod https://github.com/jboy/nim-pymod As a pragmatic Python programmer, Pymod may be of interest to you too... (As someone who learned C++ in 1999 and spent most of 200x working in C++, I also grew weary of staying on top of C++0x's new features & new gotchas. So about a year ago, I switched from C++ to Nim-lang and haven't looked back. To me, Nim combines all the best features of Python & C++, while avoiding most of the worst: C++ speed, C++ static types, C++ generics, C++ operator overloading; combined with a clean Pythonic syntax & Lispy macros.)

maxerickson 11y ago
Cython (and before it Pyrex) occupies a similar space between C and Python. Was any of your design inspired by them? I also wonder how practical it would be to reuse parts of it when making a bridge for another language.
- jboy 11y ago
  Actually, Pymod was designed to be almost an anti-Cython. :) My issue with Cython is that it's a limited sub-language within Python, where you add Cython elements incrementally & iteratively (diverging from Python in the process) until the code runs "fast enough". I'd rather work directly in a full-featured, internally-self-consistent language from the start. Nim has a clean Pythonic syntax, with all the best parts of C++ (including its runtime speed). Hence, Pymod takes the form of an `exportpy` annotation (a user-defined Nim pragma) onto existing Nim functions, which are then auto-compiled into a Python extension module. So there's no gradual divergence of my Python code (as it becomes more "Cythonized"); rather, the high-performance code is written directly in pure Nim. :)
  - maxerickson 11y ago
    Right, but Cython also supports the external library use case: http://docs.cython.org/src/tutorial/clibraries.html http://docs.cython.org/src/tutorial/clibraries.html
    - jboy 11y ago
      Interesting! Thanks for the link. After browsing that page, I observe that even when Cython is wrapping an external C library, there's still a notable difference between the way Cython does things & the way Pymod does things. The explanation on that Cython page begins: "To get started, the first step is to redefine the C API in a .pxd file, say, cqueue.pxd". So to wrap an external C library using Cython, you still need to define an entirely new header-like definition file in Cython's intermediate language. In contrast, Pymod only requires an `exportpy` pragma annotation at the end of an existing Nim procedure function header -- you don't need to create any intermediate definition files. And the `exportpy` pragma is inert unless you invoke the "pmgen.py" script (a Python script in Pymod that determines the Python C-API system configuration & automates all the compilation), so your Nim code is still completely valid Nim code after you apply the `exportpy` annotation.
      - maxerickson 11y ago
        I'm now largely out of my depth, but much of that will just come down to what is available from Nim and what is available from C? And perhaps some of it from the complexity of supporting arbitrary C code in a flexible way? The Cython wrapper code is still pretty trivial compared to the C code ( https://github.com/fragglet/c-algorithms/blob/master/src/queue.c https://github.com/fragglet/c-algorithms/blob/master/src/que... ), and this is a pretty small piece of C.
        jboy 11y ago
        Oh absolutely, Nim should definitely get the vast majority of the credit. It's not just because Nim compiles to C; it's also due to Nim's powerful macro system, which: (1) allows me to define my own first-class pragmas such as `exportpy`; (2) supplies my pragma with a detailed & expressive AST of the Nim function onto which my pragma was annotated; (3) enables my pragma to auto-generate the C-API boilerplate code, and write it to disk as a newly-created C source file, all within the Nim compilation pass. Nim is a fantastic language, perfectly suited to this scenario. I merely spotted an opportunity. :)

by jboy 11y ago

maxerickson 11y ago

Cython (and before it Pyrex) occupies a similar space between C and Python. Was any of your design inspired by them? I also wonder how practical it would be to reuse parts of it when making a bridge for another language.

jboy 11y ago
Actually, Pymod was designed to be almost an anti-Cython. :) My issue with Cython is that it's a limited sub-language within Python, where you add Cython elements incrementally & iteratively (diverging from Python in the process) until the code runs "fast enough". I'd rather work directly in a full-featured, internally-self-consistent language from the start. Nim has a clean Pythonic syntax, with all the best parts of C++ (including its runtime speed). Hence, Pymod takes the form of an `exportpy` annotation (a user-defined Nim pragma) onto existing Nim functions, which are then auto-compiled into a Python extension module. So there's no gradual divergence of my Python code (as it becomes more "Cythonized"); rather, the high-performance code is written directly in pure Nim. :)
- maxerickson 11y ago
  Right, but Cython also supports the external library use case: http://docs.cython.org/src/tutorial/clibraries.html http://docs.cython.org/src/tutorial/clibraries.html
  - jboy 11y ago
    Interesting! Thanks for the link. After browsing that page, I observe that even when Cython is wrapping an external C library, there's still a notable difference between the way Cython does things & the way Pymod does things. The explanation on that Cython page begins: "To get started, the first step is to redefine the C API in a .pxd file, say, cqueue.pxd". So to wrap an external C library using Cython, you still need to define an entirely new header-like definition file in Cython's intermediate language. In contrast, Pymod only requires an `exportpy` pragma annotation at the end of an existing Nim procedure function header -- you don't need to create any intermediate definition files. And the `exportpy` pragma is inert unless you invoke the "pmgen.py" script (a Python script in Pymod that determines the Python C-API system configuration & automates all the compilation), so your Nim code is still completely valid Nim code after you apply the `exportpy` annotation.
    - maxerickson 11y ago
      I'm now largely out of my depth, but much of that will just come down to what is available from Nim and what is available from C? And perhaps some of it from the complexity of supporting arbitrary C code in a flexible way? The Cython wrapper code is still pretty trivial compared to the C code ( https://github.com/fragglet/c-algorithms/blob/master/src/queue.c https://github.com/fragglet/c-algorithms/blob/master/src/que... ), and this is a pretty small piece of C.
      - jboy 11y ago
        Oh absolutely, Nim should definitely get the vast majority of the credit. It's not just because Nim compiles to C; it's also due to Nim's powerful macro system, which: (1) allows me to define my own first-class pragmas such as `exportpy`; (2) supplies my pragma with a detailed & expressive AST of the Nim function onto which my pragma was annotated; (3) enables my pragma to auto-generate the C-API boilerplate code, and write it to disk as a newly-created C source file, all within the Nim compilation pass. Nim is a fantastic language, perfectly suited to this scenario. I merely spotted an opportunity. :)