5 ms·

I'm using Nim's Pymod https://github.com/jboy/nim-pymod https://github.com/jboy/nim-pymod for this exact purpose and I think it's much better suited. The reaso

by lqdc13 10y ago

I'm using Nim's Pymod https://github.com/jboy/nim-pymod https://github.com/jboy/nim-pymod for this exact purpose and I think it's much better suited. The reason is that it can automatically generate C API code without FFI, because FFI calls are slower (https://gist.github.com/brentp/7e173302952b210aeaf3 https://gist.github.com/brentp/7e173302952b210aeaf3) so there is less overhead. You obviously care about overhead here. Nim's pymod is already a python module and you can send strings and numpy arrays to Nim for fast processing. I wish you could send bytes from python3, but that's not implemented yet.

ngoldbaum 10y ago
Do you have experience with cython? If so, how does Nim compare for writing fast C extensions?
- lqdc13 10y ago
  Since you can basically write C in Cython, it depends how far you take the cython optimization. You can just add a few types, or you can go nuts via scipy's BLAS interface https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/models/word2vec_inner.pyx https://github.com/RaRe-Technologies/gensim/blob/develop/gen... In general, you can get to less overhead with Cython right now because it's a much more mature project and because you can be more flexible with defining the point where you drop from python to a faster alternative. I wouldn't use that Nim/Python lib in production unless you have time to help with its development. It's good for personal projects though.
- the_mitsuhiko 10y ago
  My personal experience with cython (and why I'm less than lukewarm about it) is that it's about as much fun as C to write and only really helps wih he annoyance of dealing with PyObject. That's something that was not even required for the sourcemap lib. Debugging and writing cython that is fast is feally not a very pleasant experience and the tooling is not great. Let alone that a real ecosystem exists. There is not even a good way to deal with dependencies at compile time.
- deleted 10y ago
  [deleted]
pcwalton 10y ago
There's rust-cpython that works like this: https://github.com/dgrunwald/rust-cpython/ https://github.com/dgrunwald/rust-cpython/
- the_mitsuhiko 10y ago
  I'm really not anfan of linking libpython if I can avoid it. More wheels to build and then its limited to cpython. Cffi is plenty fast.
  - fizzbatter 10y ago
    Which method do you find easiest (read: least developer time) for using large rust codebases from Python?
    - the_mitsuhiko 10y ago
      CFFI works quite well. I will probably make a post on my blog later with some ideas on how to do it.
      - sitkack 10y ago
        Not only does it work well, it is portable across nearly all Pythons you would want to use, CPython and PyPy.
- lqdc13 10y ago
  Can you import that as a lib in Python to work in reverse? It seems to be a different use-case otherwise - using a Python library from Rust vs optimizing a hotspot within Python.
  - pcwalton 10y ago
    This page claims you can: http://ehiggs.github.io/2015/07/Python-Modules-In-Rust/ http://ehiggs.github.io/2015/07/Python-Modules-In-Rust/
ihnorton 10y ago
I've looked at Nim a bit, and it does have some nice features. But magic identifier equality [1] is an intrinsically disqualifying design choice. > If a language's lexical structure is so tortured that I need a special tool to grep it, I'm going to skip it and go back to the sanity of C++. [2] Nope, nope, no way. [1] http://nim-lang.org/docs/manual.html#lexical-analysis-identifier-equality http://nim-lang.org/docs/manual.html#lexical-analysis-identi... [2] https://news.ycombinator.com/item?id=8936542 https://news.ycombinator.com/item?id=8936542
- Tarean 10y ago
  I only used nim for some side projects but the only times I saw this used anywhere is in FFI bindings. That way you can mirror, say, C style while writing the bindings either manually or automatically and use them as if they were normal nim elsewhere.

by lqdc13 10y ago

ngoldbaum 10y ago

Do you have experience with cython? If so, how does Nim compare for writing fast C extensions?

lqdc13 10y ago
Since you can basically write C in Cython, it depends how far you take the cython optimization. You can just add a few types, or you can go nuts via scipy's BLAS interface https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/models/word2vec_inner.pyx https://github.com/RaRe-Technologies/gensim/blob/develop/gen... In general, you can get to less overhead with Cython right now because it's a much more mature project and because you can be more flexible with defining the point where you drop from python to a faster alternative. I wouldn't use that Nim/Python lib in production unless you have time to help with its development. It's good for personal projects though.
the_mitsuhiko 10y ago
My personal experience with cython (and why I'm less than lukewarm about it) is that it's about as much fun as C to write and only really helps wih he annoyance of dealing with PyObject. That's something that was not even required for the sourcemap lib. Debugging and writing cython that is fast is feally not a very pleasant experience and the tooling is not great. Let alone that a real ecosystem exists. There is not even a good way to deal with dependencies at compile time.
deleted 10y ago
[deleted]

pcwalton 10y ago

There's rust-cpython that works like this: https://github.com/dgrunwald/rust-cpython/ https://github.com/dgrunwald/rust-cpython/

the_mitsuhiko 10y ago
I'm really not anfan of linking libpython if I can avoid it. More wheels to build and then its limited to cpython. Cffi is plenty fast.
- fizzbatter 10y ago
  Which method do you find easiest (read: least developer time) for using large rust codebases from Python?
  - the_mitsuhiko 10y ago
    CFFI works quite well. I will probably make a post on my blog later with some ideas on how to do it.
    - sitkack 10y ago
      Not only does it work well, it is portable across nearly all Pythons you would want to use, CPython and PyPy.
lqdc13 10y ago
Can you import that as a lib in Python to work in reverse? It seems to be a different use-case otherwise - using a Python library from Rust vs optimizing a hotspot within Python.
- pcwalton 10y ago
  This page claims you can: http://ehiggs.github.io/2015/07/Python-Modules-In-Rust/ http://ehiggs.github.io/2015/07/Python-Modules-In-Rust/

ihnorton 10y ago

I've looked at Nim a bit, and it does have some nice features. But magic identifier equality [1] is an intrinsically disqualifying design choice. > If a language's lexical structure is so tortured that I need a special tool to grep it, I'm going to skip it and go back to the sanity of C++. [2] Nope, nope, no way. [1] http://nim-lang.org/docs/manual.html#lexical-analysis-identifier-equality http://nim-lang.org/docs/manual.html#lexical-analysis-identi... [2] https://news.ycombinator.com/item?id=8936542 https://news.ycombinator.com/item?id=8936542

Tarean 10y ago
I only used nim for some side projects but the only times I saw this used anywhere is in FFI bindings. That way you can mirror, say, C style while writing the bindings either manually or automatically and use them as if they were normal nim elsewhere.