4 ms·
Credit to the D developers for providing a concise, carefully-designed library for N-D array processing. The chained method invocations demonstrate D's UFCS (U
by jboy 11y ago
Credit to the D developers for providing a concise, carefully-designed library for N-D array processing. The chained method invocations demonstrate D's UFCS (Uniform Function Call Syntax) nicely. And it's a definite bonus that you can use underscore like a comma separator in long integer literals (eg, `100_000`).
But if you use Python + Numpy/Scipy/Matplotlib and you're looking for a modern, compiled language for execution speedups or greater flexibility than what Numpy broadcasting operations provide by default, I would recommend Nim. It's as fast as C++ or D, it has Pythonic syntax, and it already includes many of D's best features (including type inference, UFCS, and underscores in integer literals).
And best of all, you don't need to rewrite all your existing Python+Numpy code into a new language to start using Nim.
The Pymod library we've created allows you to write Nim functions, compile them as standard CPython extension modules, and simply drop them into your existing Python code: https://github.com/jboy/nim-pymod https://github.com/jboy/nim-pymod
The Pymod library even includes a type `ptr PyArrayObject` that provides native Nim access to Numpy ndarrays via the Numpy C-API [ https://github.com/jboy/nim-pymod#pyarrayobject-type https://github.com/jboy/nim-pymod#pyarrayobject-type ]. So you can bounce back and forth between your Python code and your Nim code for the cost of a Python extension module function call. All of Numpy, Scipy & Matplotlib are still available to you in Python, in addition to statically-typed C++-like iterators in Nim+Pymod [ https://github.com/jboy/nim-pymod#pyarrayiter-types https://github.com/jboy/nim-pymod#pyarrayiter-types , https://github.com/jboy/nim-pymod#pyarrayiter-loop-idioms https://github.com/jboy/nim-pymod#pyarrayiter-loop-idioms ]. The Nim for-loops will be compiled to C code that the C compiler can then auto-vectorize.
- 9il 11y agoD has integration with Python/Matplotlib too =P http://pyd.readthedocs.org http://pyd.readthedocs.org http://d.readthedocs.org/en/latest/examples.html#plotting-with-matplotlib-python http://d.readthedocs.org/en/latest/examples.html#plotting-wi...
- jboy 11y agoIt looks like you need to copy your D array to a newly-allocated Numpy ndarray before you can pass it to Python. So there's no binary PyArrayObject interoperability between D & Python (right?). Copying large N-D arrays all the time sounds slow... (That Matplotlib example uses the function `d_to_python_numpy_ndarray` in the PyD project, which I found defined here: https://github.com/ariovistus/pyd/blob/master/infrastructure/pyd/extra.d#L82 https://github.com/ariovistus/pyd/blob/master/infrastructure... . It clearly allocates a new Numpy array: https://github.com/ariovistus/pyd/blob/master/infrastructure/pyd/extra.d#L97 https://github.com/ariovistus/pyd/blob/master/infrastructure... ) Also, I couldn't find any examples of invoking D functions from Python. In fact, I could only find mentions on the D mailing list of people reporting that they couldn't get it to work: http://forum.dlang.org/post/rdhrvzhhwxgfyxzjevfu@forum.dlang.org http://forum.dlang.org/post/rdhrvzhhwxgfyxzjevfu@forum.dlang... By compiling (transpiling) to C, Nim really does have an unfair advantage in the interoperability challenge...
- 9il 11y agondslice was merged to DLang master repo today. It is not a problem to fix PyD. (compiling to C is crispy) EDIT: Exposing-d-functions-to-python http://pyd.readthedocs.org/en/latest/functions.html#exposing-d-functions-to-python http://pyd.readthedocs.org/en/latest/functions.html#exposing...
- stevieboy 11y agojboy, Can nim-pymod be used as VLA's for nim? I'm not too fond of the nim seq'type (bit slow for my usage) and prefer arrays, but need their length allocated at runtime. Can this be done via (albeit a clunky route) through nim-pymod? i.e arrays created and accessed all in nim (no python)?
- jboy 11y agoHi, the short answer is "Yes, but ...". Yes, Pymod's PyArrayObject can be created & accessed entirely in Nim; yes, its length (actually, shape) is specified at runtime; and yes, it can be resized after creation. However, Pymod's PyArrayObject is designed for maximum binary compatibility with Numpy. As such: 1. It uses Numpy's array creation functions (such as `createSimpleNew`), which in turn uses the Python runtime memory allocator. So at the very least, Pymod assumes you've linked against `-lpythonX.Y`. 2. It integrates with Python's GC rather than Nim's GC. (This is why it's passed around as `ptr PyArrayObject` rather than `ref PyArrayObject` -- it's untouchable by the Nim GC, but instead uses Python refcounts.) Of course, we do intend to extend Pymod to include a pure-Nim sibling for PyArrayObject, but it doesn't exist yet. If you want a high-quality no-Python-dependency VLA in Nim right now, I'd recommend this library by Andrea Ferretti [ https://github.com/unicredit/linear-algebra https://github.com/unicredit/linear-algebra ]. It can link against your system BLAS, in addition to offering GPU support using NVIDIA CUDA. Andrea is another member of the Nim community who does a lot of scientific computing: http://rnduja.github.io/2015/10/21/scientific-nim/ http://rnduja.github.io/2015/10/21/scientific-nim/
- stevieboy 11y agoOk, looking at the linear-algebra VLA stuff now. Thanks for the detailed reply and suggestion. (and apologies to original OP for veering off-topic... the new ndslice package looks like a pretty cool addition the D libraries, I'll be having a look at that too)