4 ms·

Python extension language using accelerators

47 points by precsim 5y ago

mkesper 5y ago
From the Quickstart Guide (https://github.com/pyccel/pyccel/blob/master/tutorial/quickstart.md https://github.com/pyccel/pyccel/blob/master/tutorial/quicks...): Pyccel's main goal is to resolve the principal bottleneck in scientific computing: the transition from prototype to production. Programmers usually develop their prototype code in a user-friendly interactive language like Python, but their final application requires an HPC implementation and therefore a new production code. In most cases this is written in a statically compiled language like Fortran/C/C++, and it uses SIMD vectorization, parallel multi-threading, MPI parallelization, GPU offloading, etc. Sounds interesting!
toxik 5y ago
The two other related projects I know are Numba and Cython. Worth checking them out as well.
- qorrect 5y ago
  Some similar projects are Pyston , PyPy and Rapids.ai Pyston I was able to get a 15% increase in speed out of the box. PyPy I could not get to work with my project (numpy/pandas/ta-lib), I finally got everything to compile but it blew through my 32 gigs in 10 seconds and crashed. Pyccel, Cython, Numba and Rapids I have yet to try but am interested in anyones experience.
  - bb1234 5y ago
    If you are going to rewrite, you may want to consider Julia. It will give you close to C speed, but will be as high level (or higher level) than Python.
  - a-dub 5y ago
    numba is pretty great, and i saw pretty extreme speedups (i can't remember exactly now, but they were orders of magnitude faster) when i used it in a project. however, to make full use of it, you pretty much have to isolate the numerical code that you want it to compile and be careful to ensure that it is easily translatable. (that is, avoid using libraries or python objects). essentially, for full effect, you need to turn on "nopython" mode which then severely limits what you can do in the accelerated code. ultimately, i'd probably look at julia or maybe rust if i was building something from scratch these days though. i'll also note that adding numba to a project is a pretty heavyweight proposition (you are, afterall, adding llvm to your project).
- 6gvONxR4sf7o 5y ago
  Cython is so great. The two easiest ways I know to speed up python are 1) can I write this arraywise with enough cleverness (it's much more frequently doable than I always expect, and often comes with clarity gains) and just use numpy, and 2) just use cython. If cython wasn't such a pain in the ass to debug, maybe we'd all be writing cython instead of python today.
  - jkrubin 5y ago
    That is exactly my experience with cython. It’s a huge pain in the ass to debug. I found pybind11 to be way less of a pain in the ass than cython. In most cases figuring out how to do it with nutty numpy is the least time consuming and is fast enough.
- abarker 5y ago
  Some other similar projects are Pythran, Transonic, and mypyc.

47 points by precsim 5y ago

From the Quickstart Guide (https://github.com/pyccel/pyccel/blob/master/tutorial/quickstart.md https://github.com/pyccel/pyccel/blob/master/tutorial/quicks...): Pyccel's main goal is to resolve the principal bottleneck in scientific computing: the transition from prototype to production. Programmers usually develop their prototype code in a user-friendly interactive language like Python, but their final application requires an HPC implementation and therefore a new production code. In most cases this is written in a statically compiled language like Fortran/C/C++, and it uses SIMD vectorization, parallel multi-threading, MPI parallelization, GPU offloading, etc. Sounds interesting!

toxik 5y ago

The two other related projects I know are Numba and Cython. Worth checking them out as well.

qorrect 5y ago
Some similar projects are Pyston , PyPy and Rapids.ai Pyston I was able to get a 15% increase in speed out of the box. PyPy I could not get to work with my project (numpy/pandas/ta-lib), I finally got everything to compile but it blew through my 32 gigs in 10 seconds and crashed. Pyccel, Cython, Numba and Rapids I have yet to try but am interested in anyones experience.
- bb1234 5y ago
  If you are going to rewrite, you may want to consider Julia. It will give you close to C speed, but will be as high level (or higher level) than Python.
- a-dub 5y ago
  numba is pretty great, and i saw pretty extreme speedups (i can't remember exactly now, but they were orders of magnitude faster) when i used it in a project. however, to make full use of it, you pretty much have to isolate the numerical code that you want it to compile and be careful to ensure that it is easily translatable. (that is, avoid using libraries or python objects). essentially, for full effect, you need to turn on "nopython" mode which then severely limits what you can do in the accelerated code. ultimately, i'd probably look at julia or maybe rust if i was building something from scratch these days though. i'll also note that adding numba to a project is a pretty heavyweight proposition (you are, afterall, adding llvm to your project).
6gvONxR4sf7o 5y ago
Cython is so great. The two easiest ways I know to speed up python are 1) can I write this arraywise with enough cleverness (it's much more frequently doable than I always expect, and often comes with clarity gains) and just use numpy, and 2) just use cython. If cython wasn't such a pain in the ass to debug, maybe we'd all be writing cython instead of python today.
- jkrubin 5y ago
  That is exactly my experience with cython. It’s a huge pain in the ass to debug. I found pybind11 to be way less of a pain in the ass than cython. In most cases figuring out how to do it with nutty numpy is the least time consuming and is fast enough.
abarker 5y ago
Some other similar projects are Pythran, Transonic, and mypyc.