5 ms·

Cython is much easier to implement than people realize. A few import statements and a few types, and then you're good to go. I urge people to give it a try.

Cython is much easier to implement than people realize. A few import statements and a few types, and then you're good to go. I urge people to give it a try. The speedups can be dramatic.

tgb 5y ago
It certainly is good, but I've never found a great dev setup. It seems like you have to force a full recompile of all cython in the project any time you make a change. At least that was the stated process for the statsmodels library. And it was always a little unclear whether I was running the latest code or hadn't yet actually compiled it.
- mywittyname 5y ago
  I've never used cython before. Is it 1-to-1 with standard Python? If so, can you do development using the regular Python binary, and leave the compilation step as a pre-commit operation?
  - shoo 5y ago
    The workflow with Cython is that it produces native python modules (e.g. .so shared objects / .dll dynamic link libraries) that the python interpreter can import. So if you change some of your Cython code, for it to be used at runtime you need to invoke the Cython build tools to rebuild the new version of your python native module. I usually use Cython for a small core of compute heavy operations, and leave the rest of the project as pure python. That way I only need to rebuild Cython code if I change something inside that small core. > Is it 1-to-1 with standard Python? Not for the best speedups, no. E.g. you might be able to get a modest speedup, say 50%, taking a pure python file, renaming it to *.pyx, and getting Cython to compile it. But that's not why I use Cython. I use it when I have compute-heavy code that I want to run at native speed (think matrix-vector product type stuff), by carefully rewriting in Cython, thinking carefully about memory allocation, data structures (prefer C arrays!) and performance, it is fairly achievable to get a 500x speedup. Cython relies on you writing specialised Cython code that is quite close to C code -- strongly typed Cython variables work like statically typed C variables, not dynamically typed Python names. You end up with Cython code that cannot be executed as if it were normal Python code by a python interpreter. But, Python code can usually not be executed very efficiently, whereas Cython can translate small loops of strongly-typed numeric code into small loops of strongly-typed C code, which can often compile to very small loops of native CPU instructions, which then run blazing fast. Under the hood, Cython works by translating the not-quite-python code into C code that uses the python interpreter's C extension API. Then it compiles the C code into a python native module using a C compiler.
    - mywittyname 5y ago
      Thanks for pointing this out. I didn't realize this until I read further into this thread and started looking at optimized cython projects. Now I understand what you're talking about.
  - doubleunplussed 5y ago
    It's not 1-1. Cython can compile regular Python, but you only really get significant speedups when you declare the types of things using cython-specific syntax.

Cython is much easier to implement than people realize. A few import statements and a few types, and then you're good to go. I urge people to give it a try. The speedups can be dramatic.

tgb 5y ago

It certainly is good, but I've never found a great dev setup. It seems like you have to force a full recompile of all cython in the project any time you make a change. At least that was the stated process for the statsmodels library. And it was always a little unclear whether I was running the latest code or hadn't yet actually compiled it.

mywittyname 5y ago
I've never used cython before. Is it 1-to-1 with standard Python? If so, can you do development using the regular Python binary, and leave the compilation step as a pre-commit operation?
- shoo 5y ago
  The workflow with Cython is that it produces native python modules (e.g. .so shared objects / .dll dynamic link libraries) that the python interpreter can import. So if you change some of your Cython code, for it to be used at runtime you need to invoke the Cython build tools to rebuild the new version of your python native module. I usually use Cython for a small core of compute heavy operations, and leave the rest of the project as pure python. That way I only need to rebuild Cython code if I change something inside that small core. > Is it 1-to-1 with standard Python? Not for the best speedups, no. E.g. you might be able to get a modest speedup, say 50%, taking a pure python file, renaming it to *.pyx, and getting Cython to compile it. But that's not why I use Cython. I use it when I have compute-heavy code that I want to run at native speed (think matrix-vector product type stuff), by carefully rewriting in Cython, thinking carefully about memory allocation, data structures (prefer C arrays!) and performance, it is fairly achievable to get a 500x speedup. Cython relies on you writing specialised Cython code that is quite close to C code -- strongly typed Cython variables work like statically typed C variables, not dynamically typed Python names. You end up with Cython code that cannot be executed as if it were normal Python code by a python interpreter. But, Python code can usually not be executed very efficiently, whereas Cython can translate small loops of strongly-typed numeric code into small loops of strongly-typed C code, which can often compile to very small loops of native CPU instructions, which then run blazing fast. Under the hood, Cython works by translating the not-quite-python code into C code that uses the python interpreter's C extension API. Then it compiles the C code into a python native module using a C compiler.
  - mywittyname 5y ago
    Thanks for pointing this out. I didn't realize this until I read further into this thread and started looking at optimized cython projects. Now I understand what you're talking about.
- doubleunplussed 5y ago
  It's not 1-1. Cython can compile regular Python, but you only really get significant speedups when you declare the types of things using cython-specific syntax.