4 ms·
Cython 0.23 released
- stesch 11y agoPeople should really think about the message a 0.x version number is sending. This project is 8 years old. Why is this still 0.x?
- klocksib 11y agoI definitely agree; as someone who uses Python daily, and previously not knowing what Cython is, my first thought was of a new, very immature project.
- nickpsecurity 11y agoThat's a good point. I'd wonder if it had any production quality seeing that number. Yet, most reviews I read talk like it's beta quality or better for their uses. Hmm.
- williamstein 11y agoI'm the guy who started the Cython project and made up the name, so here are some random remarks that don't answer your question. Not only is Cython about 8 years old, it started as a fork of Pyrex (http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/ http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/), which is itself over 11 years old, since I remember using it in 2004 when evaluating languages in which to implement http://sagemath.org http://sagemath.org. The people who did the most work on Cython on probably Stefan Behnel and Robert Bradshaw (who was my Ph.D. student). I had to sort of twist their arms to get them to be involved in the project, and they both still only do it in their spare time basically for fun. Despite Cython being an incredibly useful and powerful tool for certain specific problem domains (e.g., for SageMath it is critical), and being very heavily used in some scientific python applications, there's no company behind Cython and few people involved in the project are interested in publicity. Cython is an open source labor of love and an extremely useful tool. People that know Cython very well, know how amazingly powerful it is, and have watched many other attempts to solve similar problems come and go over the years.
- PythonicAlpha 11y agoI agree, that it is extremely useful, since before I knew Cython, I implemented some Python C extension modules by hand. That can become rather ugly and costs much sweat. Today, I can use Cython. Or when I need even more speed, than I use Cython as wrapper-generator for a pure C module, that is still much, much easier to implement, as when I would have to do the wrapping work myself. Cython is really enormous useful and precious to me! Thank you for it!
- deleted 11y ago[deleted]
- maxerickson 11y agoThe problem is that there is no pattern of version numbers that is meaningfully informative. It's pretty much necessary to figure out how reliable the releases are for each project that you are interested in. For people that aren't following along, 0.23 might have slightly worse optics than 1.2, but there is plenty of software called 1.2 that is crap.
- raymondh 11y agoI'm glad to see that this import project continues to get updates.
- PythonicAlpha 11y agoTo answer the question, that came up: Yes, the 0.x number also put me down at first. But I am using this tool now several years, since 0.13 or maybe earlier and did never find a bug or problem. Cython is great, in-spite of its version number! I also inspected the generated code several times and it looks rock-solid. Also one big advantage (for me): It can produce code that runs equally on 2.x and 3.x versions (there are some limitations, though, like several really outdated versions are not supported any more).
- grinalope 11y agoCython is amazingly productive for me. I am so happy to write typed pythonic code. If there's one thing missing it's more people showing off their Cython style and tricks, like how they do macros (since it's not supported in the language), when to turn off safeties, and balancing pure python with typed. For instance, I like C macros in header files, but I've played around with using a jinja2 pipeline in my setup.py to write cleaner looking pythonic code, and have done generics this way. more posts like the one using cymem would be welcome.
- syllogism 11y agoUsing Jinja2 for macros is an interesting idea. You could try cog instead? https://www.python.org/about/success/cog/ https://www.python.org/about/success/cog/ Currently I don't do any macros at all, although the temptation is definitely strong. I have some code that could otherwise be more generic than it is. I guess a quick tip/pattern would be to use structs as data containers, with a cdef class defined as a "shell", to provide access, but not actually hold the data. Advantages of this over standard cdef class: Data is contiguous in memory; Data can be stack allocated; Data can be stored in C arrays or C++ containers; Data can be created without GIL. Advantages of this over numpy array: More Pythonic syntax (i.e. data.attr, instead of data[column]; No fussing with memorybuffers, compiler declarations etc; Data can be created without GIL Disadvantages: More code (I wrote that cymem blog post -- I'll write up this pattern as a post when I get a chance.)
- ketralnis 11y agoIf you use Python for anything remotely performance sensitive, I highly recommend giving Cython a shot. The trick is to find your computation-intensive inner loop stuff, move it off to its own module, add types, and go. It makes your build a little more complicated (i.e. most Python projects have no build process at all) but if you're in the niche where it's useful, it's damned useful. Here what it does. Take a look at this Python code: import math def distance(x1, y1, x2, y2): return math.sqrt((y2-y1)**2+(x2-x1)**2) It looks simple, but it's doing a lot behind the scenes. It respects __sub__/__add__ metamethods, respects and promotes between floats/longs/ints and even Bignums, it has to do a __getattr__ on the `math` object to find the sqrt function in the first place, etc (and can't cache that because this is Python so you may have changed it in between calls). After every one of these calls, a Python exception is checked for, and many reference counts are incremented and decremented. Every now and then, the interpreter checks to see if it's time to release/reacquire the GIL and if any OS signals have come in. And it does all of this by compiling Python to bytecode and running that through its interpreter. Here's the same code in Cython: from libc.math cimport sqrt cdef double distance(double x1, double y1, double x2, double y2): return sqrt((y2-y1)**2+(x2-x1)**2) Cython takes that code and compiles it to this C code (a bit simplified): #include <math.h> double distance(double x1, double y1, double x2, double y2) { double a = y2-y1; double b = x2-x1; return sqrt(a*a+b*b) } So now instead of running through dozens of intermediate internal Python functions, you're just loading the numbers into your FPU and doing the computation there. Of course, you've given up all of your Python dynamicness. Now this won't work with Bignums, it can't be monkey patched, and now you have a compiling step to running your Python code. You can do this to your existing code one function at a time, so you don't have to give those things up for your entire project, just where it matters. And where performance matters, it's a godsend I ported a small number of pieces of reddit to Cython. Here are some simple examples: * https://github.com/reddit/reddit/blob/master/r2/r2/lib/db/_sorts.pyx https://github.com/reddit/reddit/blob/master/r2/r2/lib/db/_s... * https://github.com/reddit/reddit/blob/master/r2/r2/lib/utils/_utils.pyx https://github.com/reddit/reddit/blob/master/r2/r2/lib/utils... * https://github.com/reddit/reddit/blob/master/r2/r2/lib/mr_tools/_mr_tools.pyx https://github.com/reddit/reddit/blob/master/r2/r2/lib/mr_to... * https://github.com/reddit/reddit/blob/master/r2/r2/lib/wrapped.pyx https://github.com/reddit/reddit/blob/master/r2/r2/lib/wrapp... As the underscore naming implies, these are only selected performance or compute sensitive functions factored out of their normal non-underscore Python modules which generally wrap them. These functions were coming up disproportionately in profiling because they're called very often and/or do a lot of arithmetic, and afterwards they are entirely nonissues. Often something like a _sorts.pyx:_confidence call got over 25 times faster. I've also talked about the downsides in a thread here[1]. Go in knowing that stuff too, but after my excellent experience with Cython I'm a True Believer. [1] https://www.reddit.com/r/Python/comments/2xlya2/anyone_willing_to_share_experiences_of_cython/cp1ct6a?context=3 https://www.reddit.com/r/Python/comments/2xlya2/anyone_willi...
- mixmastamyk 11y agoDoes it support/is-compatible with the new python type hints?