4 ms·

Well, here's a less trivial example: https://github.com/syllog1sm/redshift/blob/develop/redshift/parser.pyx https://github.com/syllog1sm/redshift/blob/develop/

Well, here's a less trivial example: https://github.com/syllog1sm/redshift/blob/develop/redshift/parser.pyx https://github.com/syllog1sm/redshift/blob/develop/redshift/... The task/algorithm is roughly (but not precisely) the one described here: http://honnibal.wordpress.com/2013/12/18/a-simple-fast-algorithm-for-natural-language-dependency-parsing/ http://honnibal.wordpress.com/2013/12/18/a-simple-fast-algor... If you just scroll through parser.pyx, you might get the flavour of why this is better than straight C. The "core", or "skeleton" of the code is all C-typed, but I "fall-back" to Python for work-a-day stuff like reading and writing config files, collecting the set of labels, etc. There are also places in the algorithmic code where I use Python data structures. In Parser._count_feats, I use a dictionary of dictionaries. This would be quite annoying in C, and efficiency is quite irrelevant here. I think using Cython in this way has given me a competitive edge in my research. This library has by far the best published results on speech parsing[1], and its speed/accuracy trade-off on written text is as good as any other libraries out there. [1] http://www.transacl.org/wp-content/uploads/2014/04/46.pdf http://www.transacl.org/wp-content/uploads/2014/04/46.pdf

krick 12y ago
Oh, thanks, that gives some insights indeed. > I think using Cython in this way has given me a competitive edge in my research. May I ask, have you ever considered a possibility using something like OCaml? As far as I heard it's quite convenient to use ML-family languages for writing parsers and that OCaml generates pretty well-performing code while being more like "Python-expressive" than "C-expressive". > This library has by far the best published results on speech parsing Ah, yes, I remember it from seeing a while ago, when you just released it. Never looked in the code, though. Didn't recognize it was you, until this post, however. =)
Beltiras 12y ago
Which profiling tool do you use to target your optimizations?
- andreasvc 12y ago
  I use cProfile; for the Cython code you need to set profile=True as a compilation directive.

krick 12y ago

Oh, thanks, that gives some insights indeed. > I think using Cython in this way has given me a competitive edge in my research. May I ask, have you ever considered a possibility using something like OCaml? As far as I heard it's quite convenient to use ML-family languages for writing parsers and that OCaml generates pretty well-performing code while being more like "Python-expressive" than "C-expressive". > This library has by far the best published results on speech parsing Ah, yes, I remember it from seeing a while ago, when you just released it. Never looked in the code, though. Didn't recognize it was you, until this post, however. =)

Beltiras 12y ago

Which profiling tool do you use to target your optimizations?

andreasvc 12y ago
I use cProfile; for the Cython code you need to set profile=True as a compilation directive.