4 ms·

I’ve not had the same experience. Simply adding types and omitting CPython types (which results in simpler “basic C” loops and so on) gets 100x speed ups in mos

by mlthoughts2018 6y ago

I’ve not had the same experience. Simply adding types and omitting CPython types (which results in simpler “basic C” loops and so on) gets 100x speed ups in most situations, and Cython has an annotation mode that highlights CPython bottlenecks to make this very easy and interactive to spot. With numba it can be even more dramatic. Zero code changes, solely type annotation in the decorator, and you can often get 1000x speedup over pure Python, because numba can optimize away many of the extra CPython data model features that tight loop code may occasionally not need to use.

dagw 6y ago
Do you have any code you can share showing this speed up? Because I have written a lot of Cython over past 5 years, and unless my python code is monumentally stupid (and in that case I can get an order of magnitude or two speedup by just writing sane python), I have never seen anything close to 100x speedup in real world code. Even getting a 5x speed up over a baseline python/numpy implementation requires a fair amount of refactoring to make the code more cython friendly. Same with Numba, despite its promises, I've never seen anything close to 100x speedups in the real world, and I've even seen a fair few cases where numba resulted in slower code compared to 'pure' python/numpy (although admittedly that was 2-3 years ago) Sure I could probably design some pathological code where you might see 100x speed up by just adding type annotations, but that would probably require starting with some really inefficient python code.
- mlthoughts2018 6y ago
  I do have code I can show, but I do not want to reveal my GitHub username attached to my Hacker News account. Thankfully you don’t have to rely on me - tons of name brand Python scientific tools use Cython for speedups. Pandas, for example, uses Cython heavily. Here is an example tutorial that may help you, https://pandas.pydata.org/pandas-docs/stable/user_guide/enhancingperf.html https://pandas.pydata.org/pandas-docs/stable/user_guide/enha...

dagw 6y ago

Do you have any code you can share showing this speed up? Because I have written a lot of Cython over past 5 years, and unless my python code is monumentally stupid (and in that case I can get an order of magnitude or two speedup by just writing sane python), I have never seen anything close to 100x speedup in real world code. Even getting a 5x speed up over a baseline python/numpy implementation requires a fair amount of refactoring to make the code more cython friendly. Same with Numba, despite its promises, I've never seen anything close to 100x speedups in the real world, and I've even seen a fair few cases where numba resulted in slower code compared to 'pure' python/numpy (although admittedly that was 2-3 years ago) Sure I could probably design some pathological code where you might see 100x speed up by just adding type annotations, but that would probably require starting with some really inefficient python code.

mlthoughts2018 6y ago
I do have code I can show, but I do not want to reveal my GitHub username attached to my Hacker News account. Thankfully you don’t have to rely on me - tons of name brand Python scientific tools use Cython for speedups. Pandas, for example, uses Cython heavily. Here is an example tutorial that may help you, https://pandas.pydata.org/pandas-docs/stable/user_guide/enhancingperf.html https://pandas.pydata.org/pandas-docs/stable/user_guide/enha...