6 ms·

I'm not a big user of Python, but it seems to me that any language that requires you to embed a different language for performance is inherently broken. To reg

by azylman 14y ago

I'm not a big user of Python, but it seems to me that any language that requires you to embed a different language for performance is inherently broken. To regular pythonistas: Is this a common issue with Python, or only in the author's contrived scenario of summing a ridiculous amount of numbers?

benhoyt 14y ago
It's a contrived scenario. I've used Python a ton and never needed to break out to C, certainly not for summing numbers. Granted, I use Python mostly for web development, but I do care about performance. In case of number crunching, I've heard numpy is very good and fast at number crunching using arrays, and the crunchy parts of numpy are written in C. All that said, why is a language "inherently broken" if it requires you to use another language for tight-loop performance such as this? Game and graphics developer writing in C break out to assembly for things like this -- is C also "inherently broken"?
dagw 14y ago
By your definition all dynamic scripting style languages (lua, ruby, perl, etc.) are inherently broken. Very few languages out there can reasonably expect to get C like speed for numerical work without dropping down to C. All this shows is how relatively easy it is to do when you need it. One of pythons really big and popular uses is as a powerful language for tying together high performance low level C/C++/Fortran libraries for serious numerical work (see libraries like numpy, numexpr, scipy and so on), so as such calling C libraries from python is about as standard a scenario as you can get.
achompas 14y ago
I'd actually call it a feature, and not a bug or flaw, in Python. You can prototype something in Python (or another language with native hooks) very, very quickly. If that thing works, you can roll in ctypes or cython and get your code running as quickly as C. Thus, you get quick development + numerous paths to native-level performance. That's much better than taking longer to develop a library or model in C/C++, only to find they don't satisfy your needs.
IanOzsvald 14y ago
As dagw mentions - treat it as a feature. I taught High Performance Python at PyCon, I've published all the src and a 55 page PDF covering a range of techniques to optimise a Mandelbrot example (including cython, shedskin, numpy, multiprocessing, pycuda): http://ianozsvald.com/2012/03/18/high-performance-python-1-from-pycon-2012-slides-video-src/ http://ianozsvald.com/2012/03/18/high-performance-python-1-f... With some languages this is painful, with Python you can easily profile (cProfile, line_profiler) and then work on easy or complex optimisations (both within Python and using external tools) - great flexibility for when you really need it.
vitriolix 14y ago
So C/C++ are also inherently broken because game engines break out to raw ASM at their mist critical points? Or Java because it exposes native code via JNI? This is actually totally standard behavior in pretty much every language.
njharman 14y ago
Obviously broken, because all possible uses of a programming language require the absolute utmost speed. Every other consideration of language design, usability, and ridiculous fluff such as "productivity" must kowtow before the god of performance.
azylman 14y ago
Wow, I don't think I've ever had such a visceral reaction to anything I've posted on HN before. By people who clearly didn't take the time out of their day to even read what I posted! Basically, I don't use python very much and wanted to know if python had performance issues that required you to drop down to C code often, or if that was just for a scenario such as this. I think that any language that REQUIRES you to drop down to a lower level for performance (not one that SUPPORTS it, but one where it's REQUIRED) is inherently broken. I would think that would be a pretty noncontroversial statement - if every python script you wrote had to have embedded C code because otherwise it was too slow, that would be pretty broken. That's why I wanted to know how common this was.
- bryanh 14y ago
  I think the reaction was because you put Python in a broadly negative camp by infering ctype extensions are required for any sort of performant code. Its actually relatively uncommon to use ctypes and most developers who use Python know this. Python is a great general purpose language and is used as such. No one would be silly enough to solely rely on it in highly performance sensitive environments (IE: 3d renderers or high frequency trading). Just like no one would be silly enough to write an entire CRUD web app in C. The point is, Python isn't broken and claiming that it may be because you read a short article on a Python feature got you the negative reaction.
  - azylman 14y ago
    I never inferred that ctypes were required - in fact, I explicitly did the opposite. I asked if it was required for performant code. See the part of my post where I said "To regular pythonistas: Is this a common issue with Python, or only in the author's contrived scenario of summing a ridiculous amount of numbers?" I also never claimed that python was broken.
    - bryanh 14y ago
      > I'm not a big user of Python, but it seems to me that any language that requires you to embed a different language for performance is inherently broken. It seems like you imply both of those things, though perhaps not as explicit as "Python is broken because ctypes are required", which you did not say.
hythloday 14y ago
I think it's exactly the opposite--any language that doesn't allow you to break out to a different language for performance is broken.
jiaaro 14y ago
Hi there, I'm the author of the post (my username matches the domain of the post =D ) The purpose is just to show an example of how to drop to C. I chose a contrived example so the reader could expend the minimum possible number of brain cycles (so to speak) trying to understand that part of the code.

by azylman 14y ago

benhoyt 14y ago

It's a contrived scenario. I've used Python a ton and never needed to break out to C, certainly not for summing numbers. Granted, I use Python mostly for web development, but I do care about performance. In case of number crunching, I've heard numpy is very good and fast at number crunching using arrays, and the crunchy parts of numpy are written in C. All that said, why is a language "inherently broken" if it requires you to use another language for tight-loop performance such as this? Game and graphics developer writing in C break out to assembly for things like this -- is C also "inherently broken"?

dagw 14y ago

By your definition all dynamic scripting style languages (lua, ruby, perl, etc.) are inherently broken. Very few languages out there can reasonably expect to get C like speed for numerical work without dropping down to C. All this shows is how relatively easy it is to do when you need it. One of pythons really big and popular uses is as a powerful language for tying together high performance low level C/C++/Fortran libraries for serious numerical work (see libraries like numpy, numexpr, scipy and so on), so as such calling C libraries from python is about as standard a scenario as you can get.

achompas 14y ago

I'd actually call it a feature, and not a bug or flaw, in Python. You can prototype something in Python (or another language with native hooks) very, very quickly. If that thing works, you can roll in ctypes or cython and get your code running as quickly as C. Thus, you get quick development + numerous paths to native-level performance. That's much better than taking longer to develop a library or model in C/C++, only to find they don't satisfy your needs.

IanOzsvald 14y ago

As dagw mentions - treat it as a feature. I taught High Performance Python at PyCon, I've published all the src and a 55 page PDF covering a range of techniques to optimise a Mandelbrot example (including cython, shedskin, numpy, multiprocessing, pycuda): http://ianozsvald.com/2012/03/18/high-performance-python-1-from-pycon-2012-slides-video-src/ http://ianozsvald.com/2012/03/18/high-performance-python-1-f... With some languages this is painful, with Python you can easily profile (cProfile, line_profiler) and then work on easy or complex optimisations (both within Python and using external tools) - great flexibility for when you really need it.

vitriolix 14y ago

So C/C++ are also inherently broken because game engines break out to raw ASM at their mist critical points? Or Java because it exposes native code via JNI? This is actually totally standard behavior in pretty much every language.

njharman 14y ago

Obviously broken, because all possible uses of a programming language require the absolute utmost speed. Every other consideration of language design, usability, and ridiculous fluff such as "productivity" must kowtow before the god of performance.

azylman 14y ago

Wow, I don't think I've ever had such a visceral reaction to anything I've posted on HN before. By people who clearly didn't take the time out of their day to even read what I posted! Basically, I don't use python very much and wanted to know if python had performance issues that required you to drop down to C code often, or if that was just for a scenario such as this. I think that any language that REQUIRES you to drop down to a lower level for performance (not one that SUPPORTS it, but one where it's REQUIRED) is inherently broken. I would think that would be a pretty noncontroversial statement - if every python script you wrote had to have embedded C code because otherwise it was too slow, that would be pretty broken. That's why I wanted to know how common this was.

bryanh 14y ago
I think the reaction was because you put Python in a broadly negative camp by infering ctype extensions are required for any sort of performant code. Its actually relatively uncommon to use ctypes and most developers who use Python know this. Python is a great general purpose language and is used as such. No one would be silly enough to solely rely on it in highly performance sensitive environments (IE: 3d renderers or high frequency trading). Just like no one would be silly enough to write an entire CRUD web app in C. The point is, Python isn't broken and claiming that it may be because you read a short article on a Python feature got you the negative reaction.
- azylman 14y ago
  I never inferred that ctypes were required - in fact, I explicitly did the opposite. I asked if it was required for performant code. See the part of my post where I said "To regular pythonistas: Is this a common issue with Python, or only in the author's contrived scenario of summing a ridiculous amount of numbers?" I also never claimed that python was broken.
  - bryanh 14y ago
    > I'm not a big user of Python, but it seems to me that any language that requires you to embed a different language for performance is inherently broken. It seems like you imply both of those things, though perhaps not as explicit as "Python is broken because ctypes are required", which you did not say.

hythloday 14y ago

I think it's exactly the opposite--any language that doesn't allow you to break out to a different language for performance is broken.

jiaaro 14y ago

Hi there, I'm the author of the post (my username matches the domain of the post =D ) The purpose is just to show an example of how to drop to C. I chose a contrived example so the reader could expend the minimum possible number of brain cycles (so to speak) trying to understand that part of the code.