4 ms·

Ok, so why is Python so slow (comparatively)? I had understood that must of the speed-critical tasks in python were done by wrappers of lower-level C code. Exam

by oneloop 10y ago

Ok, so why is Python so slow (comparatively)? I had understood that must of the speed-critical tasks in python were done by wrappers of lower-level C code. Example: numpy. Is this not correct?

TheHydroImpulse 10y ago
The reason it's slow is because python interprets it's input. It essentially introduces major indirection for instructions to be executed. So for '1 + 1', you can't just output some fast x86 instruction and inline it. You'll usually need to jump to where the python vm will do 'a + b' in C. You're having to do a bunch of redundant work at a higher level to emulate the CPU (sorta). That's why you have JIT compilers that will read '1 + 1' and compile the appropriate machine code, store that in an executable memory region and jump to it. First time might be slow, but after that it's pretty fast. Because you need to jump to the dynamically generated code, you usually compile whole functions at a time. This topic is super complicated but also super interested but I tried to simplify it a bit here.
insertnickname 10y ago
You can indeed do that, and there are other ways to speed up Python (PyPy, Cython, or even just writing more efficient Python code). However, the problem is that you now have to write C. A lot of the slowness in Python probably comes from the many memory allocations all over the place. Go gives you a lot of control over allocations even though it is a garbage collected language. These slides posted by another commenter has some good points about why Python is so slow: https://speakerdeck.com/alex/why-python-ruby-and-javascript-are-slow https://speakerdeck.com/alex/why-python-ruby-and-javascript-... Interestingly, there was actually a talk at PyCon US 2016 about using Go's http server in Python via C. https://www.youtube.com/watch?v=CkDwb5koRTc https://www.youtube.com/watch?v=CkDwb5koRTc
SEJeff 10y ago
FWIW: There is a new set of async / await bits in CPython 3.5 that are allowing some really interesting new performance profiles. http://magic.io/blog/uvloop-blazing-fast-python-networking/ http://magic.io/blog/uvloop-blazing-fast-python-networking/ This is almost exactly on par with golang's net/http for a simple echo server, yet it is written in python.

by oneloop 10y ago

Ok, so why is Python so slow (comparatively)? I had understood that must of the speed-critical tasks in python were done by wrappers of lower-level C code. Example: numpy. Is this not correct?

TheHydroImpulse 10y ago

The reason it's slow is because python interprets it's input. It essentially introduces major indirection for instructions to be executed. So for '1 + 1', you can't just output some fast x86 instruction and inline it. You'll usually need to jump to where the python vm will do 'a + b' in C. You're having to do a bunch of redundant work at a higher level to emulate the CPU (sorta). That's why you have JIT compilers that will read '1 + 1' and compile the appropriate machine code, store that in an executable memory region and jump to it. First time might be slow, but after that it's pretty fast. Because you need to jump to the dynamically generated code, you usually compile whole functions at a time. This topic is super complicated but also super interested but I tried to simplify it a bit here.

insertnickname 10y ago

You can indeed do that, and there are other ways to speed up Python (PyPy, Cython, or even just writing more efficient Python code). However, the problem is that you now have to write C. A lot of the slowness in Python probably comes from the many memory allocations all over the place. Go gives you a lot of control over allocations even though it is a garbage collected language. These slides posted by another commenter has some good points about why Python is so slow: https://speakerdeck.com/alex/why-python-ruby-and-javascript-are-slow https://speakerdeck.com/alex/why-python-ruby-and-javascript-... Interestingly, there was actually a talk at PyCon US 2016 about using Go's http server in Python via C. https://www.youtube.com/watch?v=CkDwb5koRTc https://www.youtube.com/watch?v=CkDwb5koRTc

SEJeff 10y ago

FWIW: There is a new set of async / await bits in CPython 3.5 that are allowing some really interesting new performance profiles. http://magic.io/blog/uvloop-blazing-fast-python-networking/ http://magic.io/blog/uvloop-blazing-fast-python-networking/ This is almost exactly on par with golang's net/http for a simple echo server, yet it is written in python.