4 ms·

[disclaimer: I have contributed to a couple different Python runtimes and employed to work on Cinder] > Can CPython become faster? Yes. There are several peop

[disclaimer: I have contributed to a couple different Python runtimes and employed to work on Cinder] > Can CPython become faster? Yes. There are several people (including a team by Guido and Mark Shannon) working on making CPython faster in-tree. There are others, including the folks on Cinder and Pyjion, who are working on extensions to CPython out of tree. > Could the devs take a look [...] It's no secret how they made their runtimes fast. It's all -- more or less -- implementations of the original Smalltalk-80 paper and all that fell out of it. > Can the Python language get extended so that you can define the bottlencks of your code in another language? C extensions are already available in Python, and have been for quite some time. In fact, I would argue they are a little bit part of the problem: there are so many C extensions that runtime optimization efforts don't have as much effect as people would like; the slow stuff is already in C. > Can a new runtime take over and everybody settle on it? There are a lot of "competing" Python runtimes that serve different use-cases. Some use PyPy. Some use Jython. Some use Cinder. etc. Though everyone seems to ask "why not just use PyPy?", people forget to consider that everyone has different constraints that PyPy does not necessarily satisfy.

andi999 5y ago
Could you elaborate on the original smalltalk paper and its relation to js jit?
- KMag 5y ago
  McCarthy is generally credited with the first JIT for his early LISP work, but SmallTalk and SmallTalk-inspired languages were involved in a lot of the pioneering work related to modern JITs. My understanding is that McCarthy's JIT generated machine code as source code was first loaded, not when it was first invoked. As far as I know, the earliest VMs to wait until a function's first invocation to JIT the function were SmallTalk VMs. The Self language by David Ungar and others was an expansion of some of the SmallTalk ideas. If I remember correctly, Self is where polymorphic inline caching originated. Sun acqui-hired Urs Holzle (who did his PhD under Ungar), David Griswald, Gilad Bracha, and Lars Bak for their StrongTalk (statically typed dialect of SmallTalk) hybrid bytecode interpreter/JIT, which was the basis for the HotSpot JVM. Lars Bak later created the original V8 JavaScript JIT for Chrome. When I was on the Google web search indexing team, I did a bit of work running a pre-release V8 in the indexing pipeline (to analyze text and formatting changes made to pages by JavaScript). If I remember correctly, at that time V8 wasn't a hybrid interpreter/JIT, but did quick-and-dirty generation of x86 machine code when a function was first invoked, and leaned heavily on polymorphic inline caching. My understanding is that early V8 was rather similar to the Self VM. I believe ideas like tracing JITs originated a bit further from the SmallTalk ecosystem, but a lot of modern JIT ideas and techniques came from early SmallTalk and related research.
  - pjmlp 5y ago
    Since you mention McCarthy and JITs, there is another set of poor folks that seem forgoten on JIT research. Only recently did I learn that Dartmouth BASIC did use a JIT approach as well. It was only when dialects started to crop up on 8-bit computers that the trend for BASIC to be an interpreted language came to be.
  - rjsw 5y ago
    I don't remember any Lisp work being considered an example of a JIT, the Deutsch/Schiffman paper was felt to be a big breakthrough when it was published.
    - KMag 5y ago
      John Aycock credits McCarthy's Lisp work in "A brief history of just-in-time"[0] Though, as I mentioned, I believe the native code was generated when the module was loaded. These days, we generally consider JIT compilation to be compilation that happens at first invocation (or after, in the case of hybrid interpretation/JIT), so it doesn't precisely fit the modern notion of JIT compilation. [0] https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.97.3985 https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.97....
- pjmlp 5y ago
  Not the OP, but these papers are well known anyway. Smalltalk was one of the first JIT implementations, "Efficient implementation of the smalltalk-80 system" https://dl.acm.org/doi/10.1145/800017.800542 https://dl.acm.org/doi/10.1145/800017.800542 Then they followed up with StrongTalk, https://strongtalk.org/ https://strongtalk.org/ And SELF, which JavaScript OOP is modeled on, https://selflanguage.org https://selflanguage.org You will recognize some of the V8 names on those papers. Additionally, SELF JIT was repurposed as Hotspot by Java 1.2. Although Java looks like C++ from syntax point of view, it actually is closer to Smalltalk and Objective-C in runtime semantics, thus it could profit a lot from that JIT research as well, even though it was originally targeted to dynamic languages.
  - tekknolagi 5y ago
    Thanks for chiming in!
masklinn 5y ago
> C extensions are already available in Python, and have been for quite some time. And C extensions are an API, you don’t have to write them in C e.g. cython generates “C extensions”. Furthermore there are other micro-hooks like numba ehich integrate rather easy and can provide great bang for your buck in restricted numerical contexts.
TekMol 5y ago
I had to look up what Cinder is. Yay, a faster fork of CPython! For the love of God, please get those improvements to CPython. The world needs it! > It's no secret how they made their runtimes fast. So why is nobody doing it? The ecosystem that would benefit from it must be gigantic. Hell, I am a solo indie developer and I would happily pay $500 to see Python make a jump to match PHP's performance! If we extrapolate this to the whole Python ecosystem, the value of a fast CPython must be Billions. > C extensions are already available in Python What I meant is inline. So inside of a py file, you can rewrite the one function which you identified as the bottleneck. In C or another language.
- saagarjha 5y ago
  It requires a lot of money and the C API is restrictive when trying to perform aggressive optimization.
- BiteCode_dev 5y ago
  > So why is nobody doing it? Mostly, resources. Giants spent their energy speeding things strategically: ・ Google wanted Chrome to win, so they spend their effort on JS. ・ Facebook was a PHP shop, so they worked on make it fast. ・ MS had to promote .net, so they optimized c#. Now you also have to remember that: ・ it's a hard problem ・ python is not a performance oriented language, so it was not a priority ・ when data science became popular, c-extensions were usually enough to get the perfs you wanted ・ once you have a working python code reaching perfs limit, you could start a rewrite in a fast compiled language (Google did it with go, dropbox with rust, etc). But this could take years before you need it. Youtube ran fine on PHP then Python for almost a decade. Python can take you very, very far. In fact, I'd say half the time somebody says python is too slow for them, it's because they are doing something wrong. Case in point, I have a half-million users/day streaming website running on Python. It's been running for 10 years, and needs only one dev to maintain it. ・ the PSF had no money (https://pyfound.blogspot.com/2012/01/psf-grants-over-37000-to-python.htmlhttps://pyfound.blogspot.com/2012/01/psf-grants-over-37000-to-python.html https://pyfound.blogspot.com/2012/01/psf-grants-over-37000-t...), and most work on python was from people on their free time. Very few paid devs on Python for decades, compared to other languages ・ there are actually a lot of improvements on tooling for perfs on python. Multiprocessing have fantastic pools and shared memory. We have asyncio (not to mention uvloop). c-extensions release the GIL, so you can use threads for parallelism with numpy. Most people are just not aware of it. ・ Python is very old, 4 years older than java. It has been design way before multi-core laptop were a thing. And so the GIL is everywhere. ・ python is in a very unique position: it's the second best language for anything. Meaning it's been used by data scientist, web devs, geographers, sysadmin, pen testers, os maintainers and biologists alike. They all have very different tooling, dev envs and workloads. Speed doesn't mean the same things for all of them. E.G: a slow starting JIT that speeds up a web server would destroy the use case for cli tools. Mercurial has not computational speed problems, but import time is killing it. But lazy imports would make debugging and new comers learning curve harder. What if they speed up things for them, but slow them down for you? ・ 2->3 made the entire community change averse. And to get speed, you need to break the C api somehow. Hence the Handle Python project. So the previous attempts to speed up Python were by either interns, or people doing it out of passion/curiosity. A lot failed (unladen swallow, pyston1, etc). Some were successful but only for some dimension of speed and/or doing some compromises (pypy, stackless...). Some are still WIP (Pydgin, pyston 2..). The solutions that succeeded the most are actually side stepping the problem (cython, numpy, numba, dask) but usually became popular although some very good projects still deserve more light (nuitka, mypyc...) It's only now that Python is becoming such a huge deal that MS hired Guido to work on general computational speed for cPython. Now that there is attention, money and dedication on the problem, you can already things moving in the bug tracker (coroutine unwrapping, c stack skipping calls, HandlePython, etc). It's going to take time. Millions have already been invested in V8 to make the JS slug go steroid, so catch up is not for tomorrow. > What I meant is inline. So inside of a py file, you can rewrite the one function which you identified as the bottleneck. In C or another language. You can already do that with cython. And there are alternative for rust, nim, go... That's also why python speed is usually not a priority: you can rewrite your hot spot in another language and keep most of your python code. Now, I'm glad python speed is now something the right people work on. But there were very good reason it hasn't happen until now. The way you say it, it feels like "wow, nobody has though of that before!". That's not what happened.

andi999 5y ago

Could you elaborate on the original smalltalk paper and its relation to js jit?

KMag 5y ago
McCarthy is generally credited with the first JIT for his early LISP work, but SmallTalk and SmallTalk-inspired languages were involved in a lot of the pioneering work related to modern JITs. My understanding is that McCarthy's JIT generated machine code as source code was first loaded, not when it was first invoked. As far as I know, the earliest VMs to wait until a function's first invocation to JIT the function were SmallTalk VMs. The Self language by David Ungar and others was an expansion of some of the SmallTalk ideas. If I remember correctly, Self is where polymorphic inline caching originated. Sun acqui-hired Urs Holzle (who did his PhD under Ungar), David Griswald, Gilad Bracha, and Lars Bak for their StrongTalk (statically typed dialect of SmallTalk) hybrid bytecode interpreter/JIT, which was the basis for the HotSpot JVM. Lars Bak later created the original V8 JavaScript JIT for Chrome. When I was on the Google web search indexing team, I did a bit of work running a pre-release V8 in the indexing pipeline (to analyze text and formatting changes made to pages by JavaScript). If I remember correctly, at that time V8 wasn't a hybrid interpreter/JIT, but did quick-and-dirty generation of x86 machine code when a function was first invoked, and leaned heavily on polymorphic inline caching. My understanding is that early V8 was rather similar to the Self VM. I believe ideas like tracing JITs originated a bit further from the SmallTalk ecosystem, but a lot of modern JIT ideas and techniques came from early SmallTalk and related research.
- pjmlp 5y ago
  Since you mention McCarthy and JITs, there is another set of poor folks that seem forgoten on JIT research. Only recently did I learn that Dartmouth BASIC did use a JIT approach as well. It was only when dialects started to crop up on 8-bit computers that the trend for BASIC to be an interpreted language came to be.
- rjsw 5y ago
  I don't remember any Lisp work being considered an example of a JIT, the Deutsch/Schiffman paper was felt to be a big breakthrough when it was published.
  - KMag 5y ago
    John Aycock credits McCarthy's Lisp work in "A brief history of just-in-time"[0] Though, as I mentioned, I believe the native code was generated when the module was loaded. These days, we generally consider JIT compilation to be compilation that happens at first invocation (or after, in the case of hybrid interpretation/JIT), so it doesn't precisely fit the modern notion of JIT compilation. [0] https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.97.3985 https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.97....
pjmlp 5y ago
Not the OP, but these papers are well known anyway. Smalltalk was one of the first JIT implementations, "Efficient implementation of the smalltalk-80 system" https://dl.acm.org/doi/10.1145/800017.800542 https://dl.acm.org/doi/10.1145/800017.800542 Then they followed up with StrongTalk, https://strongtalk.org/ https://strongtalk.org/ And SELF, which JavaScript OOP is modeled on, https://selflanguage.org https://selflanguage.org You will recognize some of the V8 names on those papers. Additionally, SELF JIT was repurposed as Hotspot by Java 1.2. Although Java looks like C++ from syntax point of view, it actually is closer to Smalltalk and Objective-C in runtime semantics, thus it could profit a lot from that JIT research as well, even though it was originally targeted to dynamic languages.
- tekknolagi 5y ago
  Thanks for chiming in!

masklinn 5y ago

> C extensions are already available in Python, and have been for quite some time. And C extensions are an API, you don’t have to write them in C e.g. cython generates “C extensions”. Furthermore there are other micro-hooks like numba ehich integrate rather easy and can provide great bang for your buck in restricted numerical contexts.

TekMol 5y ago

I had to look up what Cinder is. Yay, a faster fork of CPython! For the love of God, please get those improvements to CPython. The world needs it! > It's no secret how they made their runtimes fast. So why is nobody doing it? The ecosystem that would benefit from it must be gigantic. Hell, I am a solo indie developer and I would happily pay $500 to see Python make a jump to match PHP's performance! If we extrapolate this to the whole Python ecosystem, the value of a fast CPython must be Billions. > C extensions are already available in Python What I meant is inline. So inside of a py file, you can rewrite the one function which you identified as the bottleneck. In C or another language.

saagarjha 5y ago
It requires a lot of money and the C API is restrictive when trying to perform aggressive optimization.
BiteCode_dev 5y ago
> So why is nobody doing it? Mostly, resources. Giants spent their energy speeding things strategically: ・ Google wanted Chrome to win, so they spend their effort on JS. ・ Facebook was a PHP shop, so they worked on make it fast. ・ MS had to promote .net, so they optimized c#. Now you also have to remember that: ・ it's a hard problem ・ python is not a performance oriented language, so it was not a priority ・ when data science became popular, c-extensions were usually enough to get the perfs you wanted ・ once you have a working python code reaching perfs limit, you could start a rewrite in a fast compiled language (Google did it with go, dropbox with rust, etc). But this could take years before you need it. Youtube ran fine on PHP then Python for almost a decade. Python can take you very, very far. In fact, I'd say half the time somebody says python is too slow for them, it's because they are doing something wrong. Case in point, I have a half-million users/day streaming website running on Python. It's been running for 10 years, and needs only one dev to maintain it. ・ the PSF had no money (https://pyfound.blogspot.com/2012/01/psf-grants-over-37000-to-python.htmlhttps://pyfound.blogspot.com/2012/01/psf-grants-over-37000-to-python.html https://pyfound.blogspot.com/2012/01/psf-grants-over-37000-t...), and most work on python was from people on their free time. Very few paid devs on Python for decades, compared to other languages ・ there are actually a lot of improvements on tooling for perfs on python. Multiprocessing have fantastic pools and shared memory. We have asyncio (not to mention uvloop). c-extensions release the GIL, so you can use threads for parallelism with numpy. Most people are just not aware of it. ・ Python is very old, 4 years older than java. It has been design way before multi-core laptop were a thing. And so the GIL is everywhere. ・ python is in a very unique position: it's the second best language for anything. Meaning it's been used by data scientist, web devs, geographers, sysadmin, pen testers, os maintainers and biologists alike. They all have very different tooling, dev envs and workloads. Speed doesn't mean the same things for all of them. E.G: a slow starting JIT that speeds up a web server would destroy the use case for cli tools. Mercurial has not computational speed problems, but import time is killing it. But lazy imports would make debugging and new comers learning curve harder. What if they speed up things for them, but slow them down for you? ・ 2->3 made the entire community change averse. And to get speed, you need to break the C api somehow. Hence the Handle Python project. So the previous attempts to speed up Python were by either interns, or people doing it out of passion/curiosity. A lot failed (unladen swallow, pyston1, etc). Some were successful but only for some dimension of speed and/or doing some compromises (pypy, stackless...). Some are still WIP (Pydgin, pyston 2..). The solutions that succeeded the most are actually side stepping the problem (cython, numpy, numba, dask) but usually became popular although some very good projects still deserve more light (nuitka, mypyc...) It's only now that Python is becoming such a huge deal that MS hired Guido to work on general computational speed for cPython. Now that there is attention, money and dedication on the problem, you can already things moving in the bug tracker (coroutine unwrapping, c stack skipping calls, HandlePython, etc). It's going to take time. Millions have already been invested in V8 to make the JS slug go steroid, so catch up is not for tomorrow. > What I meant is inline. So inside of a py file, you can rewrite the one function which you identified as the bottleneck. In C or another language. You can already do that with cython. And there are alternative for rust, nim, go... That's also why python speed is usually not a priority: you can rewrite your hot spot in another language and keep most of your python code. Now, I'm glad python speed is now something the right people work on. But there were very good reason it hasn't happen until now. The way you say it, it feels like "wow, nobody has though of that before!". That's not what happened.