4 ms·

Furthermore, I did my grad studies in compilers. I've thought about writing an optimizing JIT for Python. I really feel like CPython is needlessly slow, and it'

Furthermore, I did my grad studies in compilers. I've thought about writing an optimizing JIT for Python. I really feel like CPython is needlessly slow, and it's kind of embarassing, Many have tried and failed, Google and Dropbox to name a couple, and countless other attempts.

tachyonbeam 7y ago
Yes, and part of the reason they failed is the reason I pointed to: Python is a fast moving target, with an increasing number of features.
- wvenable 7y ago
  It's not the new features of Python that make it hard to optimize; it's the fundamental dynamic nature of the language that was there from day one. Syntactic sugar doesn't have an impact one way or the other on optimizing Python.
  - tachyonbeam 7y ago
    The new features aren't just syntactic, they're also new libraries that come standard with CPython, etc. If you want to implement a Python JIT that people will use, you have to match everything CPython supports. Furthermore, since the people behind CPython don't care about JIT, you also can't count on them not adding language features that will break optimizations present in your JIT. You can't count on these being just "syntactic sugar". Even if you could though, in order to keep up it means you have to use CPython's front-end, or constantly implement every syntactic tweak CPython does. Lastly, AFAIK, CPython's FFI API is actually more of a problem than the dynamic semantics of the language. You can expose Python objects directly to C code. That makes it very hard for a JIT to represent said Python objects in an efficient way internally.
yjftsjthsd-h 7y ago
It lags a bit in releases, but I understood pypy to be essentially successful?
- chc 7y ago
  Yes, PyPy is fantastic for long-running processes that aren't primarily wrappers around C code. In my experience, the speedups you see in its benchmarks translate to the real world very well.

tachyonbeam 7y ago

Yes, and part of the reason they failed is the reason I pointed to: Python is a fast moving target, with an increasing number of features.

wvenable 7y ago
It's not the new features of Python that make it hard to optimize; it's the fundamental dynamic nature of the language that was there from day one. Syntactic sugar doesn't have an impact one way or the other on optimizing Python.
- tachyonbeam 7y ago
  The new features aren't just syntactic, they're also new libraries that come standard with CPython, etc. If you want to implement a Python JIT that people will use, you have to match everything CPython supports. Furthermore, since the people behind CPython don't care about JIT, you also can't count on them not adding language features that will break optimizations present in your JIT. You can't count on these being just "syntactic sugar". Even if you could though, in order to keep up it means you have to use CPython's front-end, or constantly implement every syntactic tweak CPython does. Lastly, AFAIK, CPython's FFI API is actually more of a problem than the dynamic semantics of the language. You can expose Python objects directly to C code. That makes it very hard for a JIT to represent said Python objects in an efficient way internally.

yjftsjthsd-h 7y ago

It lags a bit in releases, but I understood pypy to be essentially successful?

chc 7y ago
Yes, PyPy is fantastic for long-running processes that aren't primarily wrappers around C code. In my experience, the speedups you see in its benchmarks translate to the real world very well.