3 ms·

The problem turns out to be that it's maintaining the C-API compatibility which is the main thing which makes it hard to make Python fast, not the other stuff -

The problem turns out to be that it's maintaining the C-API compatibility which is the main thing which makes it hard to make Python fast, not the other stuff -- Javascript has most of the nasty things Python does, and it's plenty fast on browsers. However, maintaining C-API compatibility means you need to set up lots of data structures exactly how the C API requires, and maintaining and updating those ends up losing you lots of your benefits of JITing. You could, hypothetically, introduce an entirely new API, which allowed for faster dynamic recompiling, but then you'd need to get every package anyone cares about to switch to that.

rustybolt 4y ago
Sorry for the ignorance, but what do you mean by C-API here? Normally I'd say you mean the interface you use when you call native machine code from Python, but I don't see how this would slow things down.
- radicalbyte 4y ago
  The interface to the C language. It is what makes Python fast - you write the code which needs to be fast in optimised C and call into it with Python. The Python code is then basically just the glue.
- pdpi 4y ago
  That native code still needs to be able to interact with your Python objects somehow. You can't change the API around PyObject without forcing all C libraries to make changes on their side, and that API forces you to expose things a certain way.
kaba0 4y ago
That’s why I found GraalVM’s approach intriguing. They provide a high level language API where you can simply write a language interpreter, and it will be able to JIT/AOT compile it down to fast machine code. But the most interesting aspect is that the IR they convert languages to is basically universal, so something like LLVM bitcode can also use the exact same representations. So you can interpret (and later AOT compile as well) LLVM bitcode and python, and this approach will allow cross-language optimizations as well, which were not available at all before. But feel free to add a bit of JS/Java, etc to your code as well!
vlovich123 4y ago
I really try hard to understand this argument and I must be missing something and must be super stupid. Don’t languages like JavaScript have this and yet they can still do JIT and the base runtime is still in C++? Java itself has an official way to invoke C programs from Java applications and still has a JIT. And Java also has AOT compilers. Sure. Crossing that FFI boundary is going to be expensive. But there’s lots of techniques to mitigate it or even in the limit eliminate it. if I recall correctly you can JIT a fast call that knows how to invoke the FFI directly without the extra indirection layer. Basically a fancy runtime LTO. I think a huge part of it is CPython’s interest in keeping the core codebase as simple as possible which seems to be the overriding reason for why the global lock still hasn’t been removed (which iirc even Ruby pulled off at some point). Also the reason there’s no JIT afaict and why Pypy got started to prove it is possible to JIT (and frequently sees substantial gains vs cpython). The problem they’ve had is that CPython is a moving target and it’s hard to keep a parallel runtime up to date on a shoestring amount of funding. That’s why you see alternate approaches like numba (JIT’ed Python) which are less of a departure and Cinder (better budget). To me this seems like a CPython project actively hostile to JIT than C data structures meaning you lose some benefit to FFI overhead. Performance is a virtuous cycle too - when there’s enthusiasm about a language you get more and more people paid to make your language fast. For a while companies tried. Google gave up. Facebook only has it as a fork with a public plea for the maintainers of CPython to mainline literally anything. The CPython maintainers feel like the biggest obstacle. No?
- pjmlp 4y ago
  The difference is that the other languages FFI don't expose internals like CPython does. For example, JNI only exposes handles and you need to convert an handle to a pointer, so the runtime knows for the time being that handle is special and being used by native code. When it is only an opaque handle, lots of optimizations can happen and the native code won't see them.
  - vlovich123 4y ago
    Doesn’t PyPy accomplish it via CPyExt? It sounds like Cinder, Instagram’s version is CPython+JIT (among other things). I haven’t looked at the details so maybe it’s not a sufficient speed up and that’s why all these parallel efforts haven’t been merged? The part I’m missing is how what you said makes it intractable when we have counter examples within and without the language. Sure. Maybe some optimizations aren’t possible. But that’s a world of difference from little to no benefit and impossible. Don’t get me wrong. I’m not passing a value judgement on the maintainers. But the reasons don’t feel technical to me.

rustybolt 4y ago

Sorry for the ignorance, but what do you mean by C-API here? Normally I'd say you mean the interface you use when you call native machine code from Python, but I don't see how this would slow things down.

radicalbyte 4y ago
The interface to the C language. It is what makes Python fast - you write the code which needs to be fast in optimised C and call into it with Python. The Python code is then basically just the glue.
pdpi 4y ago
That native code still needs to be able to interact with your Python objects somehow. You can't change the API around PyObject without forcing all C libraries to make changes on their side, and that API forces you to expose things a certain way.

kaba0 4y ago

That’s why I found GraalVM’s approach intriguing. They provide a high level language API where you can simply write a language interpreter, and it will be able to JIT/AOT compile it down to fast machine code. But the most interesting aspect is that the IR they convert languages to is basically universal, so something like LLVM bitcode can also use the exact same representations. So you can interpret (and later AOT compile as well) LLVM bitcode and python, and this approach will allow cross-language optimizations as well, which were not available at all before. But feel free to add a bit of JS/Java, etc to your code as well!

vlovich123 4y ago

I really try hard to understand this argument and I must be missing something and must be super stupid. Don’t languages like JavaScript have this and yet they can still do JIT and the base runtime is still in C++? Java itself has an official way to invoke C programs from Java applications and still has a JIT. And Java also has AOT compilers. Sure. Crossing that FFI boundary is going to be expensive. But there’s lots of techniques to mitigate it or even in the limit eliminate it. if I recall correctly you can JIT a fast call that knows how to invoke the FFI directly without the extra indirection layer. Basically a fancy runtime LTO. I think a huge part of it is CPython’s interest in keeping the core codebase as simple as possible which seems to be the overriding reason for why the global lock still hasn’t been removed (which iirc even Ruby pulled off at some point). Also the reason there’s no JIT afaict and why Pypy got started to prove it is possible to JIT (and frequently sees substantial gains vs cpython). The problem they’ve had is that CPython is a moving target and it’s hard to keep a parallel runtime up to date on a shoestring amount of funding. That’s why you see alternate approaches like numba (JIT’ed Python) which are less of a departure and Cinder (better budget). To me this seems like a CPython project actively hostile to JIT than C data structures meaning you lose some benefit to FFI overhead. Performance is a virtuous cycle too - when there’s enthusiasm about a language you get more and more people paid to make your language fast. For a while companies tried. Google gave up. Facebook only has it as a fork with a public plea for the maintainers of CPython to mainline literally anything. The CPython maintainers feel like the biggest obstacle. No?

pjmlp 4y ago
The difference is that the other languages FFI don't expose internals like CPython does. For example, JNI only exposes handles and you need to convert an handle to a pointer, so the runtime knows for the time being that handle is special and being used by native code. When it is only an opaque handle, lots of optimizations can happen and the native code won't see them.
- vlovich123 4y ago
  Doesn’t PyPy accomplish it via CPyExt? It sounds like Cinder, Instagram’s version is CPython+JIT (among other things). I haven’t looked at the details so maybe it’s not a sufficient speed up and that’s why all these parallel efforts haven’t been merged? The part I’m missing is how what you said makes it intractable when we have counter examples within and without the language. Sure. Maybe some optimizations aren’t possible. But that’s a world of difference from little to no benefit and impossible. Don’t get me wrong. I’m not passing a value judgement on the maintainers. But the reasons don’t feel technical to me.