3 ms·

You can't just put LLVM on it and expect it to go fast, and I can't put a turbo on a 2CV and expect to use it on the motorway now. It's _a lot_ of work to make

You can't just put LLVM on it and expect it to go fast, and I can't put a turbo on a 2CV and expect to use it on the motorway now. It's _a lot_ of work to make a language like python "superfast", and most of it is really the dynamic typing part. LLVM doesn't do that for you.

fransje26 4y ago
> and I can't put a turbo on a 2CV and expect to use it on the motorway now. Well, actually, I remember being passed on the autobahn by 2CV going well over 200 km/h. That was quite a shocker.
- stefncb 4y ago
  I'm quite sure the owner didn't get there by just adding a turbo :) I'd love to see it though.
  - fransje26 4y ago
    No doubt! As it came past in a blur, it was difficult to see what was left of the original car beyond the external shell..
winter_blue 4y ago
Option 1 is mandating mypy-style type annotations (with type inference where there’s no ambiguity).[1] Option 2 is using C++-style vtables, and putting each value (including primitives) inside an object, with each object inheriting from a common object superclass that has stub methods like __add__, __sub__, __mul__, __div__, __str__, __repr__, etc, etc. You do all the dynamic checks at runtime. With option 2, I’m sure vtable dispatch is still significantly faster than Python (while substantially slower than static typing). One could also combine approaches 1 and 2 make the static typing optional. [1] I realize implementation type inference isn’t trivial.
- stefncb 4y ago
  There are indeed lots of options. In this case, 1 is limiting and 2 is slow because of cases like integer operations. A very common way to make it "fast" is specialising at runtime based on usage, so it's kind of like option 1 but dynamic.
- whizzter 4y ago
  Re nr 2, Like stefncb's sibing comment mentions, numeric ops would be horrendeous due to excessive memory allocations (even if you did value like semantics driven you'd still be luggin an entire vtable-pointer along with your numbers and there is usually more value movement than operations in code). Type-inference depends on the language. Languages with prepared semantics do it just fine(ML family), it's just that languages like Python and JS didn't consider it from day-1 so we're always stuck with explicit subsets (or extensions like TS, Cython/Mypy) to gain back some order. (I did my thesis work on JS type inference). In practice, there is nothing that hinders the Python language to be as fast as JS (see PyPy or even faster with Cython annotations). The problem is that C-Python sees the simple FFI as an golden egg (for kind-of good reasons at this point) and this will always limit C-Pythons performance because it touches upon internals that are hard to change out without breaking the FFI contracts. So, even if one would go about and bolt LLVM onto C-Python we'd still be way behind V8/JSC/Spidermonkey, because 1: The value and FFI model still would hold it back 2: LLVM while awesome at static code, would still have a ton of issues with the dynamic nature of Python. There has been a couple of articles on the new JIT of Erlang that is based on a custom "simple" compiler that is working better than many more "advanced" variations has worked because it's more tailored to the dynamic nature of Erlang rather than try to make a regular JIT make something sane out of Erlang code. If you want to see improvements to Python there's 4 avenues.. 1: Nuitka , this is one(?) guy who's doing a lot of manual engineering work together with some JIT ideas, it probably has a performance ceiling but he seems serious about doing it whilst retaining compatibility. (No idea how it'll fare with the regular series C-Python optimizations). 2: PyPy, Jython and GraalPython. These all depart from mainline C-Python and does deprive you of many C/C++-bound libraries whilst retaining the language semantics. The PyPy team has suggested a new better FFI that I do hope gets adopted by C-Python as the way forward for new modules (and ports of older ones) that would allow PyPy and C-Python to share modules as a transition way. 3: Cython, here you have the original source (iirc?) of the MyPy like annotations and an actual compiler that will specialize and should be mostly compatible with C-Python. 4: Shedskin, this is probably the fastest "Python", but it's a specialized subset that's mendable to full type-inference.

fransje26 4y ago

> and I can't put a turbo on a 2CV and expect to use it on the motorway now. Well, actually, I remember being passed on the autobahn by 2CV going well over 200 km/h. That was quite a shocker.

stefncb 4y ago
I'm quite sure the owner didn't get there by just adding a turbo :) I'd love to see it though.
- fransje26 4y ago
  No doubt! As it came past in a blur, it was difficult to see what was left of the original car beyond the external shell..

winter_blue 4y ago

Option 1 is mandating mypy-style type annotations (with type inference where there’s no ambiguity).[1] Option 2 is using C++-style vtables, and putting each value (including primitives) inside an object, with each object inheriting from a common object superclass that has stub methods like __add__, __sub__, __mul__, __div__, __str__, __repr__, etc, etc. You do all the dynamic checks at runtime. With option 2, I’m sure vtable dispatch is still significantly faster than Python (while substantially slower than static typing). One could also combine approaches 1 and 2 make the static typing optional. [1] I realize implementation type inference isn’t trivial.

stefncb 4y ago
There are indeed lots of options. In this case, 1 is limiting and 2 is slow because of cases like integer operations. A very common way to make it "fast" is specialising at runtime based on usage, so it's kind of like option 1 but dynamic.
whizzter 4y ago
Re nr 2, Like stefncb's sibing comment mentions, numeric ops would be horrendeous due to excessive memory allocations (even if you did value like semantics driven you'd still be luggin an entire vtable-pointer along with your numbers and there is usually more value movement than operations in code). Type-inference depends on the language. Languages with prepared semantics do it just fine(ML family), it's just that languages like Python and JS didn't consider it from day-1 so we're always stuck with explicit subsets (or extensions like TS, Cython/Mypy) to gain back some order. (I did my thesis work on JS type inference). In practice, there is nothing that hinders the Python language to be as fast as JS (see PyPy or even faster with Cython annotations). The problem is that C-Python sees the simple FFI as an golden egg (for kind-of good reasons at this point) and this will always limit C-Pythons performance because it touches upon internals that are hard to change out without breaking the FFI contracts. So, even if one would go about and bolt LLVM onto C-Python we'd still be way behind V8/JSC/Spidermonkey, because 1: The value and FFI model still would hold it back 2: LLVM while awesome at static code, would still have a ton of issues with the dynamic nature of Python. There has been a couple of articles on the new JIT of Erlang that is based on a custom "simple" compiler that is working better than many more "advanced" variations has worked because it's more tailored to the dynamic nature of Erlang rather than try to make a regular JIT make something sane out of Erlang code. If you want to see improvements to Python there's 4 avenues.. 1: Nuitka , this is one(?) guy who's doing a lot of manual engineering work together with some JIT ideas, it probably has a performance ceiling but he seems serious about doing it whilst retaining compatibility. (No idea how it'll fare with the regular series C-Python optimizations). 2: PyPy, Jython and GraalPython. These all depart from mainline C-Python and does deprive you of many C/C++-bound libraries whilst retaining the language semantics. The PyPy team has suggested a new better FFI that I do hope gets adopted by C-Python as the way forward for new modules (and ports of older ones) that would allow PyPy and C-Python to share modules as a transition way. 3: Cython, here you have the original source (iirc?) of the MyPy like annotations and an actual compiler that will specialize and should be mostly compatible with C-Python. 4: Shedskin, this is probably the fastest "Python", but it's a specialized subset that's mendable to full type-inference.