5 ms·

But as Numba has shown, it's achievable to write a CPython-to-LLVM-IR compiler for subsets of the language, then add NumPy-awareness, and add multiple targets l

by p4wnc6 11y ago

But as Numba has shown, it's achievable to write a CPython-to-LLVM-IR compiler for subsets of the language, then add NumPy-awareness, and add multiple targets like GPUs, and achieve better-than-PyPy speed ups jitting pure Python (CPython) code. I think it's an interesting question as to whether PyPy "is the future" or rather something more like Numba, or even a whole ecosystem of specialized variants of Numba that may be tweaked or tuned to take advantage of different hardware optimizations, work better for different domain problems (e.g. image processing vs. network programming). In a lot of cases where PyPy improves over some standard CPython code, it's exactly the sort of code you really wouldn't care about optimizing in the context of a larger application. In any sort of scientific application, the pure CPython parts of it are usually small and insignificant compared to what's written in NumPy/SciPy/pandas and science libraries written on top of those, and Numba can be seen as the first "specializing compiler" for this sort of application -- with a lot of room for people to write other kinds of specializing compilers that compile different genres of Python code for domain-specific purposes, the way that Numba compiles generally scientific Python for numerical computing purposes.

duaneb 11y ago
> But as Numba has shown, it's achievable to write a CPython-to-LLVM-IR compiler for subsets of the language, then add NumPy-awareness, and add multiple targets like GPUs, and achieve better-than-PyPy speed ups jitting pure Python (CPython) code. Most python code I write would get 0 speedup from that. I think numeric optimization is a very different target than compiling and optimizing general purpose python code. I don't see either strategies "winning" in the near future.
Veedrac 11y ago
> But as Numba has shown, it's achievable to write a CPython-to-LLVM-IR compiler for subsets of the language, then add NumPy-awareness, and add multiple targets like GPUs, and achieve better-than-PyPy speed ups jitting pure Python (CPython) code. Other than "to-LLVM" and "targets like GPUs", jitpy does exactly that for PyPy-in-CPython. http://jitpy.readthedocs.org/en/latest/ http://jitpy.readthedocs.org/en/latest/ It's not quite as fast for numeric code (though it's close and conformant) but it's also not a language subset nor does it degrade to CPython performance on dynamic parts of the language. It should be a terrific option if you're able to section out discrete workloads that CPython's struggling with, even in non-numeric code. Few people know of it though.
- p4wnc6 11y ago
  Thanks for the link to JitPy. I will definitely read more about it. Just from the first parts of the docs, though, it looks very limited compared with Numba. In Numba, there is a distinction between 'python mode' and 'nopython mode' -- meaning that even at the final LLVM IR emission, if Numba is forced to punt on type inference (e.g. because an untyped Python list object is an argument or something), Numba can still fall back directly to the CPython API, and in general even this has speed benefits as a lot of intermediate calls or intermediate variables can be lowered or typed. So even though JitPy won't "degrade to CPython performance", it's slightly moot since you can only handle a limited set of types in JitPy. I guess one area where JitPy should win in theory is if there is a dynamic Python object inside of the function, like a dynamic list. Then JitPy is OK since it's not part of the signature, and when it can't optimize due to indeterminate type inside the function, the fallback will be PyPy instead of CPython, and hence should be faster. But Numba offers a lot on the array computing side, which is what it is specialized for. For example, from the Numba docs < http://numba.pydata.org/numba-doc/0.21.0/developer/architecture.html#stage-5-rewrite-typed-ir http://numba.pydata.org/numba-doc/0.21.0/developer/architect... >: > Numba implements a user-extensible rewriting pass that reads and possibly rewrites Numba IR. This pass’s purpose is to perform any high-level optimizations that still require, or could at least benefit from, Numba IR type information. > One example of a problem domain that isn’t as easily optimized once lowered is the domain of multidimensional array operations. When Numba lowers an array operation, Numba treats the operation like a full ufunc kernel. During lowering a single array operation, Numba generates an inline broadcasting loop that creates a new result array. Then Numba generates an application loop that applies the operator over the array inputs. Recognizing and rewriting these loops once they are lowered into LLVM is hard, if not impossible. > An example pair of optimizations in the domain of array operators is loop fusion and shortcut deforestation. When the optimizer recognizes that the output of one array operator is being fed into another array operator, and only to that array operator, it can fuse the two loops into a single loop. The optimizer can further eliminate the temporary array allocated for the initial operation by directly feeding the result of the first operation into the second, skipping the store and load to the intermediate array. This elimination is known as shortcut deforestation. Numba currently uses the rewrite pass to implement these array optimizations. For more information, please consult the “Case study: Array Expressions” subsection, later in this document. One reason why I see the Numba route as more effective than the PyPy route is that these kinds of highly-specific optimizations seem likely to occur all over the place, and to be very related to the genre of computing you are doing. One person may not want Numba to perform these optimizations, because loop fusion isn't important to their codebase which doesn't do a lot of array computing. Instead, they might want some kind of domain-specific optimization that assists with a kernel bypass for some low-latency algorithm. This could all be exposed in Numba, but as separate domain-specific sub-modules or via user-defined IR optimization passes or something. Or it could exist as totally separate JIT compiler projects, of which JitPy/PyPy's JIT compiler and Numba's JIT compiler would each just be genre-specific examples. I tend to see this as the more likely future, and that apart from cases where someone is really just taking a bunch of pure Python code and running it with PyPy interpreter instead of CPython interpreter (which I forecast would be a rare case), the difference between PyPy and CPython won't matter nearly as much as the difference between domain-specific JIT compilers.

by p4wnc6 11y ago

duaneb 11y ago

> But as Numba has shown, it's achievable to write a CPython-to-LLVM-IR compiler for subsets of the language, then add NumPy-awareness, and add multiple targets like GPUs, and achieve better-than-PyPy speed ups jitting pure Python (CPython) code. Most python code I write would get 0 speedup from that. I think numeric optimization is a very different target than compiling and optimizing general purpose python code. I don't see either strategies "winning" in the near future.

Veedrac 11y ago

> But as Numba has shown, it's achievable to write a CPython-to-LLVM-IR compiler for subsets of the language, then add NumPy-awareness, and add multiple targets like GPUs, and achieve better-than-PyPy speed ups jitting pure Python (CPython) code. Other than "to-LLVM" and "targets like GPUs", jitpy does exactly that for PyPy-in-CPython. http://jitpy.readthedocs.org/en/latest/ http://jitpy.readthedocs.org/en/latest/ It's not quite as fast for numeric code (though it's close and conformant) but it's also not a language subset nor does it degrade to CPython performance on dynamic parts of the language. It should be a terrific option if you're able to section out discrete workloads that CPython's struggling with, even in non-numeric code. Few people know of it though.

p4wnc6 11y ago
Thanks for the link to JitPy. I will definitely read more about it. Just from the first parts of the docs, though, it looks very limited compared with Numba. In Numba, there is a distinction between 'python mode' and 'nopython mode' -- meaning that even at the final LLVM IR emission, if Numba is forced to punt on type inference (e.g. because an untyped Python list object is an argument or something), Numba can still fall back directly to the CPython API, and in general even this has speed benefits as a lot of intermediate calls or intermediate variables can be lowered or typed. So even though JitPy won't "degrade to CPython performance", it's slightly moot since you can only handle a limited set of types in JitPy. I guess one area where JitPy should win in theory is if there is a dynamic Python object inside of the function, like a dynamic list. Then JitPy is OK since it's not part of the signature, and when it can't optimize due to indeterminate type inside the function, the fallback will be PyPy instead of CPython, and hence should be faster. But Numba offers a lot on the array computing side, which is what it is specialized for. For example, from the Numba docs < http://numba.pydata.org/numba-doc/0.21.0/developer/architecture.html#stage-5-rewrite-typed-ir http://numba.pydata.org/numba-doc/0.21.0/developer/architect... >: > Numba implements a user-extensible rewriting pass that reads and possibly rewrites Numba IR. This pass’s purpose is to perform any high-level optimizations that still require, or could at least benefit from, Numba IR type information. > One example of a problem domain that isn’t as easily optimized once lowered is the domain of multidimensional array operations. When Numba lowers an array operation, Numba treats the operation like a full ufunc kernel. During lowering a single array operation, Numba generates an inline broadcasting loop that creates a new result array. Then Numba generates an application loop that applies the operator over the array inputs. Recognizing and rewriting these loops once they are lowered into LLVM is hard, if not impossible. > An example pair of optimizations in the domain of array operators is loop fusion and shortcut deforestation. When the optimizer recognizes that the output of one array operator is being fed into another array operator, and only to that array operator, it can fuse the two loops into a single loop. The optimizer can further eliminate the temporary array allocated for the initial operation by directly feeding the result of the first operation into the second, skipping the store and load to the intermediate array. This elimination is known as shortcut deforestation. Numba currently uses the rewrite pass to implement these array optimizations. For more information, please consult the “Case study: Array Expressions” subsection, later in this document. One reason why I see the Numba route as more effective than the PyPy route is that these kinds of highly-specific optimizations seem likely to occur all over the place, and to be very related to the genre of computing you are doing. One person may not want Numba to perform these optimizations, because loop fusion isn't important to their codebase which doesn't do a lot of array computing. Instead, they might want some kind of domain-specific optimization that assists with a kernel bypass for some low-latency algorithm. This could all be exposed in Numba, but as separate domain-specific sub-modules or via user-defined IR optimization passes or something. Or it could exist as totally separate JIT compiler projects, of which JitPy/PyPy's JIT compiler and Numba's JIT compiler would each just be genre-specific examples. I tend to see this as the more likely future, and that apart from cases where someone is really just taking a bunch of pure Python code and running it with PyPy interpreter instead of CPython interpreter (which I forecast would be a rare case), the difference between PyPy and CPython won't matter nearly as much as the difference between domain-specific JIT compilers.