4 ms·

though, Julia is faster than Python. Does anyone mind explaining why Python can't have a JIT ?

by dzonga 7y ago

throwlaplace 7y ago
http://www.pypy.org/ http://www.pypy.org/
setr 7y ago
Pypy exists, but as I recall, cpython never made use of a JIT simply because they wanted to keep the compiler intentionally simpler. However as a result, the language isn't well designed for a JIT, and pypy has run into several headaches/blockers. Not sure if there's anything fundamentally blocking usage, or simply lack of manpower
eigenspace 7y ago
As others have mentioned, Python does have JIT compilers. The problem is that havign a JIT doesn't solve the problem. PyPy is often a factor of 10 behind julia performance and projects like Numba, PyTorch (the PyTorch people had to build their own Python JIT compiler yikes!), etc. will always have a more restricted scope than a project like Julia because Python's very semantics make many optimizations impossible. Here's a great talk from the author of the famous Flask library in Python: https://www.youtube.com/watch?v=qCGofLIzX6g https://www.youtube.com/watch?v=qCGofLIzX6g where he discusses the fundamental problems with Python's semantics. If you fix these problems, you will end up changing Python so fundamentaly that you'll really have a new language. Generic CPython 3._ code will certainly not be compatible with it.
- dzonga 7y ago
  so in this case, once say the Julia ecosystem grows then migrate to Julia. or wait for optimizations to be done, e.g have pandas, numpy etc handle multi-core processors etc ?
  - ChrisRackauckas 7y ago
    There's quite fundamental optimizations that will be missing if separately compiled pieces cannot optimize together. These barriers disallow many things. Additionally, anything that relies on higher order functions provided by the user, such as optimization or differential equations, will run into issues because those functions cannot be optimized by some super good package developer, but instead have to come from the user. This blog post highlights the massive performance advantages Julia can have when you start integrating optimized packages and allow for optimizing across these barriers in Julia, and also the pros and cons of allowing these kinds of cross-function optimizations. https://www.stochasticlifestyle.com/why-numba-and-cython-are-not-substitutes-for-julia/ https://www.stochasticlifestyle.com/why-numba-and-cython-are...
  - simonbyrne 7y ago
    > or wait for optimizations to be done, e.g have pandas, numpy etc handle multi-core processors etc ? All those exist already. Indeed, other than DataFrames.jl (the pandas equivalent) they are part of the language itself.
deleted 7y ago
[deleted]

though, Julia is faster than Python. Does anyone mind explaining why Python can't have a JIT ?

by dzonga 7y ago

though, Julia is faster than Python. Does anyone mind explaining why Python can't have a JIT ?

throwlaplace 7y ago

http://www.pypy.org/ http://www.pypy.org/

setr 7y ago

Pypy exists, but as I recall, cpython never made use of a JIT simply because they wanted to keep the compiler intentionally simpler. However as a result, the language isn't well designed for a JIT, and pypy has run into several headaches/blockers. Not sure if there's anything fundamentally blocking usage, or simply lack of manpower

eigenspace 7y ago

As others have mentioned, Python does have JIT compilers. The problem is that havign a JIT doesn't solve the problem. PyPy is often a factor of 10 behind julia performance and projects like Numba, PyTorch (the PyTorch people had to build their own Python JIT compiler yikes!), etc. will always have a more restricted scope than a project like Julia because Python's very semantics make many optimizations impossible. Here's a great talk from the author of the famous Flask library in Python: https://www.youtube.com/watch?v=qCGofLIzX6g https://www.youtube.com/watch?v=qCGofLIzX6g where he discusses the fundamental problems with Python's semantics. If you fix these problems, you will end up changing Python so fundamentaly that you'll really have a new language. Generic CPython 3._ code will certainly not be compatible with it.

dzonga 7y ago
so in this case, once say the Julia ecosystem grows then migrate to Julia. or wait for optimizations to be done, e.g have pandas, numpy etc handle multi-core processors etc ?
- ChrisRackauckas 7y ago
  There's quite fundamental optimizations that will be missing if separately compiled pieces cannot optimize together. These barriers disallow many things. Additionally, anything that relies on higher order functions provided by the user, such as optimization or differential equations, will run into issues because those functions cannot be optimized by some super good package developer, but instead have to come from the user. This blog post highlights the massive performance advantages Julia can have when you start integrating optimized packages and allow for optimizing across these barriers in Julia, and also the pros and cons of allowing these kinds of cross-function optimizations. https://www.stochasticlifestyle.com/why-numba-and-cython-are-not-substitutes-for-julia/ https://www.stochasticlifestyle.com/why-numba-and-cython-are...
- simonbyrne 7y ago
  > or wait for optimizations to be done, e.g have pandas, numpy etc handle multi-core processors etc ? All those exist already. Indeed, other than DataFrames.jl (the pandas equivalent) they are part of the language itself.

deleted 7y ago

[deleted]