6 ms·

An open-source Python implementation using JIT techniques

83 points by daGrevis 13y ago

webmaven 13y ago
Given that this is a new project, why focus on python 2.7, rather than 3.4?
- duggan 13y ago
  Not wishing to put words in the mouths of the creators, but there are a few reasons why one might target 2.7 rather than 3.x: 1. 2.7 is a stable target. There will be no further features added to the 2.x series. 2. 2.7 is a known target. Other experiments like PyPy have already tried many paths to improve 2.7, so what doesn't work is reasonably well known. 3. 2.7 is a widely deployed target. This is less important from an "adoption" point of view, more so that there is a lot of software written that targets 2.x, meaning there's more software to test with. 4. Finally, I suspect much of Dropbox's codebase is 2.7[1], and since this is an effort to improve things for Dropbox, it makes sense to spend the initial efforts there :) [1] checking the version of Python bundled with the Dropbox client confirms that it, at least, is 2.7 /Applications/Dropbox.app/Contents/MacOS/python --version Python 2.7.3
  - sanxiyn 13y ago
    1 is a big item. It is not fun playing catchup. CPython actually suspended any "language change" from 2009 to 2011 so that alternative implementations can catch up.
    - stormbrew 13y ago
      On the other hand, starting behind just means you have even more catching up to do. This happened in my personal VM project with ruby. I targeted ruby 1.8.7 when all these things held true for it, but by the time I got to it running most of the rubyspec language specs, 1.8.7 was basically obsolete.
- pygy_ 13y ago
  Their code base is written in Python 2.7. In the previous thread, they said that contributions to implement Python 3 compatibility were welcome.
mercurial 13y ago
It's interesting, but it looks like it has a loooong way to go before reaching feature parity with PyPy, which has itself a number of challenges to overcome before becoming mainstream. I would be interested to read the rationale for starting work on Pyston vs using/improving PyPy.
- nitraat 13y ago
  From the site: https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/ https://tech.dropbox.com/2014/04/introducing-pyston-an-upcom... Why a new implementation There are already a number of Python implementations using JIT techniques, often in sophisticated ways. PyPy has achieved impressive performance with its tracing JIT; Jython and IronPython are both built on top of mature VMs with extensive JIT support. So why do we think it’s worth starting a new implementation? In short, it’s because we think the most promising techniques are incompatible with existing implementations. For instance, the JavaScript world has switched from tracing JITs to method-at-a-time JITs, due to the compelling performance benefits. Whether or not the same performance advantage holds for Python is an open question, but since the two approaches are fundamentally incompatible, the only way to start answering the question is to build a new method-at-a-time JIT. Another point of differentiation is the planned use of a conservative garbage collector to support extension modules efficiently. Again, we won’t know until later whether this is a better approach or not, but it’s a decision that’s integral enough to a JIT that it is difficult to test in an existing implementation. The downside of starting from scratch is, unsurprisingly, that creating a new language implementation is an enormous task. Luckily, tools are starting to come out that can help with this process; in particular, Pyston is built on top of LLVM, which lets us achieve top-tier code generation quality without having to deal with the details ourselves. Nonetheless, a new Python implementation is a huge undertaking, and Pyston will not be ready for use soon.
  - mercurial 13y ago
    Thanks for that. Well, I won't say no to open-source R&D :)
- hcarvalhoalves 13y ago
ksec 13y ago
Google V8 is certainly a brilliant piece of engineering, but by no means Method JIT ( V8 is based on ) has won or the best. Because the King, LuaJIT is based on Tracing JIT. I have to wonder why Python as a communities have always have the interested to substantially speed up Python. While the Ruby Camp has always been about if you want more speed, do it in C. ( Best Tools for the Job manta? ) Although I hope Ruby MRI will one day get its own Google V8 / LuaJIT treatment.
- mkesper 13y ago
  Because people good at writing Python/Ruby/whatever dynamic language might not be the best suited for writing C code (or because seen from a Python programmer's point of view, C definitely sucks).
- benjamincburns 13y ago
  To paint broad strokes, I think it's more accurate to say that the Ruby maintainers have generally prioritized user experience (where the user here is the programmer) over performance. I think the Python maintainers have maybe prioritized performance a bit higher than their Ruby counterparts, but there's definitely still a higher emphasis on ease of use. Given the goals of both languages and the need to make pragmatic choices, I think their priorities are well set. That said, I think a Java-fast Python or Ruby is an incredibly laudable goal, and I'm happy every time I see people working on it.
  - TheCondor 13y ago
    To use some other broad strokes.. Are there actually any large performance sensitive ruby projects? Python has quite a following in scientific computing and there are some people trying to do some large performance matters projects in python. I can also tell you that the idea of simply using c for the slow parts is good but it's not too difficult to build a largish project in a dynamic language where that's very difficult. A high speed kernel with slow control plane can still be slow. Pyston seems like it has some interesting goals and it might achieve them. There are some substantial ones they haven't addressed yet, cough Gil cough. If they are good, it'll just be a faster python, whereas pypy is very good and if the community embraces what they are doing, it's a new kid of thing.
jnowlan 13y ago
Would this allow python to be run in the browser (by going through Emscripten and asm.js)? Is that a goal that makes sense?
- ris 13y ago
  Not really any more than any other python implementation, no.
sanxiyn 13y ago
Previously here: https://news.ycombinator.com/item?id=7524712 https://news.ycombinator.com/item?id=7524712

83 points by daGrevis 13y ago

webmaven 13y ago

Given that this is a new project, why focus on python 2.7, rather than 3.4?

duggan 13y ago
Not wishing to put words in the mouths of the creators, but there are a few reasons why one might target 2.7 rather than 3.x: 1. 2.7 is a stable target. There will be no further features added to the 2.x series. 2. 2.7 is a known target. Other experiments like PyPy have already tried many paths to improve 2.7, so what doesn't work is reasonably well known. 3. 2.7 is a widely deployed target. This is less important from an "adoption" point of view, more so that there is a lot of software written that targets 2.x, meaning there's more software to test with. 4. Finally, I suspect much of Dropbox's codebase is 2.7[1], and since this is an effort to improve things for Dropbox, it makes sense to spend the initial efforts there :) [1] checking the version of Python bundled with the Dropbox client confirms that it, at least, is 2.7 /Applications/Dropbox.app/Contents/MacOS/python --version Python 2.7.3
- sanxiyn 13y ago
  1 is a big item. It is not fun playing catchup. CPython actually suspended any "language change" from 2009 to 2011 so that alternative implementations can catch up.
  - stormbrew 13y ago
    On the other hand, starting behind just means you have even more catching up to do. This happened in my personal VM project with ruby. I targeted ruby 1.8.7 when all these things held true for it, but by the time I got to it running most of the rubyspec language specs, 1.8.7 was basically obsolete.
pygy_ 13y ago
Their code base is written in Python 2.7. In the previous thread, they said that contributions to implement Python 3 compatibility were welcome.

mercurial 13y ago

It's interesting, but it looks like it has a loooong way to go before reaching feature parity with PyPy, which has itself a number of challenges to overcome before becoming mainstream. I would be interested to read the rationale for starting work on Pyston vs using/improving PyPy.

nitraat 13y ago
From the site: https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/ https://tech.dropbox.com/2014/04/introducing-pyston-an-upcom... Why a new implementation There are already a number of Python implementations using JIT techniques, often in sophisticated ways. PyPy has achieved impressive performance with its tracing JIT; Jython and IronPython are both built on top of mature VMs with extensive JIT support. So why do we think it’s worth starting a new implementation? In short, it’s because we think the most promising techniques are incompatible with existing implementations. For instance, the JavaScript world has switched from tracing JITs to method-at-a-time JITs, due to the compelling performance benefits. Whether or not the same performance advantage holds for Python is an open question, but since the two approaches are fundamentally incompatible, the only way to start answering the question is to build a new method-at-a-time JIT. Another point of differentiation is the planned use of a conservative garbage collector to support extension modules efficiently. Again, we won’t know until later whether this is a better approach or not, but it’s a decision that’s integral enough to a JIT that it is difficult to test in an existing implementation. The downside of starting from scratch is, unsurprisingly, that creating a new language implementation is an enormous task. Luckily, tools are starting to come out that can help with this process; in particular, Pyston is built on top of LLVM, which lets us achieve top-tier code generation quality without having to deal with the details ourselves. Nonetheless, a new Python implementation is a huge undertaking, and Pyston will not be ready for use soon.
- mercurial 13y ago
  Thanks for that. Well, I won't say no to open-source R&D :)
hcarvalhoalves 13y ago

ksec 13y ago

Google V8 is certainly a brilliant piece of engineering, but by no means Method JIT ( V8 is based on ) has won or the best. Because the King, LuaJIT is based on Tracing JIT. I have to wonder why Python as a communities have always have the interested to substantially speed up Python. While the Ruby Camp has always been about if you want more speed, do it in C. ( Best Tools for the Job manta? ) Although I hope Ruby MRI will one day get its own Google V8 / LuaJIT treatment.

mkesper 13y ago
Because people good at writing Python/Ruby/whatever dynamic language might not be the best suited for writing C code (or because seen from a Python programmer's point of view, C definitely sucks).
benjamincburns 13y ago
To paint broad strokes, I think it's more accurate to say that the Ruby maintainers have generally prioritized user experience (where the user here is the programmer) over performance. I think the Python maintainers have maybe prioritized performance a bit higher than their Ruby counterparts, but there's definitely still a higher emphasis on ease of use. Given the goals of both languages and the need to make pragmatic choices, I think their priorities are well set. That said, I think a Java-fast Python or Ruby is an incredibly laudable goal, and I'm happy every time I see people working on it.
- TheCondor 13y ago
  To use some other broad strokes.. Are there actually any large performance sensitive ruby projects? Python has quite a following in scientific computing and there are some people trying to do some large performance matters projects in python. I can also tell you that the idea of simply using c for the slow parts is good but it's not too difficult to build a largish project in a dynamic language where that's very difficult. A high speed kernel with slow control plane can still be slow. Pyston seems like it has some interesting goals and it might achieve them. There are some substantial ones they haven't addressed yet, cough Gil cough. If they are good, it'll just be a faster python, whereas pypy is very good and if the community embraces what they are doing, it's a new kid of thing.

jnowlan 13y ago

Would this allow python to be run in the browser (by going through Emscripten and asm.js)? Is that a goal that makes sense?

ris 13y ago
Not really any more than any other python implementation, no.

sanxiyn 13y ago

Previously here: https://news.ycombinator.com/item?id=7524712 https://news.ycombinator.com/item?id=7524712