4 ms·

I think this is interesting, but I'm wondering why there isn't a corresponding project for Python. It seems like people using Python could derive the exact sa

I think this is interesting, but I'm wondering why there isn't a corresponding project for Python. It seems like people using Python could derive the exact same benefits, so, why doesn't the Python version exist of this project?

karlding 5y ago
> I'm wondering why there isn't a corresponding project for Python. But there are various projects for Python. For example, there's Pyston [0] which was started by Dropbox. Or Cinder [1] which was started by Facebook. Or PyPy [2]. [0] https://github.com/pyston/pyston https://github.com/pyston/pyston [1] https://github.com/facebookincubator/cinder https://github.com/facebookincubator/cinder [2] https://www.pypy.org/ https://www.pypy.org/
- joelbluminator 5y ago
  Still, adoption of these solutions seems low (PyPy seems to be a JIT as well is it really a corresponding solution?). If it magically reduces memory by as much as 30%-50% why isn't adoption higher? In other words what are the drawbacks of doing this, there must be cons.
  - FooBarWidget 5y ago
    Author of Fullstaq Ruby here. From what I understand by talking to people, the reasons why Fullstaq Ruby doesn't have wider adoption are as follows: It's not a project officially blessed by Ruby core. Upstream Ruby is actually "good enough" for most. Most people don't feel the pain enough to try something which appears as unofficial. Perhaps their thinking are similar to yours: I'm hesitant because not everbody else is already using this. I want to make this an official project. But before I do, I want to make sure that the project is sufficiently ready: healthy, democratized, not reliant on myself: https://www.joyfulbikeshedding.com/blog/2020-05-15-why-fullstaq-ruby.html https://www.joyfulbikeshedding.com/blog/2020-05-15-why-fulls... I am also not marketing this as hard as I should. I lack free time. Another reason is because of the beta label. The only reason why the label is there is because I have a specific, ambitious vision on how "final" should look like (i.e. more features), not because it's unstable. Feedback from users regarding stability has been very good so far. I might be shooting myself in the foot marketing-wise by not removing the beta label earlier, but I'd rather do a good job than to hurry.
wokwokwok 5y ago
You understand this is mostly ruby using jemalloc and some other memory allocation tricks? Support for doing this in python was added in PEP-445 (https://www.python.org/dev/peps/pep-0445/ https://www.python.org/dev/peps/pep-0445/) and implementation details can be found here from 2013 (https://bugs.python.org/issue3329 https://bugs.python.org/issue3329). For more info see: https://docs.python.org/3/c-api/memory.html https://docs.python.org/3/c-api/memory.html You can see some commentary on using malloc-trim here too -> https://www.softwareatscale.dev/p/run-python-servers-more-efficiently https://www.softwareatscale.dev/p/run-python-servers-more-ef... ...long story short: basically, you can already, but it doesn't really make much difference for most workloads. Or, if your question is, why doesn't python get optimisation love and become fast like the javascript runtimes? dunno. I guess we just don't have any big FANG companies pouring money on that, because they don't have a use for a faster python. A faster python does not make numpy, tensorflow or pytorch run faster because those are not written in python. So I guess, there's a lack of value-for-effort in trying to 'fix' the problem of 'pure' python being slow.
- jsmeaton 5y ago
  FYI Microsoft is funding Guido and a small team to specifically work on performance see https://m.slashdot.org/story/385526 https://m.slashdot.org/story/385526
  - karlding 5y ago
    This [0] is the announcement from the PSF. The team also includes core developers Eric Snow and Mark Shannon (of the faster-cpython [1] proposal). [0] https://pyfound.blogspot.com/2021/05/the-2021-python-language-summit-making.html https://pyfound.blogspot.com/2021/05/the-2021-python-languag... [1] https://github.com/markshannon/faster-cpython https://github.com/markshannon/faster-cpython

karlding 5y ago

> I'm wondering why there isn't a corresponding project for Python. But there are various projects for Python. For example, there's Pyston [0] which was started by Dropbox. Or Cinder [1] which was started by Facebook. Or PyPy [2]. [0] https://github.com/pyston/pyston https://github.com/pyston/pyston [1] https://github.com/facebookincubator/cinder https://github.com/facebookincubator/cinder [2] https://www.pypy.org/ https://www.pypy.org/

joelbluminator 5y ago
Still, adoption of these solutions seems low (PyPy seems to be a JIT as well is it really a corresponding solution?). If it magically reduces memory by as much as 30%-50% why isn't adoption higher? In other words what are the drawbacks of doing this, there must be cons.
- FooBarWidget 5y ago
  Author of Fullstaq Ruby here. From what I understand by talking to people, the reasons why Fullstaq Ruby doesn't have wider adoption are as follows: It's not a project officially blessed by Ruby core. Upstream Ruby is actually "good enough" for most. Most people don't feel the pain enough to try something which appears as unofficial. Perhaps their thinking are similar to yours: I'm hesitant because not everbody else is already using this. I want to make this an official project. But before I do, I want to make sure that the project is sufficiently ready: healthy, democratized, not reliant on myself: https://www.joyfulbikeshedding.com/blog/2020-05-15-why-fullstaq-ruby.html https://www.joyfulbikeshedding.com/blog/2020-05-15-why-fulls... I am also not marketing this as hard as I should. I lack free time. Another reason is because of the beta label. The only reason why the label is there is because I have a specific, ambitious vision on how "final" should look like (i.e. more features), not because it's unstable. Feedback from users regarding stability has been very good so far. I might be shooting myself in the foot marketing-wise by not removing the beta label earlier, but I'd rather do a good job than to hurry.

wokwokwok 5y ago

You understand this is mostly ruby using jemalloc and some other memory allocation tricks? Support for doing this in python was added in PEP-445 (https://www.python.org/dev/peps/pep-0445/ https://www.python.org/dev/peps/pep-0445/) and implementation details can be found here from 2013 (https://bugs.python.org/issue3329 https://bugs.python.org/issue3329). For more info see: https://docs.python.org/3/c-api/memory.html https://docs.python.org/3/c-api/memory.html You can see some commentary on using malloc-trim here too -> https://www.softwareatscale.dev/p/run-python-servers-more-efficiently https://www.softwareatscale.dev/p/run-python-servers-more-ef... ...long story short: basically, you can already, but it doesn't really make much difference for most workloads. Or, if your question is, why doesn't python get optimisation love and become fast like the javascript runtimes? dunno. I guess we just don't have any big FANG companies pouring money on that, because they don't have a use for a faster python. A faster python does not make numpy, tensorflow or pytorch run faster because those are not written in python. So I guess, there's a lack of value-for-effort in trying to 'fix' the problem of 'pure' python being slow.

jsmeaton 5y ago
FYI Microsoft is funding Guido and a small team to specifically work on performance see https://m.slashdot.org/story/385526 https://m.slashdot.org/story/385526
- karlding 5y ago
  This [0] is the announcement from the PSF. The team also includes core developers Eric Snow and Mark Shannon (of the faster-cpython [1] proposal). [0] https://pyfound.blogspot.com/2021/05/the-2021-python-language-summit-making.html https://pyfound.blogspot.com/2021/05/the-2021-python-languag... [1] https://github.com/markshannon/faster-cpython https://github.com/markshannon/faster-cpython