4 ms·

> We estimate a total cost of $50k... Just looking at it from a financial perspective, having a great Python interpreter that doesn't have a GIL seems like a n

> We estimate a total cost of $50k... Just looking at it from a financial perspective, having a great Python interpreter that doesn't have a GIL seems like a no brainer for $50,000, and it creates another reason why people should take a look at PyPy. Side note: if you haven't looked at PyPy, check it out, along with RPython https://rpython.readthedocs.io/en/latest/ https://rpython.readthedocs.io/en/latest/

andruby 9y ago
How can they estimate this? What about all the libraries that might not be compatible with the solution PyPy comes up with? This feels like a number that might in the end blow up to 10x the original estimate.
- joaodlf 9y ago
  This reminds me how the sales/marketing teams in my company typically sell new features: "Not having this feature costs us 50k a month!" It might as well not be the case here, I just found it funny, 50k is our little magic number.
- rguillebert 9y ago
  It's not the PyPy developers' job to make every Python library threadsafe, people writing libraries will have to make their code threadsafe, like in every other language.
  - taeric 9y ago
    There is a clear difference here, though. Making a change that could lead to poorly written libraries now being broken is clearly the fault of the change. Userspace for these libraries is defined by how it is, not how it was intended. (And really, was it intended to be dangerous in this way?)
    - masklinn 9y ago
      > There is a clear difference here, though. Making a change that could lead to poorly written libraries now being broken is clearly the fault of the change. No, these libraries are already semantically broken in the same way e.g. libraries which didn't properly close their files and assumed the CPython refcounting GC would wipe there asses were broken. They're already broken under two non-GIL'd implementations.
    - koolba 9y ago
      No the fault in that situation is a user blindly upgrading PyPy without testing the totality of their software package and its dependencies. Expecting bad code to magically work forever is unrealistic and hinders progress.
    - rguillebert 9y ago
      Then just use the version of PyPy with a GIL?
    - cjhanks 9y ago
      I agree. Even developers who are well aware of how to write thread-safe code probably don't even bother with mutex locking in Python. That code isn't poorly written... it's just code targeting the implementation.
  - anonacct37 9y ago
    That's not the concern. Python already has threads and race conditions (although the GIL means that the interpreter itself probably won't get corrupted while executing a piece of bytecode). What python doesn't have is a C api for extensions that makes sense without a GIL. So ideally a correct threadsafe C extension will continue to be correct, which probably implies that a function called "PyEval_AcquireLock" will continue to provide similar guarantees. Which means that the process for utilizing more cores with pure python code in one process will probably be a gradual upgrade process.
    - rguillebert 9y ago
      C extensions will still run under the GIL
      - int_19h 9y ago
        Given the amount of C extension code running in a typical large Python app these days, isn't this basically defeating the purpose?
        rguillebert 9y ago
        It really depends on the use case I think
- lemoncucumber 9y ago
  Would you say the Python ecosystem is stuffed to the GILs with incompatible libraries?
chubot 9y ago
Who uses PyPy? I have been hearing about it for so long now, maybe 10 years. And I have been programming in Python almost full-time for 14 years. But still I don't know anybody who uses it? It seems like the C extension API is still an issue, or am I mistaken?
- est 9y ago
  we run a large scale sockjs cluster. Switched from CPython to PyPy, instant 3x performance boost.
heavenlyblue 9y ago
Andrew Godwin had raised £17K out of £2.5K expected in order to implement (I believe excellent) Django migrations that are now part of the official repository: https://www.kickstarter.com/projects/andrewgodwin/schema-migrations-for-django https://www.kickstarter.com/projects/andrewgodwin/schema-mig... Neither do I think that raising $50K for Python interpreter would be an issue. PS: I don't find Django an excellent ORM per se. On the other hand it's highly pragmatic, and their implementation of automatically-generated migrations have saved a good chunk of my time.

andruby 9y ago

How can they estimate this? What about all the libraries that might not be compatible with the solution PyPy comes up with? This feels like a number that might in the end blow up to 10x the original estimate.

joaodlf 9y ago
This reminds me how the sales/marketing teams in my company typically sell new features: "Not having this feature costs us 50k a month!" It might as well not be the case here, I just found it funny, 50k is our little magic number.
rguillebert 9y ago
It's not the PyPy developers' job to make every Python library threadsafe, people writing libraries will have to make their code threadsafe, like in every other language.
- taeric 9y ago
  There is a clear difference here, though. Making a change that could lead to poorly written libraries now being broken is clearly the fault of the change. Userspace for these libraries is defined by how it is, not how it was intended. (And really, was it intended to be dangerous in this way?)
  - masklinn 9y ago
    > There is a clear difference here, though. Making a change that could lead to poorly written libraries now being broken is clearly the fault of the change. No, these libraries are already semantically broken in the same way e.g. libraries which didn't properly close their files and assumed the CPython refcounting GC would wipe there asses were broken. They're already broken under two non-GIL'd implementations.
  - koolba 9y ago
    No the fault in that situation is a user blindly upgrading PyPy without testing the totality of their software package and its dependencies. Expecting bad code to magically work forever is unrealistic and hinders progress.
  - rguillebert 9y ago
    Then just use the version of PyPy with a GIL?
  - cjhanks 9y ago
    I agree. Even developers who are well aware of how to write thread-safe code probably don't even bother with mutex locking in Python. That code isn't poorly written... it's just code targeting the implementation.
- anonacct37 9y ago
  That's not the concern. Python already has threads and race conditions (although the GIL means that the interpreter itself probably won't get corrupted while executing a piece of bytecode). What python doesn't have is a C api for extensions that makes sense without a GIL. So ideally a correct threadsafe C extension will continue to be correct, which probably implies that a function called "PyEval_AcquireLock" will continue to provide similar guarantees. Which means that the process for utilizing more cores with pure python code in one process will probably be a gradual upgrade process.
  - rguillebert 9y ago
    C extensions will still run under the GIL
    - int_19h 9y ago
      Given the amount of C extension code running in a typical large Python app these days, isn't this basically defeating the purpose?
      - rguillebert 9y ago
        It really depends on the use case I think
lemoncucumber 9y ago
Would you say the Python ecosystem is stuffed to the GILs with incompatible libraries?

chubot 9y ago

Who uses PyPy? I have been hearing about it for so long now, maybe 10 years. And I have been programming in Python almost full-time for 14 years. But still I don't know anybody who uses it? It seems like the C extension API is still an issue, or am I mistaken?

est 9y ago
we run a large scale sockjs cluster. Switched from CPython to PyPy, instant 3x performance boost.

heavenlyblue 9y ago

Andrew Godwin had raised £17K out of £2.5K expected in order to implement (I believe excellent) Django migrations that are now part of the official repository: https://www.kickstarter.com/projects/andrewgodwin/schema-migrations-for-django https://www.kickstarter.com/projects/andrewgodwin/schema-mig... Neither do I think that raising $50K for Python interpreter would be an issue. PS: I don't find Django an excellent ORM per se. On the other hand it's highly pragmatic, and their implementation of automatically-generated migrations have saved a good chunk of my time.