4 ms·

How can they estimate this? What about all the libraries that might not be compatible with the solution PyPy comes up with? This feels like a number that might

How can they estimate this? What about all the libraries that might not be compatible with the solution PyPy comes up with? This feels like a number that might in the end blow up to 10x the original estimate.

joaodlf 9y ago
This reminds me how the sales/marketing teams in my company typically sell new features: "Not having this feature costs us 50k a month!" It might as well not be the case here, I just found it funny, 50k is our little magic number.
rguillebert 9y ago
It's not the PyPy developers' job to make every Python library threadsafe, people writing libraries will have to make their code threadsafe, like in every other language.
- taeric 9y ago
  There is a clear difference here, though. Making a change that could lead to poorly written libraries now being broken is clearly the fault of the change. Userspace for these libraries is defined by how it is, not how it was intended. (And really, was it intended to be dangerous in this way?)
  - masklinn 9y ago
    > There is a clear difference here, though. Making a change that could lead to poorly written libraries now being broken is clearly the fault of the change. No, these libraries are already semantically broken in the same way e.g. libraries which didn't properly close their files and assumed the CPython refcounting GC would wipe there asses were broken. They're already broken under two non-GIL'd implementations.
  - koolba 9y ago
    No the fault in that situation is a user blindly upgrading PyPy without testing the totality of their software package and its dependencies. Expecting bad code to magically work forever is unrealistic and hinders progress.
  - rguillebert 9y ago
    Then just use the version of PyPy with a GIL?
  - cjhanks 9y ago
    I agree. Even developers who are well aware of how to write thread-safe code probably don't even bother with mutex locking in Python. That code isn't poorly written... it's just code targeting the implementation.
- anonacct37 9y ago
  That's not the concern. Python already has threads and race conditions (although the GIL means that the interpreter itself probably won't get corrupted while executing a piece of bytecode). What python doesn't have is a C api for extensions that makes sense without a GIL. So ideally a correct threadsafe C extension will continue to be correct, which probably implies that a function called "PyEval_AcquireLock" will continue to provide similar guarantees. Which means that the process for utilizing more cores with pure python code in one process will probably be a gradual upgrade process.
  - rguillebert 9y ago
    C extensions will still run under the GIL
    - int_19h 9y ago
      Given the amount of C extension code running in a typical large Python app these days, isn't this basically defeating the purpose?
      - rguillebert 9y ago
        It really depends on the use case I think
lemoncucumber 9y ago
Would you say the Python ecosystem is stuffed to the GILs with incompatible libraries?

joaodlf 9y ago

This reminds me how the sales/marketing teams in my company typically sell new features: "Not having this feature costs us 50k a month!" It might as well not be the case here, I just found it funny, 50k is our little magic number.

rguillebert 9y ago

It's not the PyPy developers' job to make every Python library threadsafe, people writing libraries will have to make their code threadsafe, like in every other language.

taeric 9y ago
There is a clear difference here, though. Making a change that could lead to poorly written libraries now being broken is clearly the fault of the change. Userspace for these libraries is defined by how it is, not how it was intended. (And really, was it intended to be dangerous in this way?)
- masklinn 9y ago
  > There is a clear difference here, though. Making a change that could lead to poorly written libraries now being broken is clearly the fault of the change. No, these libraries are already semantically broken in the same way e.g. libraries which didn't properly close their files and assumed the CPython refcounting GC would wipe there asses were broken. They're already broken under two non-GIL'd implementations.
- koolba 9y ago
  No the fault in that situation is a user blindly upgrading PyPy without testing the totality of their software package and its dependencies. Expecting bad code to magically work forever is unrealistic and hinders progress.
- rguillebert 9y ago
  Then just use the version of PyPy with a GIL?
- cjhanks 9y ago
  I agree. Even developers who are well aware of how to write thread-safe code probably don't even bother with mutex locking in Python. That code isn't poorly written... it's just code targeting the implementation.
anonacct37 9y ago
That's not the concern. Python already has threads and race conditions (although the GIL means that the interpreter itself probably won't get corrupted while executing a piece of bytecode). What python doesn't have is a C api for extensions that makes sense without a GIL. So ideally a correct threadsafe C extension will continue to be correct, which probably implies that a function called "PyEval_AcquireLock" will continue to provide similar guarantees. Which means that the process for utilizing more cores with pure python code in one process will probably be a gradual upgrade process.
- rguillebert 9y ago
  C extensions will still run under the GIL
  - int_19h 9y ago
    Given the amount of C extension code running in a typical large Python app these days, isn't this basically defeating the purpose?
    - rguillebert 9y ago
      It really depends on the use case I think

lemoncucumber 9y ago

Would you say the Python ecosystem is stuffed to the GILs with incompatible libraries?