5 ms·

"The biggest source of problems might be multi-threaded programs with concurrency-related bugs that have been masked by the GIL until now."

• Animats 5y ago
  Yes. I once discovered that CPickle was not thread-safe. The response was that much of the library didn't really work in multi-threaded programs.
  • formerly_proven 5y ago
    You mean programs where you put an object into pickle and some other threads modify it while pickle is processing it? Doesn't surprise me - the equivalent written in plain Python would be very thread unsafe as well.
    • Animats 5y ago
      No, I mean several threads doing completely separate CPickle streams with no shared data or variables at the Python level.
      • kzrdude 5y ago
        Has it since been fixed?
        • toyg 5y ago
          Probably not. CPickle is famously shunned by anyone who has to do serious, performance-critical serialization/deserialization.
          • kzrdude 5y ago
            I was curious, and an issue that fits the description was fixed in Py 3.7.x here: https://bugs.python.org/issue34572 https://bugs.python.org/issue34572 but other threading bugs remain: https://bugs.python.org/issue38884 https://bugs.python.org/issue38884
          • azinman2 5y ago
            What’s the most performance critical alternative? Pickle is tied to the VM, so it’s not a generally good persistence option in a prod setup, but it can mighty convenient.
          • garmaine 5y ago
            What do people use instead?
            • da39a3ee 5y ago
              JSON, protocol buffers, thrift, etc. Saving python-native objects such as functions, class instances etc is usually not the right thing to do in production code.
• zinodaur 5y ago
  > concurrency-related bugs that have been masked by the GIL Yeah... could phrase this as "All programs written with the assumption of a GIL are now broken" instead. Wish they had done this as part of the breaking changes for python 3, I guess they'll have to wait for Python 4 for this?
  • sherjilozair 5y ago
    GIL can remain default on. Users can simply disable it for chosen parts of their program.
    • orf 5y ago
      No they can’t, it would have to be disabled or enabled globally at the process and/or interpreter level.
  • smcl 5y ago
    I think I read that there won’t be another big jump like there was for Python 2-3. If I understood correctly, there could be a Python 4, but it won’t indicate huge breaking changes, it’ll just be another release