3 ms·

I feel like this could unlock some huge performance gains with Python. If you want to truly "defeat the GIL" you must use processes instead of threads. This cou

• short_sells_poo 7mo ago
  How would this really help python though? This doesn't solve the difficult problem, which is that python objects don't support parallel access by multiple threads/processes, no? Concurrent threads, yes, but only one thread can be operating on a python object at a time (I'm simplifying here for brevity). There are already means of passing around bulk data with zero copy characteristics in python, but there's a lot of bureaucracy around it. A true solution must work with the GIL (or remove it altogether), no?
  • jer-irl 7mo ago
    I'm not familiar with CPython GC internals, but I there there are mechanisms for Python objects to be safely handed to C,C++ libraries and used there in parallel? Perhaps one could implement a handoff mechanism that uses those same mechanisms? Interesting idea!
• hun3 7mo ago
  This is exactly what subinterpreters are for! Basically isolated copies of Python in the same process. https://docs.python.org/3/library/concurrent.interpreters.html https://docs.python.org/3/library/concurrent.interpreters.ht... If you want a higher-level interface, there is InterpreterPoolExecutor: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.InterpreterPoolExecutor https://docs.python.org/3/library/concurrent.futures.html#co...