2 ms·
Sure. There's already variants like Jython that have removed the GIL. The larger problem is the existing codebase, which is based around the assumption of non
by paulmd 10y ago
Sure. There's already variants like Jython that have removed the GIL.
The larger problem is the existing codebase, which is based around the assumption of non-concurrency. You also have a lot of libraries that use C extensions for performance, and a lot of those are going to break horribly if you suddenly throw them into a concurrent environment where the Python state is mutating underneath them.
Again though, "rewriting code for concurrency" is not a fundamentally unsolvable problem, it just takes a lot of engineer time to change everything over. Moving global/static state into instances, adding locks, marking atomic/critical segments, that kind of thing. There's just a lot of things built with Python that would need to be gone over.
I really think you could do a switchover on the fly by adding a Java-style "synchronized" attribute. Before you can enter a synchronized method, you set a flag and all other threads must yield at their next return, function call, loop iteration, the end of their atomic segment, or at safe points marked by a "yield" statement (pick some combination of reasonable behavior). While a synchronized method is on the call stack, no other thread may execute. All existing code is marked synchronized - perhaps any code in a .py file is assumed unsafe by default, while any code in a .jy file is assumed safe. Boom, start converting code.
The thing that really gets me is that Python 3 is already pushing breaking changes that necessitate a complete overhaul anyway. The failure to thread the interpreter/remove the GIL at the same time is a stunningly idiotic decision. How about since we are making everyone review their code anyway, we have them look at thread safety too?
Which leads to the other problem - Python is GVR's baby, and at the end of the day the reason Python 3 has a GIL is because he says so. By all means, Google could go ahead and rewrite everything, but he'd never let them call it Python. It's hard to build momentum for a serious fork like that. And you don't want to spend a bunch of engineer time and end up with an unsupported "toy" that nobody uses.
It's a shame, Python hits real close to the mark but concurrency is its Achilles' Heel. I love the language but I am gunshy about using it because you never know if your project will go from "toy" to "real product that may need to adapt/scale" and you need concurrency.