4 ms·
Yes, the question is whether Python intends to support the growing number of usage scenarios for which the GIL is a problem. You could write something like Cass
by fauigerzigerk 16y ago
Yes, the question is whether Python intends to support the growing number of usage scenarios for which the GIL is a problem. You could write something like Cassandra or Lucene using a combination of Python and C, but the GIL prevents that.
- rbanffy 16y agoYou could write something like Cassandra or Lucene in Python. Python is a language (much like Java, BTW). You could then run it under Jython or PyPy. The GIL is unique do CPython, AFAIK.
- fauigerzigerk 16y agoPyPy and Jython lag far behind the current version of Python, which is defined by the implementation of CPython. Also, as I said, I would like to use a combination of Python and C. That's for a reason. The JVM does not have structured value types (structs in C). All structured types in Java have to be referenced, which is the reason why Java software eats so much more memory than software implemented in C, C++, C# or Go. Maybe PyPy can work some day, if they can catch up. That day is definately not today.
- rbanffy 16y agoPyPy and Jython may lag behind, but that's not a huge problem. I would love to use dict comprehensions and {} sets like I can in CPython 2.7, but I know I can't always deploy against the latest version. And, more than that, I know I can work around those features. The code may look uglier, but it's usually as fast as it would be with the newest features. And you are right - CPython is the canonical implementation of the language. The fact we can use a reference implementation in production environments is great. You probably wouldn't be able to write something like Cassandra in Python 2.7 and expect it to run unmodified under Jython or IronPython, but you could write it in Python 2.5 and run it under both 2.7 CPython (with the expected multi-core annoyances) or under current Jython. And you can't have painless C modules in Java either.
- j_baker 16y agoActually, I believe PyPy has a GIL as well (unless they've removed it recently).
- rbanffy 16y agoOops... May have gotten that one wrong, but the point stands. Even if Jython is the only GIL-free implementation of Python, you can run your thread-heavy code on it. There is no such thing as "You wouldn't be able to do [cool-multi-core-product] in Python because of the GIL"
- fuzzyman 16y agoIronPython is another GIL free Python implementation.
- jnoller 16y agoAre you sure you couldn't write it?
- fauigerzigerk 16y agoI could write it, but it would either be dog slow or take 10 times as much time to write. The only way of using multiple cores would be to implement everything on top of shared memory. That implies using no pointers in any of the data structures and implementing a custom garbage collector.
- jnoller 16y agoAlright, so it's not the optimal language for something with a large amount of mutable shared state being hacked at by threads. No one ever tried to bill it as the "one true language" - maybe it's better for glue in this case, or APIs or clients, etc. Maybe it needs some jython, etc. This is why multiple languages (a diverse toolbox) and multiple interpreters each with their pros and cons is also a good thing. Each one will have its shortcomings, and it's almost impossible to achieve all of the goals of all of the modern programming languages in a single language. So, sure - I'd love to see python used for this problem - and all problems but thats me being irrational, that will never be true. What is true is that eventually the GIL will go away, or a non cPython interpreter will rise above cPython as the defacto standard.
- fauigerzigerk 16y agoI don't see why it is irrational to wish that a general purpose programming language be able to run data intensive, parallel server code. It's not exactly asking for HTML to be modified so you can write signal processing code in it. The sad thing is that a Python + C combination would actually be _more_ suitable for the task than any of the JVM based language implementations, because the JVM has the unfortunate feature of eating memory like crazy due to the lack of structured value types. This type of application is all about conserving memory. The core data structures must be extremely tight. But they make up only like 5% of the code. So Python + C would be way superior to Java for something like Cassandra if it wasn't for the GIL. I guess I have to practice meditation for a few years so I can one day be as relaxed about these things as you are ;-)