4 ms·
I don't see why it is irrational to wish that a general purpose programming language be able to run data intensive, parallel server code. It's not exactly askin
by fauigerzigerk 16y ago
I don't see why it is irrational to wish that a general purpose programming language be able to run data intensive, parallel server code. It's not exactly asking for HTML to be modified so you can write signal processing code in it.
The sad thing is that a Python + C combination would actually be _more_ suitable for the task than any of the JVM based language implementations, because the JVM has the unfortunate feature of eating memory like crazy due to the lack of structured value types.
This type of application is all about conserving memory. The core data structures must be extremely tight. But they make up only like 5% of the code. So Python + C would be way superior to Java for something like Cassandra if it wasn't for the GIL.
I guess I have to practice meditation for a few years so I can one day be as relaxed about these things as you are ;-)
- jnoller 16y agoWell, if it was I/O bound - the GIL shouldn't bother it too much ;) Don't mistake me for calm though, it drives me nuts :)
- rbanffy 16y agoIt's not irrational. But, since you are willing to rely on C in addition to Python, why not code the parallelizable parts in C and leave the rest in Python. And you can always write a proof-of-concept in Python, prove the algorithm is right and rewrite the performance-sensitive parts in C. If your tests are thorough, all is good.
- fauigerzigerk 16y agoThat's unfortunately not an option because all requests would have to go through that single threaded bottleneck to get to the parallelized C code, which is only acceptable for long running operations. For a request/response type of scenario like Cassandra, Lucene or the kind of analytics systems I'm working on it would make the whole parallelization pointless. The realistic option is to use a separate C based process behind the usual multi-process mod_python setup. That works but is more complicated and error prone than an in-process solution. So I think that's why so many new projects that have these kinds of scenarios (NoSQL for instance) go with Java in spite of its flaws.