3 ms·
This kind of makes it sound like you have a very limited variety of experience programming. Two things: 1) Race conditions exist with any sort of asynchronous
by jwarkentin 13y ago
This kind of makes it sound like you have a very limited variety of experience programming. Two things:
1) Race conditions exist with any sort of asynchronous or concurrent software development, even when there is no parallelism or threading. Have you ever written a moderately complex JavaScript app? There's no shortage of race conditions and no threads either.
2) There are in fact cases where forking isn't a reasonable solution. For one of my previous jobs I developed a multi-threaded server with Python (before I learned about the GIL). The problem with forking came in the fact that there was one particular variable that needed to be shared between threads for processing. It happened to contain about 5GB of data, memory which I didn't want duplicated and which would have been a pain in the butt to keep synced between processes.
For most of these shared data structures I could do multiple processes and just use Redis as a shared data store. However, this piece of data was accessed far too frequently and that would have just been a major performance bottleneck.
To say the least, I was terribly frustrated when I learned about the GIL. I wish they would put a big warning up on their home page or at least in the docs relating to threading.
- mzs 13y agomultiprocessing is just pretty broken in general. I've never been able to get it to work well in FreeBSD in particular. Essentially python calls all sorts of unsafe routines after the fork, here's an example bug report: http://bugs.python.org/issue3770 http://bugs.python.org/issue3770