3 ms·

By parallelism do you mean something different from multiprocessing? [1] [1]: https://docs.python.org/3/library/multiprocessing.html https://docs.python.org/3/

• triceratops 5y ago
  Pretty sure they're referring to the GIL.[1] 1. https://realpython.com/python-gil/ https://realpython.com/python-gil/
• samsquire 5y ago
  I want Thread to run in parallel. I've tried multiprocessing but I found it buggy for my use case. I tried to create a worker queue and submit items to worker threads with JoinableQueue but eventually the system deadlocks and no progress is made. I'm not sure what the problem is. It never finishes. It's a sentence correlator using multiprocessing - it generates correlations of words in your sentences. https://github.com/samsquire/notebook/blob/master/sentence-correlator.py https://github.com/samsquire/notebook/blob/master/sentence-c... https://github.com/samsquire/notebook/blob/master/workers.py https://github.com/samsquire/notebook/blob/master/workers.py Maybe somebody can catch what I'm doing wrong. edit: it may be because i've forgotten to run task_done on joinable queue. edit2: yes it was that! it works!
  • aw1621107 5y ago
    Out of curiosity, is using multiprocessing.Pool’s map/etc.() methods not an option? It’s a higher-level API which would remove the need to manually manage workers/a queue.
• dekhn 5y ago
  yes, for many of us, we really do want a CPython that runs multiple threads of the same interpreter, with memory sharing. That said after decades of CPython not supporting it, I've rewritten most of my code to use multiprocessing with granular parallelism.