3 ms·
I want Thread to run in parallel. I've tried multiprocessing but I found it buggy for my use case. I tried to create a worker queue and submit items to worker
by samsquire 5y ago
I want Thread to run in parallel.
I've tried multiprocessing but I found it buggy for my use case.
I tried to create a worker queue and submit items to worker threads with JoinableQueue but eventually the system deadlocks and no progress is made. I'm not sure what the problem is. It never finishes.
It's a sentence correlator using multiprocessing - it generates correlations of words in your sentences.
https://github.com/samsquire/notebook/blob/master/sentence-correlator.py https://github.com/samsquire/notebook/blob/master/sentence-c...
https://github.com/samsquire/notebook/blob/master/workers.py https://github.com/samsquire/notebook/blob/master/workers.py
Maybe somebody can catch what I'm doing wrong.
edit: it may be because i've forgotten to run task_done on joinable queue.
edit2: yes it was that! it works!
- aw1621107 5y agoOut of curiosity, is using multiprocessing.Pool’s map/etc.() methods not an option? It’s a higher-level API which would remove the need to manually manage workers/a queue.