4 ms·
Python Concurrency: An Example of a Queue
- mekoka 14y agoRather off-topic, but why is the module's name "Queue" and not "queue"? That kind of things just rubs me the wrong way.
- deleted 14y ago[deleted]
- hoov 14y agoHistorical reasons. It's been renamed in Python 3.
- smegel 14y agoDo we need a HN front-page post now for every module in the Python standard library?
- salman89 14y agoIf it makes it to the front page the it means enough people found it insightful or helpful. I for one did, and when/if I want to do some concurrent threading in Python, I know where to start.
- zalew 14y agothey complement the revelations about discovering git yesterday and tutorials how to run a django project.
- gsibble 14y agoThis is indeed awesome and I've used it in several applications. I forget though: does this use a single global interpreter or do the additional threads use their own interpreters?
- hythloday 14y agoThreads inherit the process's interpreter (which is how they share the GIL). Contrast to multiprocessing, which spawns a new interpreter and communicates with it via a python IPC, such as multiprocessing.Queue, which shares the interface with queue.Queue or the slightly clumsy multiprocessing.Value class: from multiprocessing import Process, Value def f(n): n.value = 3.1415927 num = Value('d', 0.0) p = Process(target=f, args=(num)) p.start() p.join() print num.value # 3.1415927
- Luyt 14y agoAh, args should be a tuple: p = Process(target=f, args=(num,))
- dudus 14y agoWould have been more helpful to link directly to the original post, that has a lot more information and which he just copied to his site on the most part: http://www.ibm.com/developerworks/aix/library/au-threadingpython/ http://www.ibm.com/developerworks/aix/library/au-threadingpy... But I think that doing so wouldn't generate visits to his blog.
- deleted 14y ago[deleted]
- simgidacav 14y agoI heard that python concurrency model is based on userspace threads, so when I saw the "setDaemon" call I investigated and learned[1] the difference between daemon and non-daemon python thread (I hoped to see a different difference - oh, a pun). That said, why making daemonize a thread when we'll be joining it later? [1]: http://stackoverflow.com/questions/190010/daemon-threads-explanation http://stackoverflow.com/questions/190010/daemon-threads-exp...
- wslh 14y agoIf you find it interesting I recommend my own articles on the subject: - Esoteric Queue Disciplines: http://blog.databigbang.com/esoteric-queue-scheduling-disciplines/ http://blog.databigbang.com/esoteric-queue-scheduling-discip... - Adding Acknowledgement Semantics to a Persistent Queue: http://blog.databigbang.com/adding-acknowledgement-semantics-to-a-persistent-queue/ http://blog.databigbang.com/adding-acknowledgement-semantics... - Persisting Native Python Queues: http://blog.databigbang.com/persisting-native-python-queues/ http://blog.databigbang.com/persisting-native-python-queues/ - Using Queues in Web Crawling and Analysis Infrastructure: http://blog.databigbang.com/using-queues-in-web-crawling-and-analysis-infrastructure/ http://blog.databigbang.com/using-queues-in-web-crawling-and... - Using Queues in a crawler to discover facebook, twitter, blogs on web pages: http://blog.databigbang.com/automated-discovery-of-blog-feeds-and-twitter-facebook-linkedin-accounts-connected-to-business-website/ http://blog.databigbang.com/automated-discovery-of-blog-feed...
- hoka 14y agoAs someone learning python (on mostly windows, but I test on ubuntu now and then to see if it works well) my advice is to just learn the multiprocessing version as shown by hythloday. I had a lot of problems with threads not exiting/joining properly, and even my multiprocessing code was goofy on windows. I used the "poison pill" method at http://www.doughellmann.com/PyMOTW/multiprocessing/communication.html#multiprocessing-queues http://www.doughellmann.com/PyMOTW/multiprocessing/communica... and everything works on ubuntu. I'm aware of the __main__ thing for windows, but I don't think that's my problem :-/