5 ms·
Easy Concurrency in Python
- theandrewbailey 7y agoOne thing to remember about ProcessPoolExecutor is that parameters and return values are pickled (serialized/deserialized) in order to communicate with other Python processes. Subinterpreters (PEP 554) aims to alleviate this. https://www.python.org/dev/peps/pep-0554/ https://www.python.org/dev/peps/pep-0554/
- auraham 7y agoHow can I determine whether a function can be pickled? A few months ago, I developed a concurrent system using ProcessPoolExecutor. In that case, the function made use of numpy, scipy, and some instaces of classes. I worked as expected, but I am not sure if that function was pickable.
- hechang1997 7y agojust like the author said in the end, this is literally the simplest case of concurrent programming where it's very easy to break your works down into independent pieces and no synchronization is required.
- vonseel 7y agoThis is all great and exactly how I approached basic async stuff in Python 2.7 but I’m surprised there’s literally no mention of async/await syntax or the AsyncEventLoop.