3 ms·
This isn't necessarily a direct reply but instead another anecdote which reinforces OP's claim that asynchronous programming is a thorn in the side of Python.
by dwrodri 4y ago
This isn't necessarily a direct reply but instead another anecdote which reinforces OP's claim that asynchronous programming is a thorn in the side of Python.
I was tasked with improving a performance-critical object detection codebase for video that was written in Python. Before attempting a complete rewrite, I identified that outside of FFI calls related to PyTorch/OpenCV, some of the biggest latency overhead was due to the fact that the network I/O for fetching data to process was blocking the actual inference, so I tried to decouple the two so they could happen synchronously.
My attempt at doing this with asyncio went horrendously at best. There were no mature client libraries for the cloud services we used at the time. I was digging through PyTorch forums to figure out the relationship between the GIL and CUDA Streams. Boundless headaches.
Then I realized, I could just separate the code that loaded the data onto the machine from the inference code into two separate code bases and just have them communicate over really simple signals on a UNIX socket. Two different processes, sending a simple fixed-size string back and forth, and then I just used multiprocessing.Pool for parallelizing the downloads.