5 ms·

Not mentioned: The behavior is different between python 3.6 and 3.8. That was fun to debug. Python is simply not the right language for asynchronous programm

Not mentioned: The behavior is different between python 3.6 and 3.8. That was fun to debug. Python is simply not the right language for asynchronous programming. Things that are easy in many other languages are hard and don't work as you'd expect. If you are in a python-only shop, brush up on your presentation skills and see if you can convince them to branch out. Or write a shell script. You're better off with "./read_socket.sh &" than you are with any python library.

brrrrrm 4y ago
All of distributed machine learning would benefit from async, but it's unfortunately a Python-only ecosystem for the time being. Probably gonna be a couple years before that has a chance of changing
- nine_k 4y ago
  How would ML benefit from async? I thought everything interesting in things like TF or PyTorch is done by native code which is free to use multithreading and anything at all. (I know little about ML.)
  - brrrrrm 4y ago
    Specifically distributed would be helped (multiple nodes). Right now the typical paradigm is this really clunky lock-step across nodes waiting for each other (everything is blocking). If your hardware is non-homogenous (both interconnects and accelerators) or your program needs to be run in a pipelined way, you're fighting an extremely uphill battle. Go, JavaScript etc are the usual languages of choice for this type of stuff because it's easy to express non-blocking code.
    - nine_k 4y ago
      I suspect multiprocessing + shared memory could help this. The stdlib has provisions for both, but a coordination layer is needed. That would be closest to true multithreading.
      - brrrrrm 4y ago
        it's not a matter of performance but expressibility. const y = await remote_model_a(x) // different machine const z = await remote_model_b(y) // different machine await z.backward() is trivially pipelined when run in parallel. With multithreading suddenly the backing C++ library has to be aware of this and figure things out for you
ollien 4y ago
What changed between those two releases? 3.6 was the last time I worked with asyncio
- deleted 4y ago
  [deleted]
- nineplay 4y ago
  Some of it is here https://tryexceptpass.org/article/asyncio-in-37/ https://tryexceptpass.org/article/asyncio-in-37/ One of my ongoing frustrations is python isn't just the lack of backwards compatibility but that if you inherit a script there's no way to know which version of the language it was written for. If you're lucky you can track down the developers, but IME even they often say "I'm not sure, let me run python --version".
  - gpderetta 4y ago
    I have only a bit of experience with asyncio, but it seem that I have hit all issues that are described there and improved in 3.7. I still can't believe that wait_closed wasn't there from day 1.
packetlost 4y ago
The asyncio module was heavily refactored in 3.7, which introduced the async/await keywords which was a *huge* improvement over the 3.6 and below implementation. I don't agree with this, but I will concede that async Python is rarely the correct choice.
- gpderetta 4y ago
  the async/await keyword predates 3.7. As far as I can tell it was introduced in 3.5.
gmadsen 4y ago
that is an absurd exaggeration, or you don't realize all the issues of low level sockets that are abstracted away
- nineplay 4y ago
  To follow up, the contents of read_socket.sh --- #!/bin/bash python3.8 read_one_socket.py $1 --- Use the python socket libraries if you must, but don't use python asynchronously.
  - gmadsen 4y ago
    id say 90% > of async python usecases are aiohttp , which with a single session, doesn't really have the issues you are describing.
dwrodri 4y ago
This isn't necessarily a direct reply but instead another anecdote which reinforces OP's claim that asynchronous programming is a thorn in the side of Python. I was tasked with improving a performance-critical object detection codebase for video that was written in Python. Before attempting a complete rewrite, I identified that outside of FFI calls related to PyTorch/OpenCV, some of the biggest latency overhead was due to the fact that the network I/O for fetching data to process was blocking the actual inference, so I tried to decouple the two so they could happen synchronously. My attempt at doing this with asyncio went horrendously at best. There were no mature client libraries for the cloud services we used at the time. I was digging through PyTorch forums to figure out the relationship between the GIL and CUDA Streams. Boundless headaches. Then I realized, I could just separate the code that loaded the data onto the machine from the inference code into two separate code bases and just have them communicate over really simple signals on a UNIX socket. Two different processes, sending a simple fixed-size string back and forth, and then I just used multiprocessing.Pool for parallelizing the downloads.
deleted 4y ago
[deleted]

brrrrrm 4y ago

All of distributed machine learning would benefit from async, but it's unfortunately a Python-only ecosystem for the time being. Probably gonna be a couple years before that has a chance of changing

nine_k 4y ago
How would ML benefit from async? I thought everything interesting in things like TF or PyTorch is done by native code which is free to use multithreading and anything at all. (I know little about ML.)
- brrrrrm 4y ago
  Specifically distributed would be helped (multiple nodes). Right now the typical paradigm is this really clunky lock-step across nodes waiting for each other (everything is blocking). If your hardware is non-homogenous (both interconnects and accelerators) or your program needs to be run in a pipelined way, you're fighting an extremely uphill battle. Go, JavaScript etc are the usual languages of choice for this type of stuff because it's easy to express non-blocking code.
  - nine_k 4y ago
    I suspect multiprocessing + shared memory could help this. The stdlib has provisions for both, but a coordination layer is needed. That would be closest to true multithreading.
    - brrrrrm 4y ago
      it's not a matter of performance but expressibility. const y = await remote_model_a(x) // different machine const z = await remote_model_b(y) // different machine await z.backward() is trivially pipelined when run in parallel. With multithreading suddenly the backing C++ library has to be aware of this and figure things out for you

ollien 4y ago

What changed between those two releases? 3.6 was the last time I worked with asyncio

deleted 4y ago
[deleted]
nineplay 4y ago
Some of it is here https://tryexceptpass.org/article/asyncio-in-37/ https://tryexceptpass.org/article/asyncio-in-37/ One of my ongoing frustrations is python isn't just the lack of backwards compatibility but that if you inherit a script there's no way to know which version of the language it was written for. If you're lucky you can track down the developers, but IME even they often say "I'm not sure, let me run python --version".
- gpderetta 4y ago
  I have only a bit of experience with asyncio, but it seem that I have hit all issues that are described there and improved in 3.7. I still can't believe that wait_closed wasn't there from day 1.

packetlost 4y ago

The asyncio module was heavily refactored in 3.7, which introduced the async/await keywords which was a *huge* improvement over the 3.6 and below implementation. I don't agree with this, but I will concede that async Python is rarely the correct choice.

gpderetta 4y ago
the async/await keyword predates 3.7. As far as I can tell it was introduced in 3.5.

gmadsen 4y ago

that is an absurd exaggeration, or you don't realize all the issues of low level sockets that are abstracted away

nineplay 4y ago
To follow up, the contents of read_socket.sh --- #!/bin/bash python3.8 read_one_socket.py $1 --- Use the python socket libraries if you must, but don't use python asynchronously.
- gmadsen 4y ago
  id say 90% > of async python usecases are aiohttp , which with a single session, doesn't really have the issues you are describing.

dwrodri 4y ago

This isn't necessarily a direct reply but instead another anecdote which reinforces OP's claim that asynchronous programming is a thorn in the side of Python. I was tasked with improving a performance-critical object detection codebase for video that was written in Python. Before attempting a complete rewrite, I identified that outside of FFI calls related to PyTorch/OpenCV, some of the biggest latency overhead was due to the fact that the network I/O for fetching data to process was blocking the actual inference, so I tried to decouple the two so they could happen synchronously. My attempt at doing this with asyncio went horrendously at best. There were no mature client libraries for the cloud services we used at the time. I was digging through PyTorch forums to figure out the relationship between the GIL and CUDA Streams. Boundless headaches. Then I realized, I could just separate the code that loaded the data onto the machine from the inference code into two separate code bases and just have them communicate over really simple signals on a UNIX socket. Two different processes, sending a simple fixed-size string back and forth, and then I just used multiprocessing.Pool for parallelizing the downloads.

deleted 4y ago

[deleted]