2 ms·

It IS easy. The Python project just don’t make it obvious how to do it easy.

xwowsersx 4y ago
Right. So you think it's mainly an education/documentation issue? I'm totally open to that in fact being the case. One really simple use case I had was just annotating a method to run async and then calling that in a loop elsewhere which did speed up a bunch of work I was doing pretty dramatically. import asyncio def background(f): def wrapped(*args, **kwargs): return asyncio.get_event_loop().run_in_executor(None, f, *args, **kwargs) return wrapped @background def my_io_bound_function():...
- m3047 4y ago
  You need to run the executor somewhere ( loop.run_forever() or loop.run_until_complete() ), that can be in the current thread or a separate thread, keep in mind Python is still conceptually a single core. Things I've found particularly useful: * Optional / debug logging of coroutine start/exit. * Stats logging, including count, runtime, request queue (multiple instances of the same function) depth. * Printing tracebacks within the coroutine context. Haven't expended any effort on decorators, good for you.
  - xwowsersx 4y ago
    Those do sound useful, thanks for this. Will keep in mind as I continue exploring.
  - m3047 4y ago
    I think I hit a foul ball there, as run_in_executor() is for a threadpool. So yeah you are running inside of run_forever() or run_until_complete(), we just don't see it here, and that's where your sending things to run in a different thread. ;-)
    - xwowsersx 4y ago
      Ah got it, thanks for the correction and info.
btilly 4y ago
It is easy if you know what to do. But you'll CONSTANTLY run into things where it is obvious that this is a toy that hasn't been thought through. I've found random slowdowns of a factor of 10 for no reason. A context manager needs to have __enter__ and __exit__ methods. An async context manager needs to have __aenter__ and __aexit__ methods. The error you get if you use a context manager as an async context manager or vice versa is big and scary and offers no clue to most programmers of what the problem is. Using async with SQLAlchemy it isn't hard to wind up exiting the event loop before all cleanup has happened. They fixed the main cause of this, but it isn't hard to still trigger it. And so on.

Right. So you think it's mainly an education/documentation issue? I'm totally open to that in fact being the case. One really simple use case I had was just annotating a method to run async and then calling that in a loop elsewhere which did speed up a bunch of work I was doing pretty dramatically. import asyncio def background(f): def wrapped(*args, **kwargs): return asyncio.get_event_loop().run_in_executor(None, f, *args, **kwargs) return wrapped @background def my_io_bound_function():...

m3047 4y ago
You need to run the executor somewhere ( loop.run_forever() or loop.run_until_complete() ), that can be in the current thread or a separate thread, keep in mind Python is still conceptually a single core. Things I've found particularly useful: * Optional / debug logging of coroutine start/exit. * Stats logging, including count, runtime, request queue (multiple instances of the same function) depth. * Printing tracebacks within the coroutine context. Haven't expended any effort on decorators, good for you.
- xwowsersx 4y ago
  Those do sound useful, thanks for this. Will keep in mind as I continue exploring.
- m3047 4y ago
  I think I hit a foul ball there, as run_in_executor() is for a threadpool. So yeah you are running inside of run_forever() or run_until_complete(), we just don't see it here, and that's where your sending things to run in a different thread. ;-)
  - xwowsersx 4y ago
    Ah got it, thanks for the correction and info.

btilly 4y ago

It is easy if you know what to do. But you'll CONSTANTLY run into things where it is obvious that this is a toy that hasn't been thought through. I've found random slowdowns of a factor of 10 for no reason. A context manager needs to have __enter__ and __exit__ methods. An async context manager needs to have __aenter__ and __aexit__ methods. The error you get if you use a context manager as an async context manager or vice versa is big and scary and offers no clue to most programmers of what the problem is. Using async with SQLAlchemy it isn't hard to wind up exiting the event loop before all cleanup has happened. They fixed the main cause of this, but it isn't hard to still trigger it. And so on.