6 ms·

Overall I've been extremely unimpressed with Python's async story. They've spent a decade baking into the core of the language and the result is that it's no b

Overall I've been extremely unimpressed with Python's async story. They've spent a decade baking into the core of the language and the result is that it's no better than just using gevent or twisted or any of those other "old school" frameworks. Except now everyone is forced to support it because there are colored functions everywhere and you have to write library code that has to deal with all the myriad ways your stuff can be used depending on if you're in an event loop or not. It's baffling. Async/await at the language level is a total scam. Give me Nginx+Lua and let me just write my regular procedural code and the underlying runtime will handle yielding/resuming for me.

anonymoushn 5y ago
It has certainly been depressing, as a greenlet user over a decade ago, to watch people spend weeks rewriting codebases to add "await" in all the right places.
- moffkalast 5y ago
  Name two things that go better together, Python devs and breaking shit that has no reason to be broken.
MrBuddyCasino 5y ago
> async/await at the language level is a total scam This view will magically become mainstream as soon as "having worked with async" stops being a resume inflating selling point. I'm very glad Java is doing the right thing and hiding everything under the same "Thread" abstraction.
rich_sasha 5y ago
Most code seems to take sides: it’s either synchronous or not (which I suppose in itself isn’t great). For many good standard libraries/packages, there is now an async variant. I used asyncio for the first time seriously recently and was pleasantly surprised. But, crucially, I didn’t want throughput, I wanted a low-cpu use app with easy-to-write concurrency. I really hope GIL goes away and we can just use threads.
- nextaccountic 5y ago
  I really hope GIL goes away so that async tasks can be dispatched in many threads, like done in Go or Rust. Something like one thread per core, plus work stealing to distribute tasks in threads. Dealing with threads directly is significantly worse. It's harder to do task cancellation, harder to do timeouts, you often need to reach for locks for the simplest things.
- abrookins 5y ago
  Async Python has proven faster in my uses for IO and non-CPU-related stuff. But I think Python, either as a community or within the language, needs to solve the anti-pattern of maintaining separate sync and async versions of a library. I'm thinking specifically of aioredis and redis-py, both of which I've worked on. Some people are looking at ways to solve this. I know urllib3, elasticsearch-py, and a few others use unasync (https://github.com/python-trio/unasync https://github.com/python-trio/unasync) to transform async code into sync code, leaving one codebase supporting both uses in different namespaces. This leaves you with some conditional logic (is_async_mode() -- https://github.com/python-trio/hip/blob/master/src/ahip/util/unasync.py#L8 https://github.com/python-trio/hip/blob/master/src/ahip/util...). I'm seriously considering this approach.
  - VWWHFSfQ 5y ago
    > to transform async code into sync code oh my goodness.. So now we have sync code, that we've made into async code, that we now have to turn back into sync code! It's just workarounds upon workarounds upon workarounds. This is beyond the pale.
Azsy 5y ago
Async await at the language level isn't a scam. Its an attempt to build an alternative to statemachines( large switch statements) and nested callbacks. It just so happened that we were trying to break 100k connections on a single machine at the same time, which requires you to avoid thread switching. Something which an async executor is also doing. Those two goals got combined into to "the great next thing that's better in every way" when in reality, 99% will never hit that bottleneck, and the people who write good async code could write it in either of the two other forms as well.
taude 5y ago
I think the forcing of Async into Python is largely a waste of effort. There's better languages and tooling for those jobs. Effective devs use the right tool for the right job. I've had to work on an async Python codebase the other day, and was surprised at how janky and cumbersome the syntax is (though I shouldn't be surprised). The whole setting up the event_loop, create_task, run_until_complete, etc. just feels pretty gross. I'd rather just implement something I knew to be performant in something like Go, C#, etc. a language that supports first class coroutine-style/message passsing support...
skohan 5y ago
Honestly I have yet to see a language really nail async. For instance much attention has been given to async rust, but in my experience it is 100x more difficult to work with async than standard synchronous rust. So many gotchas, and the errors are extremely cryptic which is counter to what you expect in normal rust. I found working with GCD was actually pretty good: of course it doesn't solve the safety concerns which rust aims to, but it's a sensible set of primitives which can be composed to achieve virtually whatever async problem you want. I think there are two possibilities: 1. We haven't really landed on the right abstraction for async yet. Current approaches try to hide the wrong bits of complexity and this makes async difficult to reason about, or at least makes it a bit of a minefield of special cases and considerations. 2. Or, async programming is just complex, and there's never going to be a way to tie it up as a neat little consistent language feature.
- dnautics 5y ago
  > We haven't really landed on the right abstraction for async yet I think there are two 'correct' abstractions for async. One is where you build it as a high level abstraction on top of other concurrency primitives. Elixir's Task.async* functions do this spectacularly, and even do things that you don't get natively from Erlang that make it very nice to work with (by giving you an opinionated way of keying into global state partitions). The other "correct" async abstraction, imo, is a very thin, low-level abstraction over function frames and context switches. The zig programming language gets this right, IMO. I was able to write a feature that wraps zig's low-level async primitive in an event loop around the erlang's cooperative FFI scheduler, the rust equivalent hasn't figured that out yet, and since zig is colorless, you can use the same function in a non-async context, the yield points are just no-ops (which rust will never let you do, for reasons beyond just safety).
- fwip 5y ago
  Honestly, I think Go's approach of syntactically light green threads is just better than async-as-keyword. There's a lot of other unergonomic stuff (like the verbosity of channel definition) that can make it a pain to use, but that's more Go-the-language's fault. Everything a modern computer does has potential latency - whether that's waiting for the database to respond or waiting for L2 cache to respond. The programmer and the application needs decide when it's worth waiting around for the result, and when it's important to do other work in the meantime.
stjohnswarts 5y ago
Is someone or something stopping you from using nginx and lua?
dehrmann 5y ago
> the result is that it's no better than just using gevent or twisted That's because they both use generator cleverness under the hood. What asyncio does do is remove all the monkey patching that makes gevent work.
keewee7 5y ago
Language level async/await in C# has been a success. However there is some RTFM to avoid shooting yourself in the foot.

anonymoushn 5y ago

It has certainly been depressing, as a greenlet user over a decade ago, to watch people spend weeks rewriting codebases to add "await" in all the right places.

moffkalast 5y ago
Name two things that go better together, Python devs and breaking shit that has no reason to be broken.

MrBuddyCasino 5y ago

> async/await at the language level is a total scam This view will magically become mainstream as soon as "having worked with async" stops being a resume inflating selling point. I'm very glad Java is doing the right thing and hiding everything under the same "Thread" abstraction.

rich_sasha 5y ago

Most code seems to take sides: it’s either synchronous or not (which I suppose in itself isn’t great). For many good standard libraries/packages, there is now an async variant. I used asyncio for the first time seriously recently and was pleasantly surprised. But, crucially, I didn’t want throughput, I wanted a low-cpu use app with easy-to-write concurrency. I really hope GIL goes away and we can just use threads.

nextaccountic 5y ago
I really hope GIL goes away so that async tasks can be dispatched in many threads, like done in Go or Rust. Something like one thread per core, plus work stealing to distribute tasks in threads. Dealing with threads directly is significantly worse. It's harder to do task cancellation, harder to do timeouts, you often need to reach for locks for the simplest things.
abrookins 5y ago
Async Python has proven faster in my uses for IO and non-CPU-related stuff. But I think Python, either as a community or within the language, needs to solve the anti-pattern of maintaining separate sync and async versions of a library. I'm thinking specifically of aioredis and redis-py, both of which I've worked on. Some people are looking at ways to solve this. I know urllib3, elasticsearch-py, and a few others use unasync (https://github.com/python-trio/unasync https://github.com/python-trio/unasync) to transform async code into sync code, leaving one codebase supporting both uses in different namespaces. This leaves you with some conditional logic (is_async_mode() -- https://github.com/python-trio/hip/blob/master/src/ahip/util/unasync.py#L8 https://github.com/python-trio/hip/blob/master/src/ahip/util...). I'm seriously considering this approach.
- VWWHFSfQ 5y ago
  > to transform async code into sync code oh my goodness.. So now we have sync code, that we've made into async code, that we now have to turn back into sync code! It's just workarounds upon workarounds upon workarounds. This is beyond the pale.

Azsy 5y ago

Async await at the language level isn't a scam. Its an attempt to build an alternative to statemachines( large switch statements) and nested callbacks. It just so happened that we were trying to break 100k connections on a single machine at the same time, which requires you to avoid thread switching. Something which an async executor is also doing. Those two goals got combined into to "the great next thing that's better in every way" when in reality, 99% will never hit that bottleneck, and the people who write good async code could write it in either of the two other forms as well.

taude 5y ago

I think the forcing of Async into Python is largely a waste of effort. There's better languages and tooling for those jobs. Effective devs use the right tool for the right job. I've had to work on an async Python codebase the other day, and was surprised at how janky and cumbersome the syntax is (though I shouldn't be surprised). The whole setting up the event_loop, create_task, run_until_complete, etc. just feels pretty gross. I'd rather just implement something I knew to be performant in something like Go, C#, etc. a language that supports first class coroutine-style/message passsing support...

skohan 5y ago

Honestly I have yet to see a language really nail async. For instance much attention has been given to async rust, but in my experience it is 100x more difficult to work with async than standard synchronous rust. So many gotchas, and the errors are extremely cryptic which is counter to what you expect in normal rust. I found working with GCD was actually pretty good: of course it doesn't solve the safety concerns which rust aims to, but it's a sensible set of primitives which can be composed to achieve virtually whatever async problem you want. I think there are two possibilities: 1. We haven't really landed on the right abstraction for async yet. Current approaches try to hide the wrong bits of complexity and this makes async difficult to reason about, or at least makes it a bit of a minefield of special cases and considerations. 2. Or, async programming is just complex, and there's never going to be a way to tie it up as a neat little consistent language feature.

dnautics 5y ago
> We haven't really landed on the right abstraction for async yet I think there are two 'correct' abstractions for async. One is where you build it as a high level abstraction on top of other concurrency primitives. Elixir's Task.async* functions do this spectacularly, and even do things that you don't get natively from Erlang that make it very nice to work with (by giving you an opinionated way of keying into global state partitions). The other "correct" async abstraction, imo, is a very thin, low-level abstraction over function frames and context switches. The zig programming language gets this right, IMO. I was able to write a feature that wraps zig's low-level async primitive in an event loop around the erlang's cooperative FFI scheduler, the rust equivalent hasn't figured that out yet, and since zig is colorless, you can use the same function in a non-async context, the yield points are just no-ops (which rust will never let you do, for reasons beyond just safety).
fwip 5y ago
Honestly, I think Go's approach of syntactically light green threads is just better than async-as-keyword. There's a lot of other unergonomic stuff (like the verbosity of channel definition) that can make it a pain to use, but that's more Go-the-language's fault. Everything a modern computer does has potential latency - whether that's waiting for the database to respond or waiting for L2 cache to respond. The programmer and the application needs decide when it's worth waiting around for the result, and when it's important to do other work in the meantime.

stjohnswarts 5y ago

Is someone or something stopping you from using nginx and lua?

dehrmann 5y ago

> the result is that it's no better than just using gevent or twisted That's because they both use generator cleverness under the hood. What asyncio does do is remove all the monkey patching that makes gevent work.

keewee7 5y ago

Language level async/await in C# has been a success. However there is some RTFM to avoid shooting yourself in the foot.