7 ms·

What's wrong with the async/await syntax?

by luhn 6y ago

jjtheblunt 6y ago
I usually see folks cite that async/await exposes, to the programmer, implementation details that one wishes were hidden by the compiler/interpreter.
- jonahbenton 6y ago
  That is python's MO, is why things like the double underscore functions exist. Being able to declare a coroutine (not just use) is not a bad idea, but IMHO it was done badly.
- gonational 6y ago
  Everybody wishes asynchronous functionality was magically hidden by the interpreter until unexpected shit starts happening.
VWWHFSfQ 6y ago
It should have been implemented as functions or attributes instead of language syntax sugar. Like Lua did. Like Rust did. Imagine this: def foo(): return "bar" async def foo(): return "bar" How do you know how to call that function? It's called the same thing. It has the same signature.. It returns the same thing. One is sync, the other is async. This is "colored functions". You have to call the 2nd one like: await foo() why do you have to care about that? It's either a coroutine or it's not. All functions in Python should just be callable like normal functions. But the Computer Scientists went and messed that up and made colored functions. It's absurd.
- tick_tock_tick 6y ago
  Isn't your example exactly how Rust works to?
  - haimez 6y ago
    Rust is statically typed, so it’s not quite the same effect
- contravariant 6y ago
  This was already the case with generators wasn't it? Worse in that case it depend on the yield statement, which cannot be seen from the function definition. In fact I'm pretty sure generators and async/await are almost the same feature, with possibly just some additional logic to allow await to suspend computation (so you don't have to write a loop `while not done(): sleep()` manually).
  - pansa2 6y ago
    Yes, `await` is just a modified form of `yield from`.
- pansa2 6y ago
  The argument for requiring `await` is that it makes explicit which calls might yield control back to the event loop. With Lua’s stackful coroutines, this information would either be hidden or just a naming convention, e.g. read_file vs read_file_async.
- pdonis 6y ago
  I don't mind the "await foo()" so much; as others have noted, it makes explicit that your code is yielding control at that point. What I mind is having to do "async def", "async for", "async with" all over the place for no good reason. Python didn't do that with generators; you defined an ordinary function, and if it had a "yield" somewhere in its body, it returned a generator. I don't have to do "gen def", "gen with", "gen for", etc. to remind the interpreter that it's a generator. The obvious way to handle async coroutines is the same: you define an ordinary function, and if it has "await" somewhere in its body, it's a coroutine. I shouldn't have to be reminding the interpreter everywhere that it's a coroutine. (The reason "await" is a special case is that, with a generator, it's just an ordinary iterable, so it's not yielding control if I just use it like any other iterable, say in a for statement. If I want to do something with the generator that might yield control (as in the "using generators as coroutines" paradigm that came before async/await), I have to call its "send()" method, or otherwise do something special that will stand out. So there's no need to add a keyword for it. But with a coroutine, ordinary call sites that wouldn't yield control with any other kind of function can yield control, so it makes sense to add a keyword to make that explicit.)
  - pansa2 6y ago
    > What I mind is having to do "async def", "async for", "async with" all over the place for no good reason. IMO `async for` and `async with` are misnomers - they are statements that call asynchronous functions (`__anext__`, `__aenter__` and `__aexit__`) so would have been better named something like `for await` and `with await`. `async def`, on the other hand, was just copied cargo-cult-style from C#. I don't think it's necessarily bad, but you're right to say that it's inconsistent with Python's existing generators.
  - BiteCode_dev 6y ago
    I have the opposition opinion. I think generators should have had an explicit keyword for defining them, like: gen def stuff() Because it doesn't, people are very confused about generators, and think they are functions, which they are not. They are completely different objects. Plus there is no way to tell if a definition is a generator without reading all of it. Explicit is better than implicit and all that.
- olau 6y ago
  Writing multi-threaded programs is much, much harder than single-threaded code. I think part of the attraction of async/await is that you can write mostly non-concurrent code. But then it gets extremely important that you're aware when you might have concurrency. For instance: x = get_something_from_global_state() y = foo() insert_into_global_state(x, y) If the program is suddenly possibly concurrent when calling foo(), then perhaps x is no longer valid. Good luck debugging that. await foo() makes it much more obvious. Now, of course, in scenarios where you don't really care about this property, e.g. if you're dealing with a database in another process where it is obvious that stuff may happen behind your back, or if you don't actually have any concurrency, I get that await is probably pretty annoying.
- peterkelly 6y ago
  If all you're ever going to do with the result of an async function is await it immediately after calling, then you don't need async functions because you're just doing synchronous programming with extra syntax. The reason there's a difference between foo() and await foo() is that the former gives you a future (aka Promise in JavaScript), while the latter gives you the result. You can then wait on multiple futures, so that things can happen concurrently. If you add type annotations to your code and use mypy (which you definitely should), it'll pick up the distinction between the two types - Awaitable[str] vs str - and give you an error if you try to pass the future object to a function that expects a string. The only case where I've run into problems is where I've called an async function that returns None, but have forgotten to await the result. In that case the function never actually begins execution.
- BiteCode_dev 6y ago
  > How do you know how to call that function? async def foo() is not a function and it doesn't have the same signature: the async is very explicit. It's a different primitive, and it's not used AT ALL for the same thing at a function. It's like writting: def foo(): yield "bar" and saying, 'how do you cal that function? How do you know how to call that function? It's called the same thing. It has the same signature.. It returns the same thing. One has one exit point, the other has several. This is "colored functions". You have to call the 2nd one like: next(iter(foo())) It's absurd' Well, no, it's not. It's a different tool. It's not a function, it's a generator. You don't have to care about async/await. In fact, you can live your entire life without using them. But it's a handy tool that allows you do: - delegate asynchronous behavior, like I/O, outside of your callable - make you match the syntaxic order with execution order despite said delegation - make it clear and explicit where you chose to do so, indentifying the context switching of your program Does it seems complicated ? It's because it is, and you probably don't need it. But if you need it, it's easier than the alternative of using callbacks, and less error prones that implicit asyncronicity.
jonahbenton 6y ago
I'll bite. Opinion follows, no need to downvote. async def is terrible, of the syntactic/semantic options it is the worst. Best would have been to just allow creation of a name of a coroutine-usable version of a function- def f: # normal f .... a = async(f) That allows existing functions to be turned into coroutines while not introducing the new syntactic pattern of a double word declaration.
- takeda 6y ago
  You still can do that using @asyncio.coroutine decorator for example @asyncio.coroutine def hello(): return "foo" Although not for long, they deprecated it in 3.8 and will be removed in 3.10. I wish they would leave it, since I was able to use them to write API that could be synchronous or not depending which constructor was invoked. I'm wondering if there are better ways to do it.
  - deleted 6y ago
    [deleted]
- oefrha 6y ago
  You can write a trivial decorator to make any normal blocking function async-usable (i.e. your suggestion is possible today and only takes a few lines of code — not even using deprecated functionality, I mean), but that doesn’t magically make it async-capable if it’s not designed for async in the first place. If you’re arguing for generator-based coroutines, then (1) that’s not just any normal function (check inspect.isgeneratorfunction and inspect.iscoroutinefunction); (2) it was already explored and deprecated. Guess we can have another talk on why if you wish.
  - jonahbenton 6y ago
    Sure, there are a few different issues here, and my response above was mostly off the cuff. I have had the pleasure of engaging with cooperative vs scheduled concurrency since the 1980s, in the early days of the Mac, and asynchronous IO since it started to enter the mainstream in the 1990s. I don't write enough python any more that this impacts me much; more that I remember finding out about it and being astonished. That's not to say- the async PEP is well written and one gets the mental model the author is promoting and the problems the PEP is trying to solve. I used tornado a bunch in its early days and it was and I'm sure still is a lovely piece of engineering. Nevertheless, among the reasons for my astonishment were: * there is a difference between cooperative concurrency and largely IO-scheduling driven asynchrony. There are a range of protocols that callers and callees engage in, a level of greater or lesser interest that the caller has on what and when the callee does it, and a variety of ways the callee can make what it did more or less eventually known to the caller. Cooperation and async capture two distinct common use cases in this rich space, yet the python implementation seems to improperly lump the two together. And the added keywords are crude and only provide partial expression of the range of options. For instance, deadlines are a top level consideration both in cooperation and in asynchrony, yet there is no syntax for specifying them. * there is a level of sophistication required for effective cooperative or asynchronous programming that, despite the rise of node and so forth, is simply not met by the bulk of programmers employing it, nor necessary in the programs they are writing. Elevating the syntax to first class as though it is something for most python programmers to consider- well, you can have a synchronous function, or an asynchronous function, pick one- is putting power tools in the hands of people who mostly don't know how to use them working on problems that entirely don't need them. Especially for python, this seems like the wrong thing to have done. * the doubled keywords pattern is ugly. Yes, if you are going to allow for cooperative behavior, decorating the affected syntactic entities is necessary. If it had to get added to the language I would much rather have seen new single tokens- adef and afor- for this purpose, maybe added to an "expert" module to be "imported" before use. With a whole new declarative keyword, one asks- what else can this be applied to? Why not have async variables or instance members? The syntax suggests itself and there are potential semantics. Forgive me, but I was shocked when I saw this and still think it was an immature and unnecessary addition to the language.

What's wrong with the async/await syntax?

by luhn 6y ago

What's wrong with the async/await syntax?

I usually see folks cite that async/await exposes, to the programmer, implementation details that one wishes were hidden by the compiler/interpreter.

jonahbenton 6y ago
That is python's MO, is why things like the double underscore functions exist. Being able to declare a coroutine (not just use) is not a bad idea, but IMHO it was done badly.
gonational 6y ago
Everybody wishes asynchronous functionality was magically hidden by the interpreter until unexpected shit starts happening.

VWWHFSfQ 6y ago

It should have been implemented as functions or attributes instead of language syntax sugar. Like Lua did. Like Rust did. Imagine this: def foo(): return "bar" async def foo(): return "bar" How do you know how to call that function? It's called the same thing. It has the same signature.. It returns the same thing. One is sync, the other is async. This is "colored functions". You have to call the 2nd one like: await foo() why do you have to care about that? It's either a coroutine or it's not. All functions in Python should just be callable like normal functions. But the Computer Scientists went and messed that up and made colored functions. It's absurd.

tick_tock_tick 6y ago
Isn't your example exactly how Rust works to?
- haimez 6y ago
  Rust is statically typed, so it’s not quite the same effect
contravariant 6y ago
This was already the case with generators wasn't it? Worse in that case it depend on the yield statement, which cannot be seen from the function definition. In fact I'm pretty sure generators and async/await are almost the same feature, with possibly just some additional logic to allow await to suspend computation (so you don't have to write a loop `while not done(): sleep()` manually).
- pansa2 6y ago
  Yes, `await` is just a modified form of `yield from`.
pansa2 6y ago
The argument for requiring `await` is that it makes explicit which calls might yield control back to the event loop. With Lua’s stackful coroutines, this information would either be hidden or just a naming convention, e.g. read_file vs read_file_async.
pdonis 6y ago
I don't mind the "await foo()" so much; as others have noted, it makes explicit that your code is yielding control at that point. What I mind is having to do "async def", "async for", "async with" all over the place for no good reason. Python didn't do that with generators; you defined an ordinary function, and if it had a "yield" somewhere in its body, it returned a generator. I don't have to do "gen def", "gen with", "gen for", etc. to remind the interpreter that it's a generator. The obvious way to handle async coroutines is the same: you define an ordinary function, and if it has "await" somewhere in its body, it's a coroutine. I shouldn't have to be reminding the interpreter everywhere that it's a coroutine. (The reason "await" is a special case is that, with a generator, it's just an ordinary iterable, so it's not yielding control if I just use it like any other iterable, say in a for statement. If I want to do something with the generator that might yield control (as in the "using generators as coroutines" paradigm that came before async/await), I have to call its "send()" method, or otherwise do something special that will stand out. So there's no need to add a keyword for it. But with a coroutine, ordinary call sites that wouldn't yield control with any other kind of function can yield control, so it makes sense to add a keyword to make that explicit.)
- pansa2 6y ago
  > What I mind is having to do "async def", "async for", "async with" all over the place for no good reason. IMO `async for` and `async with` are misnomers - they are statements that call asynchronous functions (`__anext__`, `__aenter__` and `__aexit__`) so would have been better named something like `for await` and `with await`. `async def`, on the other hand, was just copied cargo-cult-style from C#. I don't think it's necessarily bad, but you're right to say that it's inconsistent with Python's existing generators.
- BiteCode_dev 6y ago
  I have the opposition opinion. I think generators should have had an explicit keyword for defining them, like: gen def stuff() Because it doesn't, people are very confused about generators, and think they are functions, which they are not. They are completely different objects. Plus there is no way to tell if a definition is a generator without reading all of it. Explicit is better than implicit and all that.
olau 6y ago
Writing multi-threaded programs is much, much harder than single-threaded code. I think part of the attraction of async/await is that you can write mostly non-concurrent code. But then it gets extremely important that you're aware when you might have concurrency. For instance: x = get_something_from_global_state() y = foo() insert_into_global_state(x, y) If the program is suddenly possibly concurrent when calling foo(), then perhaps x is no longer valid. Good luck debugging that. await foo() makes it much more obvious. Now, of course, in scenarios where you don't really care about this property, e.g. if you're dealing with a database in another process where it is obvious that stuff may happen behind your back, or if you don't actually have any concurrency, I get that await is probably pretty annoying.
peterkelly 6y ago
If all you're ever going to do with the result of an async function is await it immediately after calling, then you don't need async functions because you're just doing synchronous programming with extra syntax. The reason there's a difference between foo() and await foo() is that the former gives you a future (aka Promise in JavaScript), while the latter gives you the result. You can then wait on multiple futures, so that things can happen concurrently. If you add type annotations to your code and use mypy (which you definitely should), it'll pick up the distinction between the two types - Awaitable[str] vs str - and give you an error if you try to pass the future object to a function that expects a string. The only case where I've run into problems is where I've called an async function that returns None, but have forgotten to await the result. In that case the function never actually begins execution.
BiteCode_dev 6y ago
> How do you know how to call that function? async def foo() is not a function and it doesn't have the same signature: the async is very explicit. It's a different primitive, and it's not used AT ALL for the same thing at a function. It's like writting: def foo(): yield "bar" and saying, 'how do you cal that function? How do you know how to call that function? It's called the same thing. It has the same signature.. It returns the same thing. One has one exit point, the other has several. This is "colored functions". You have to call the 2nd one like: next(iter(foo())) It's absurd' Well, no, it's not. It's a different tool. It's not a function, it's a generator. You don't have to care about async/await. In fact, you can live your entire life without using them. But it's a handy tool that allows you do: - delegate asynchronous behavior, like I/O, outside of your callable - make you match the syntaxic order with execution order despite said delegation - make it clear and explicit where you chose to do so, indentifying the context switching of your program Does it seems complicated ? It's because it is, and you probably don't need it. But if you need it, it's easier than the alternative of using callbacks, and less error prones that implicit asyncronicity.

jonahbenton 6y ago

I'll bite. Opinion follows, no need to downvote. async def is terrible, of the syntactic/semantic options it is the worst. Best would have been to just allow creation of a name of a coroutine-usable version of a function- def f: # normal f .... a = async(f) That allows existing functions to be turned into coroutines while not introducing the new syntactic pattern of a double word declaration.

takeda 6y ago
You still can do that using @asyncio.coroutine decorator for example @asyncio.coroutine def hello(): return "foo" Although not for long, they deprecated it in 3.8 and will be removed in 3.10. I wish they would leave it, since I was able to use them to write API that could be synchronous or not depending which constructor was invoked. I'm wondering if there are better ways to do it.
- deleted 6y ago
  [deleted]
oefrha 6y ago
You can write a trivial decorator to make any normal blocking function async-usable (i.e. your suggestion is possible today and only takes a few lines of code — not even using deprecated functionality, I mean), but that doesn’t magically make it async-capable if it’s not designed for async in the first place. If you’re arguing for generator-based coroutines, then (1) that’s not just any normal function (check inspect.isgeneratorfunction and inspect.iscoroutinefunction); (2) it was already explored and deprecated. Guess we can have another talk on why if you wish.
- jonahbenton 6y ago
  Sure, there are a few different issues here, and my response above was mostly off the cuff. I have had the pleasure of engaging with cooperative vs scheduled concurrency since the 1980s, in the early days of the Mac, and asynchronous IO since it started to enter the mainstream in the 1990s. I don't write enough python any more that this impacts me much; more that I remember finding out about it and being astonished. That's not to say- the async PEP is well written and one gets the mental model the author is promoting and the problems the PEP is trying to solve. I used tornado a bunch in its early days and it was and I'm sure still is a lovely piece of engineering. Nevertheless, among the reasons for my astonishment were: * there is a difference between cooperative concurrency and largely IO-scheduling driven asynchrony. There are a range of protocols that callers and callees engage in, a level of greater or lesser interest that the caller has on what and when the callee does it, and a variety of ways the callee can make what it did more or less eventually known to the caller. Cooperation and async capture two distinct common use cases in this rich space, yet the python implementation seems to improperly lump the two together. And the added keywords are crude and only provide partial expression of the range of options. For instance, deadlines are a top level consideration both in cooperation and in asynchrony, yet there is no syntax for specifying them. * there is a level of sophistication required for effective cooperative or asynchronous programming that, despite the rise of node and so forth, is simply not met by the bulk of programmers employing it, nor necessary in the programs they are writing. Elevating the syntax to first class as though it is something for most python programmers to consider- well, you can have a synchronous function, or an asynchronous function, pick one- is putting power tools in the hands of people who mostly don't know how to use them working on problems that entirely don't need them. Especially for python, this seems like the wrong thing to have done. * the doubled keywords pattern is ugly. Yes, if you are going to allow for cooperative behavior, decorating the affected syntactic entities is necessary. If it had to get added to the language I would much rather have seen new single tokens- adef and afor- for this purpose, maybe added to an "expert" module to be "imported" before use. With a whole new declarative keyword, one asks- what else can this be applied to? Why not have async variables or instance members? The syntax suggests itself and there are potential semantics. Forgive me, but I was shocked when I saw this and still think it was an immature and unnecessary addition to the language.