6 ms·

I'm extremely enthusiastic about `async/await` semantics, and in particular async iterators and the `for async` loop. It's astonishing to see how fast Python i

by inglor 11y ago

I'm extremely enthusiastic about `async/await` semantics, and in particular async iterators and the `for async` loop. It's astonishing to see how fast Python is moving in this direction. It is truly a powerful language for async computations now - surpassing in this ability both JavaScript and C# (at least for now). For example Python got `async with` but JS isn't even close (userland solutions like Bluebird's using exist) and C# is only starting to work on `IAsyncDisposable`

rayalez 11y ago
This is awesome! I want to ask: - Does it mean that it will be possible to create realtime apps with python? Will it be possible to use it instead of node.js? Do you think Django will implement that functionality? - Now that WebAssembly is coming - doesn't it mean that it could be possible to create both backend and frontend for realtime apps in python? That would be so cool....
- inglor 11y ago
  > Does it mean that it will be possible to create realtime apps with python? I'm not sure what you mean by realtime but usually real time means something else and requires more deterministic behavior. If you mean applications that use non-blocking IO to great extent then yes, but this depends a lot on the ecosystem to provide libraries for async file io, database io and so on. The Python language left good infrastructure to use though. > Now that WebAssembly is coming - doesn't it mean that it could be possible to create both backend and frontend for realtime apps in python? Not anymore than we could before with asm.js unfortunately as seen in http://repl.it/languages/python3 http://repl.it/languages/python3 the problem is wrapping and the library, I hope a company "picks up the glove" and makes "Python for the frontend", sharing Python code between backend and frontend could be amazing.
  - JustSomeNobody 11y ago
    People have started to hijack the word realtime. It's truly sad, because now it just muddies up people's conversations. So now people have to ask, is it the one that keeps planes in the air or the one that makes sure your email badge counter updates.
    - hamiltonkibbe 11y ago
      It is unfortunate. In most cases you can assume until proven otherwise that when someone says "realtime" they mean "really really soft soft-realtime" (sometimes referred to as "flaccidtime") rather than hard- or soft-realtime. Who needs planes when you have Javascript?
    - jkarneges 11y ago
      Yes, for better or for worse, "realtime" (and sometimes "near-realtime") are used in the web community to refer to push technology as opposed to time constraints.
    - Serow225 11y ago
      I wish there was a better alternative term for the web sense that I could promote, just to help un-hijack the word before it gets out of control. But I don't know of one that is likely to succeed. 'Responsive' makes sense to me but it already has another meaning in the web world. What about something like 'reactive' or 'on-the-fly'? People will never go for 'near real-time' because it sounds too weak in terms of its advertising value. Someone else below me suggested 'push', which is pretty decent but again it's an overloaded term... Thoughts or suggestions? :)
      - JustSomeNobody 11y ago
        I'm not much of a wordsmith, but I am leaning towards "forthcoming".
- bdarnell 11y ago
  It's always been possible to create realtime apps with Python. Callbacks have been available since the beginning, and frameworks like Tornado and Twisted have been doing something very similar to the new async/await functionality with generators since Python 2.5 (or maybe even older). The async/await keywords are a nice performance improvement and work in a few places that "yield" doesn't, but they're not adding fundamentally new capabilities.
- hamiltonkibbe 11y ago
  I don't know that it's possible to have a realtime application running in an interpreter (which makes no realtime guarantees as far as I know) running on top of a soft-at-best-realtime OS.
- Scarblac 11y ago
  As for Django, I think this is an exciting idea: https://gist.github.com/andrewgodwin/b3f826a879eb84a70625 https://gist.github.com/andrewgodwin/b3f826a879eb84a70625
- jerf 11y ago
  "Will it be possible to use it instead of node.js?" It's been possible to use Python instead of node.js since before node.js existed. And by "possible" I mean "tons of people have been shipping code", not just "it's theoretically possible but nobody does it". Node ported a well-established existing technique to Javascript, it didn't invent it.
epberry 11y ago
Me too. There will be less of a switch in mindset for me after working in JS for a while then switching to Python for the backend.
muraiki 11y ago
I admit to not fully understanding the async PEP (partially since I don't do much Python programming anymore). Are there any downsides to having to explicitly mark an async function using the "async def" syntax? I'm thinking of how in Go I can launch a goroutine with "go foo()", where foo can be any function and not one that happened to be declared explicitly as async. Or in Perl 6, I can "my $p100 = start { (1..Inf).grep(*.is-prime)[99] }; say await $p100;" This provides the similar blocking "await" keyword, but I'm still free to execute any arbitrary code inside the block denoted by "start". I have a feeling that these questions are both answered by whatever coroutine functionality existed prior to the introduction of "async def", but that's an area of Python that I have no experience with. I'd appreciate any clarification that could be offered. Thanks! Edit: re: the P6 example, it looks like it works somewhat like Python, where a "Future-like object" (from the PEP) is provided to await; in P6 start returns a Promise. However it looks like in Python I still need to wrap the enclosing function with "await def". https://www.python.org/dev/peps/pep-0492/#await-expression https://www.python.org/dev/peps/pep-0492/#await-expression
- rdtsc 11y ago
  > Are there any downsides to having to explicitly mark an async function using the "async def" syntax? Yes. One main one is splintering the library ecosystem. So some libraries are using Twisted, some are using base threading system, some run in Tornado, some will use async and so on. They are usually not mixable. One day in the distant future they will all support async, but that means updating all of them. Special markers ("async" waits points/objects) for IO co-routines tend to propagate vertically through all the API layers. If, say, a low level library you found is returning a Deferred/Promise/Future or generates values, then top level has to handle that as well. Then its parent also has to handle it. It is infecting the ecosystem with low levels details about how the IO needs special casing because of how the GIL works. Yes, it is more elegant in Go, Rust, Erlang, C#, Java etc. BTW Python has something like it based on the greenlet library (eventlet and gevent libraries are based on). Those monkeypatch system libraries so it manages to hide this complexity, but there are other costs, unsupported or untested side-effects being one.
  - mwcampbell 11y ago
rdtsc 11y ago
Python could do green threads (IO-base coroutines) for a long time -- see eventlet, gevent, even Twisted's inlineCallback pattern. There are probably others, there was even one based on "yields" as well.
- duaneb 11y ago
  Stackless python pairs well with green threads. Very, very well. I believe pypy based a lot of their initial work on the design of stackless python.
  - rdtsc 11y ago
    Yap, I've used and shipped code based on eventlet. And you are right PyPy supports that as well. I have never tested it using PyPy. Perhaps one day. For large concurrency cases I would probably look at Erlang/Elixir though...
Spiritus 11y ago
One thing I don't really get, why is the "async" keyword necessary at all? Couldn't just the mere presence of "await" in the function body implicitly make the function "async"?
- balls187 11y ago
  Self documenting code for one. I suppose you could follow the .NET Naming convention of appending "Async" to all routines.
- skrebbel 11y ago
  I guess the reason is that it changes the type of the return value. Compare: def moo(): return 5 async def oink(): return 5 moo returns 5, oink returns an awaitable. For a human, without the 'async' marker, you'd have to scan the entire function source to know. I guess also type checkers and documentation tools like to have an idea about the return type.
  - Spiritus 11y ago
    Ah right, that makes sense.
  - kevin_thibedeau 11y ago
    The presence of "yield" also changes the return type. No extra keywords needed for that.
- svieira 11y ago
  This is directly addressed in the PEP [1] - the short of it is, without the `async` modifier if you have two functions: def important(): await something() def something(): # ... stuff ... await something_else() # ... more stuff ... and you refactor `await something_else()` into a new method then you've changed the return type of `something` from `Awaitable[ReturnType]` to just `ReturnType` and `important` will break at run-time when it tries to `await something()`. [1]: https://www.python.org/dev/peps/pep-0492/#importance-of-async-keyword
Ultimatt 11y ago
C# has the reactive extensions API https://rx.codeplex.com/ https://rx.codeplex.com/ which is the inspiration for the Perl 6 concurrency model which is also quite advanced https://www.youtube.com/watch?v=JBHsdc0IVIg https://www.youtube.com/watch?v=JBHsdc0IVIg So not sure why you think Python is somehow unique in doing all this.

by inglor 11y ago

rayalez 11y ago

This is awesome! I want to ask: - Does it mean that it will be possible to create realtime apps with python? Will it be possible to use it instead of node.js? Do you think Django will implement that functionality? - Now that WebAssembly is coming - doesn't it mean that it could be possible to create both backend and frontend for realtime apps in python? That would be so cool....

inglor 11y ago
> Does it mean that it will be possible to create realtime apps with python? I'm not sure what you mean by realtime but usually real time means something else and requires more deterministic behavior. If you mean applications that use non-blocking IO to great extent then yes, but this depends a lot on the ecosystem to provide libraries for async file io, database io and so on. The Python language left good infrastructure to use though. > Now that WebAssembly is coming - doesn't it mean that it could be possible to create both backend and frontend for realtime apps in python? Not anymore than we could before with asm.js unfortunately as seen in http://repl.it/languages/python3 http://repl.it/languages/python3 the problem is wrapping and the library, I hope a company "picks up the glove" and makes "Python for the frontend", sharing Python code between backend and frontend could be amazing.
- JustSomeNobody 11y ago
  People have started to hijack the word realtime. It's truly sad, because now it just muddies up people's conversations. So now people have to ask, is it the one that keeps planes in the air or the one that makes sure your email badge counter updates.
  - hamiltonkibbe 11y ago
    It is unfortunate. In most cases you can assume until proven otherwise that when someone says "realtime" they mean "really really soft soft-realtime" (sometimes referred to as "flaccidtime") rather than hard- or soft-realtime. Who needs planes when you have Javascript?
  - jkarneges 11y ago
    Yes, for better or for worse, "realtime" (and sometimes "near-realtime") are used in the web community to refer to push technology as opposed to time constraints.
  - Serow225 11y ago
    I wish there was a better alternative term for the web sense that I could promote, just to help un-hijack the word before it gets out of control. But I don't know of one that is likely to succeed. 'Responsive' makes sense to me but it already has another meaning in the web world. What about something like 'reactive' or 'on-the-fly'? People will never go for 'near real-time' because it sounds too weak in terms of its advertising value. Someone else below me suggested 'push', which is pretty decent but again it's an overloaded term... Thoughts or suggestions? :)
    - JustSomeNobody 11y ago
      I'm not much of a wordsmith, but I am leaning towards "forthcoming".
bdarnell 11y ago
It's always been possible to create realtime apps with Python. Callbacks have been available since the beginning, and frameworks like Tornado and Twisted have been doing something very similar to the new async/await functionality with generators since Python 2.5 (or maybe even older). The async/await keywords are a nice performance improvement and work in a few places that "yield" doesn't, but they're not adding fundamentally new capabilities.
hamiltonkibbe 11y ago
I don't know that it's possible to have a realtime application running in an interpreter (which makes no realtime guarantees as far as I know) running on top of a soft-at-best-realtime OS.
Scarblac 11y ago
As for Django, I think this is an exciting idea: https://gist.github.com/andrewgodwin/b3f826a879eb84a70625 https://gist.github.com/andrewgodwin/b3f826a879eb84a70625
jerf 11y ago
"Will it be possible to use it instead of node.js?" It's been possible to use Python instead of node.js since before node.js existed. And by "possible" I mean "tons of people have been shipping code", not just "it's theoretically possible but nobody does it". Node ported a well-established existing technique to Javascript, it didn't invent it.

epberry 11y ago

Me too. There will be less of a switch in mindset for me after working in JS for a while then switching to Python for the backend.

muraiki 11y ago

I admit to not fully understanding the async PEP (partially since I don't do much Python programming anymore). Are there any downsides to having to explicitly mark an async function using the "async def" syntax? I'm thinking of how in Go I can launch a goroutine with "go foo()", where foo can be any function and not one that happened to be declared explicitly as async. Or in Perl 6, I can "my $p100 = start { (1..Inf).grep(*.is-prime)[99] }; say await $p100;" This provides the similar blocking "await" keyword, but I'm still free to execute any arbitrary code inside the block denoted by "start". I have a feeling that these questions are both answered by whatever coroutine functionality existed prior to the introduction of "async def", but that's an area of Python that I have no experience with. I'd appreciate any clarification that could be offered. Thanks! Edit: re: the P6 example, it looks like it works somewhat like Python, where a "Future-like object" (from the PEP) is provided to await; in P6 start returns a Promise. However it looks like in Python I still need to wrap the enclosing function with "await def". https://www.python.org/dev/peps/pep-0492/#await-expression https://www.python.org/dev/peps/pep-0492/#await-expression

rdtsc 11y ago
> Are there any downsides to having to explicitly mark an async function using the "async def" syntax? Yes. One main one is splintering the library ecosystem. So some libraries are using Twisted, some are using base threading system, some run in Tornado, some will use async and so on. They are usually not mixable. One day in the distant future they will all support async, but that means updating all of them. Special markers ("async" waits points/objects) for IO co-routines tend to propagate vertically through all the API layers. If, say, a low level library you found is returning a Deferred/Promise/Future or generates values, then top level has to handle that as well. Then its parent also has to handle it. It is infecting the ecosystem with low levels details about how the IO needs special casing because of how the GIL works. Yes, it is more elegant in Go, Rust, Erlang, C#, Java etc. BTW Python has something like it based on the greenlet library (eventlet and gevent libraries are based on). Those monkeypatch system libraries so it manages to hide this complexity, but there are other costs, unsupported or untested side-effects being one.
- mwcampbell 11y ago

rdtsc 11y ago

Python could do green threads (IO-base coroutines) for a long time -- see eventlet, gevent, even Twisted's inlineCallback pattern. There are probably others, there was even one based on "yields" as well.

duaneb 11y ago
Stackless python pairs well with green threads. Very, very well. I believe pypy based a lot of their initial work on the design of stackless python.
- rdtsc 11y ago
  Yap, I've used and shipped code based on eventlet. And you are right PyPy supports that as well. I have never tested it using PyPy. Perhaps one day. For large concurrency cases I would probably look at Erlang/Elixir though...

Spiritus 11y ago

One thing I don't really get, why is the "async" keyword necessary at all? Couldn't just the mere presence of "await" in the function body implicitly make the function "async"?

balls187 11y ago
Self documenting code for one. I suppose you could follow the .NET Naming convention of appending "Async" to all routines.
skrebbel 11y ago
I guess the reason is that it changes the type of the return value. Compare: def moo(): return 5 async def oink(): return 5 moo returns 5, oink returns an awaitable. For a human, without the 'async' marker, you'd have to scan the entire function source to know. I guess also type checkers and documentation tools like to have an idea about the return type.
- Spiritus 11y ago
  Ah right, that makes sense.
- kevin_thibedeau 11y ago
  The presence of "yield" also changes the return type. No extra keywords needed for that.
svieira 11y ago
This is directly addressed in the PEP [1] - the short of it is, without the `async` modifier if you have two functions: def important(): await something() def something(): # ... stuff ... await something_else() # ... more stuff ... and you refactor `await something_else()` into a new method then you've changed the return type of `something` from `Awaitable[ReturnType]` to just `ReturnType` and `important` will break at run-time when it tries to `await something()`. [1]: https://www.python.org/dev/peps/pep-0492/#importance-of-async-keyword

Ultimatt 11y ago

C# has the reactive extensions API https://rx.codeplex.com/ https://rx.codeplex.com/ which is the inspiration for the Perl 6 concurrency model which is also quite advanced https://www.youtube.com/watch?v=JBHsdc0IVIg https://www.youtube.com/watch?v=JBHsdc0IVIg So not sure why you think Python is somehow unique in doing all this.