4 ms·

Gevent was doing it right and async syntax was a huge mistake that fractioned community-contributed libraries into two incompatible camps with lots of unnecessa

Gevent was doing it right and async syntax was a huge mistake that fractioned community-contributed libraries into two incompatible camps with lots of unnecessary cloning happening at present moment. In high-level languages with virtual machines and/or garbage collectors, the runtime system should be solely responsible for scheduling green threads around IO entry points, all without special syntactic markers. GHC has it right (https://www.aosabook.org/en/posa/warp.html https://www.aosabook.org/en/posa/warp.html), Gevent was a right development with on-par async performance metrics (https://gist.github.com/rfyiamcool/41d4004b7fd46516d0b4f34f6065e93b https://gist.github.com/rfyiamcool/41d4004b7fd46516d0b4f34f6...), that had a standard synchronous coding style. It could be adopted into the core language and improved further without splitting the community.

divbzero 7y ago
I have run Python in its traditional synchronous form, using gevent, and with the more recent async/await syntax. I don’t hold this opinion strongly, but do lean towards async/await syntax for the sake of explicit is better than implicit [1]. Node.js which was asynchronous from the start also separates async from sync explicitly with, for example, distinct fs.readFile() and fs.readFileSync() functions [2]. [1]: https://www.python.org/dev/peps/pep-0020/ https://www.python.org/dev/peps/pep-0020/ "PEP 20" [2]: https://nodejs.org/dist/latest-v12.x/docs/api/fs.html#fs_fs_readfile_path_options_callback https://nodejs.org/dist/latest-v12.x/docs/api/fs.html#fs_fs_... "Node.js File System" (Edit: Commenting only on clarity of syntax. Those performance metrics are interesting and I’ve admittedly never hit a scale where the difference has a practical impact.)
- ghostwriter 7y ago
  > I don’t hold this opinion strongly, but do lean towards async/await syntax for sake of explicit is better than implicit I guess it's a question of where the line that defines "too implicit" should be drawn. I'm totally fine with implicit gevent yields, yet sometimes when I need to do heavy Python meta-programming, I wish things were more explicit around language semantics, namely everything around inheritance handling inside meta-classes (for instance, see the current implementation of enum.Enum).
int_19h 7y ago
What is the basis for that assertion? The "high-level VM with green threads" approach has been tried for a long time - most prominently, Java - and it just doesn't seem to stick. For Python especially, it is problematic because it is a glue language more often than not, and VM-specific green threads are not good for cross-language interop. When you have promises and async/await around them, at ABI level it can all be mapped to a simple callback, which any language that has C FFI can handle. When you have green threads, every language in the picture has to be aware of them - and god forbid you have two different VMs with different notions of green threads interacting.
- YawningAngel 7y ago
  It works nicely in Golang and Haskell. The main issue with Java and Python is that the core runtime developers, reasonably, do not wish to spend a lot of time developing equivalent systems.
  - int_19h 7y ago
    I can't speak for Haskell, but inadequate performance of C FFI in Go is routinely mentioned as the reason why the community is so reluctant to wrap existing C libraries, rather than reimplementing them from scratch in Go.
    - YawningAngel 7y ago
      To be completely honest, I don't know much about C interfaces or systems programming in general. Looking at benchmarks Go's FFI does indeed seem to perform pretty poorly. However, as a web dev, I find it works well for the concurrent programming tasks I find myself dealing with.
- ghostwriter 7y ago
  > What is the basis for that assertion? The fact it's implemented in a runtime system I use nowadays > For Python especially, it is problematic Shall we say it's a complex task instead of a problematic case? > The "high-level VM with green threads" approach has been tried for a long time - most prominently, Java - and it just doesn't seem to stick. afaik, it didn't stick because JNI related to green threads needed to be scalable on SMP, while the runtime implementation used a single thread, and then a decision was made to move to native threads, which doesn't necessarily indicate any inherent issues with the VM-managed green threads (and CPython specifically cannot utilise SMP with its threads anyways). At least, this was mentioned in https://www.microsoft.com/en-us/research/publication/extending-the-haskell-foreign-function-interface-with-concurrency/ https://www.microsoft.com/en-us/research/publication/extendi... (Section 7). > When you have promises and async/await around them, at ABI level it can all be mapped to a simple callback, which any language that has C FFI can handle. Why a VM wouldn't be able register those callbacks and bound them to a concrete OS thread when it knows that an FFI interop is going to happen? I don't see the point where explicit async/await is needed for it. It may require thread-safety markers (and that's what GHC's FFI interface has - https://wiki.haskell.org/GHC/Using_the_FFI#Improving_efficiency https://wiki.haskell.org/GHC/Using_the_FFI#Improving_efficie...), but that's not the story about the invasive async syntax we have in contemporary Python.

divbzero 7y ago

I have run Python in its traditional synchronous form, using gevent, and with the more recent async/await syntax. I don’t hold this opinion strongly, but do lean towards async/await syntax for the sake of explicit is better than implicit [1]. Node.js which was asynchronous from the start also separates async from sync explicitly with, for example, distinct fs.readFile() and fs.readFileSync() functions [2]. [1]: https://www.python.org/dev/peps/pep-0020/ https://www.python.org/dev/peps/pep-0020/ "PEP 20" [2]: https://nodejs.org/dist/latest-v12.x/docs/api/fs.html#fs_fs_readfile_path_options_callback https://nodejs.org/dist/latest-v12.x/docs/api/fs.html#fs_fs_... "Node.js File System" (Edit: Commenting only on clarity of syntax. Those performance metrics are interesting and I’ve admittedly never hit a scale where the difference has a practical impact.)

ghostwriter 7y ago
> I don’t hold this opinion strongly, but do lean towards async/await syntax for sake of explicit is better than implicit I guess it's a question of where the line that defines "too implicit" should be drawn. I'm totally fine with implicit gevent yields, yet sometimes when I need to do heavy Python meta-programming, I wish things were more explicit around language semantics, namely everything around inheritance handling inside meta-classes (for instance, see the current implementation of enum.Enum).

int_19h 7y ago

What is the basis for that assertion? The "high-level VM with green threads" approach has been tried for a long time - most prominently, Java - and it just doesn't seem to stick. For Python especially, it is problematic because it is a glue language more often than not, and VM-specific green threads are not good for cross-language interop. When you have promises and async/await around them, at ABI level it can all be mapped to a simple callback, which any language that has C FFI can handle. When you have green threads, every language in the picture has to be aware of them - and god forbid you have two different VMs with different notions of green threads interacting.

YawningAngel 7y ago
It works nicely in Golang and Haskell. The main issue with Java and Python is that the core runtime developers, reasonably, do not wish to spend a lot of time developing equivalent systems.
- int_19h 7y ago
  I can't speak for Haskell, but inadequate performance of C FFI in Go is routinely mentioned as the reason why the community is so reluctant to wrap existing C libraries, rather than reimplementing them from scratch in Go.
  - YawningAngel 7y ago
    To be completely honest, I don't know much about C interfaces or systems programming in general. Looking at benchmarks Go's FFI does indeed seem to perform pretty poorly. However, as a web dev, I find it works well for the concurrent programming tasks I find myself dealing with.
ghostwriter 7y ago
> What is the basis for that assertion? The fact it's implemented in a runtime system I use nowadays > For Python especially, it is problematic Shall we say it's a complex task instead of a problematic case? > The "high-level VM with green threads" approach has been tried for a long time - most prominently, Java - and it just doesn't seem to stick. afaik, it didn't stick because JNI related to green threads needed to be scalable on SMP, while the runtime implementation used a single thread, and then a decision was made to move to native threads, which doesn't necessarily indicate any inherent issues with the VM-managed green threads (and CPython specifically cannot utilise SMP with its threads anyways). At least, this was mentioned in https://www.microsoft.com/en-us/research/publication/extending-the-haskell-foreign-function-interface-with-concurrency/ https://www.microsoft.com/en-us/research/publication/extendi... (Section 7). > When you have promises and async/await around them, at ABI level it can all be mapped to a simple callback, which any language that has C FFI can handle. Why a VM wouldn't be able register those callbacks and bound them to a concrete OS thread when it knows that an FFI interop is going to happen? I don't see the point where explicit async/await is needed for it. It may require thread-safety markers (and that's what GHC's FFI interface has - https://wiki.haskell.org/GHC/Using_the_FFI#Improving_efficiency https://wiki.haskell.org/GHC/Using_the_FFI#Improving_efficie...), but that's not the story about the invasive async syntax we have in contemporary Python.