3 ms·

Could someone familiar with the Fiber/Async thing explain how does this differ from Python's (or Node's)? How come Python (and Node) needs explicit awaits but R

by pas 6y ago

Could someone familiar with the Fiber/Async thing explain how does this differ from Python's (or Node's)? How come Python (and Node) needs explicit awaits but Ruby seems to "just work"?

burlesona 6y ago
Not an implementer but I would say those are just blocking method calls, like Thread.join. No magic happening.
__s 6y ago
Python Zen: Explicit is better than implicit There's a python library gevent which experimented with a more implicit design. Or more, this is about green threads vs async/await It can also be a bit confusing when you don't allow separating await from the call: how do you implement Promise.all? For official rationale see https://www.python.org/dev/peps/pep-0492/#why-async-and-await-keywords https://www.python.org/dev/peps/pep-0492/#why-async-and-awai...
- ioquatix 6y ago
  async/await is a huge anti-pattern IMHO.
grncdr 6y ago
From an implementation perspective this has much more in common with Go than either Python or Node. In Go, you simply write a blocking IO call, and the scheduler in the Go runtime pauses your current lightweight thread ("parks the go routine" in their terminology) until the IO operation has some results to read. While waiting, the runtime is free to run any other go routine(s) on any number of OS threads. What is coming to Ruby is a similar system, but where a library can provide this scheduler functionality. So when a lightweight thread (a "Fiber" in Rubys terms) calls a blocking operation, the runtime is notified, and can manage the IO operation while running other fibers. As another commenter mentioned, there was a similar thing named gevent implemented for Python some years ago, but it didn't become "the way" to do things. I feel there are three main reasons it didn't catch on: 1. A lot of Python programs had blocking IO being performed inside native extensions (e.g. bindings to C/C++ database drivers). If you depended on such a library, the benefits of gevent were severely limited because it's runtime couldn't intercept the IO operations. 2. It was never part of the stdlib, so there was not a huge amount of incentive for the aforementioned driver authors to support it explicitly. 3. There were not that many examples of successful implementation of this model at the time. Go was around but pre 1.0 and not nearly as popular as it is now. Lua was also around, but again, not extremely well known. I expect that Ruby will also suffer a bit from #1, but hopefully, by providing this concept as part of the language and standard library, problem #2 will be mitigated. #3 is IMO a thing of the past, while I'm not a huge Go fan, it does serve as a compelling argument for an implementation of runtime-managed lightweight threads. Edit: I should mention I'm in no way involved in this, just interested in what will come of it over the next few years.
- nerdponx 6y ago
  So the Ruby runtime has some kind of internal list of operations that are blocking, and can optionally be executed asynchronously when called from inside a fiber?
  - ioquatix 6y ago
    Well, it's not a list. It's just a set of hooks. But that's basically it.
lloeki 6y ago
Fibers are more like Python green threads, or generators when you abuse them to do coroutines, where you explicitly yield from one to the other. The thing is, if your Fiber calls a blocking IO operation you can’t explicitly yield to another Fiber unless IO unblocks. The scheduler solution attempts to yield to another Fiber when IO blocks. Ractors resemble Node workers: you get true parallelism, but the two worlds are entirely isolated, sharing nothing. You communicate by passing messages between both, where message content is either copied or moved (ownership change) if it’s not immutable. (What I described is not 100% accurate, more of a flawed bird’s eye analogy for Ruby foreigners)

by pas 6y ago

Could someone familiar with the Fiber/Async thing explain how does this differ from Python's (or Node's)? How come Python (and Node) needs explicit awaits but Ruby seems to "just work"?

burlesona 6y ago

Not an implementer but I would say those are just blocking method calls, like Thread.join. No magic happening.

__s 6y ago

Python Zen: Explicit is better than implicit There's a python library gevent which experimented with a more implicit design. Or more, this is about green threads vs async/await It can also be a bit confusing when you don't allow separating await from the call: how do you implement Promise.all? For official rationale see https://www.python.org/dev/peps/pep-0492/#why-async-and-await-keywords https://www.python.org/dev/peps/pep-0492/#why-async-and-awai...

ioquatix 6y ago
async/await is a huge anti-pattern IMHO.

grncdr 6y ago

From an implementation perspective this has much more in common with Go than either Python or Node. In Go, you simply write a blocking IO call, and the scheduler in the Go runtime pauses your current lightweight thread ("parks the go routine" in their terminology) until the IO operation has some results to read. While waiting, the runtime is free to run any other go routine(s) on any number of OS threads. What is coming to Ruby is a similar system, but where a library can provide this scheduler functionality. So when a lightweight thread (a "Fiber" in Rubys terms) calls a blocking operation, the runtime is notified, and can manage the IO operation while running other fibers. As another commenter mentioned, there was a similar thing named gevent implemented for Python some years ago, but it didn't become "the way" to do things. I feel there are three main reasons it didn't catch on: 1. A lot of Python programs had blocking IO being performed inside native extensions (e.g. bindings to C/C++ database drivers). If you depended on such a library, the benefits of gevent were severely limited because it's runtime couldn't intercept the IO operations. 2. It was never part of the stdlib, so there was not a huge amount of incentive for the aforementioned driver authors to support it explicitly. 3. There were not that many examples of successful implementation of this model at the time. Go was around but pre 1.0 and not nearly as popular as it is now. Lua was also around, but again, not extremely well known. I expect that Ruby will also suffer a bit from #1, but hopefully, by providing this concept as part of the language and standard library, problem #2 will be mitigated. #3 is IMO a thing of the past, while I'm not a huge Go fan, it does serve as a compelling argument for an implementation of runtime-managed lightweight threads. Edit: I should mention I'm in no way involved in this, just interested in what will come of it over the next few years.

nerdponx 6y ago
So the Ruby runtime has some kind of internal list of operations that are blocking, and can optionally be executed asynchronously when called from inside a fiber?
- ioquatix 6y ago
  Well, it's not a list. It's just a set of hooks. But that's basically it.

lloeki 6y ago

Fibers are more like Python green threads, or generators when you abuse them to do coroutines, where you explicitly yield from one to the other. The thing is, if your Fiber calls a blocking IO operation you can’t explicitly yield to another Fiber unless IO unblocks. The scheduler solution attempts to yield to another Fiber when IO blocks. Ractors resemble Node workers: you get true parallelism, but the two worlds are entirely isolated, sharing nothing. You communicate by passing messages between both, where message content is either copied or moved (ownership change) if it’s not immutable. (What I described is not 100% accurate, more of a flawed bird’s eye analogy for Ruby foreigners)