4 ms·

Tokio author here. Generally speaking, I recommend strongly against using FuturesUnordered unless you know all the pitfalls. We are working on an alternative ut

Tokio author here. Generally speaking, I recommend strongly against using FuturesUnordered unless you know all the pitfalls. We are working on an alternative utility that should hopefully avoid the issues described here and others: https://github.com/tokio-rs/tokio/pull/4335 https://github.com/tokio-rs/tokio/pull/4335

psarna 5y ago
That's great news! Especially that the observed performance of the test program based on FuturesUnordered, even though it stopped being quadratic, it was still considerably slower than the task::unconstrained one, which suggests there's room for improvement. Probably due to the fact that you still pay with a constant number of polls (32) each time you go out of budget.
- carllerche 5y ago
  IMO FuturesUnordered should stop executing futures when it sees a "yield". An explicit yield signals control should be returned to the runtime. FuturesUnordered does not respect this.
  - dathinab 5y ago
    IMHO the main problem is tokio introducing preempting behaviour which in subtle ways can mess with all kinds of normally fully valid rust futures. Sure it sometimes magically fixes problematic code, but it's in effect still a magic extension to how rust polling works which can have surprising side effects. In a certain way tokios coperative-preempting is not adequate to handle any future which multiplexes multiple futures but such futures are a stable of rust async since the get to go.
    - carllerche 5y ago
      I don't think it is an issue w/ the pre-emption code. I believe FuturesUnordered is just doing the wrong thing: not respecting yields.
      - dathinab 5y ago
        > wrong thing: not respecting yields This is not quite right. As far as I understand they did respect yield in the way it's defined by rust. There yields mean just that your future returns `Pending`. Which means only _this_ future is not ready. But in tokio returning `Pending` means "not-ready" and "maybe as a _magic side effect_ also make all other futures return pending even if it is ready internally". So lets step away from FuturesUnordered for a moment and instead just look at a future which multiplexes X-futures and polls each (not completed one) once and then yields, which should be 100% fine. But with tokio it isn't as after just polling the first few, the "budget" might be consumed and polling all other will forcefully fail where it shouldn't, adding a lot of overhead. Worse if you just poll them in the same order every time you will de-facto starve all futures later in the list. Which means you need to add a bunch of complexity which should be unnecessary just because tokio changed what `Poll::Pending` means. Also if you write scheduler independent futures (you should if you can) then you can not opt-out of it. Generally in a multiplexed future you don't want to opt. out anyway, you want to have a budged per logical strange of execution, which tokio doesn't provide. Instead tokio assumes a task (which in the end is just a future polled by it) represents a single logical strange of execution. But that is simple NOT how rust futures are designed!! (Through often it happens to play out that way.) This doesn't mean that tokios idea is bad, it's just not compatible with rusts future design in subtle edge cases. I think it would be a nice thing to add it to the future design, but then you would need a standardized way to properly handle multiplexing. (Which as a side not tokio doesn't provide, it only provides opt. out `unconstrained`, but what you need is to snapshot and restore the budget or something similar, i.e. snapshot it when entering the multiplexer and restoring it after each multiplexed future or similar, the only solution futures unordered could take is speculative adding yields, but that's _not_ a proper solution as it subtle de-prioritize multiplexed futures compared to spawned futures and also can easily fall apart if you nest multiplexed futures....). Also I have no idea why they call it cooperative scheduling. Futures are cooperative scheduling. What they do is preemting futures (i.e. force full yield them), but only in places where they could have yielded in context of cooperative scheduling. So it's some in between solution.
      - psarna 5y ago
        wrt. preemption - in Seastar we have maybe_yield(), which gives up the cpu, but only if the task quota (more or less a semantic equivalent for Tokio's budget) has passed. Wouldn't it make sense to have a similar capability in Tokio? Then, if somebody is not a big fan of the default preemption, they could run their tasks under tokio::task::unconstrained and only check the budget in very specific, explicitly chosen places - where they call maybe_yield(). That could of course also be open-coded by trying to implement maybe_yield on top of yield_now and some time measurements, but since the whole budgeting code is already there... Do you think it's feasible?

psarna 5y ago

That's great news! Especially that the observed performance of the test program based on FuturesUnordered, even though it stopped being quadratic, it was still considerably slower than the task::unconstrained one, which suggests there's room for improvement. Probably due to the fact that you still pay with a constant number of polls (32) each time you go out of budget.

carllerche 5y ago
IMO FuturesUnordered should stop executing futures when it sees a "yield". An explicit yield signals control should be returned to the runtime. FuturesUnordered does not respect this.
- dathinab 5y ago
  IMHO the main problem is tokio introducing preempting behaviour which in subtle ways can mess with all kinds of normally fully valid rust futures. Sure it sometimes magically fixes problematic code, but it's in effect still a magic extension to how rust polling works which can have surprising side effects. In a certain way tokios coperative-preempting is not adequate to handle any future which multiplexes multiple futures but such futures are a stable of rust async since the get to go.
  - carllerche 5y ago
    I don't think it is an issue w/ the pre-emption code. I believe FuturesUnordered is just doing the wrong thing: not respecting yields.
    - dathinab 5y ago
      > wrong thing: not respecting yields This is not quite right. As far as I understand they did respect yield in the way it's defined by rust. There yields mean just that your future returns `Pending`. Which means only _this_ future is not ready. But in tokio returning `Pending` means "not-ready" and "maybe as a _magic side effect_ also make all other futures return pending even if it is ready internally". So lets step away from FuturesUnordered for a moment and instead just look at a future which multiplexes X-futures and polls each (not completed one) once and then yields, which should be 100% fine. But with tokio it isn't as after just polling the first few, the "budget" might be consumed and polling all other will forcefully fail where it shouldn't, adding a lot of overhead. Worse if you just poll them in the same order every time you will de-facto starve all futures later in the list. Which means you need to add a bunch of complexity which should be unnecessary just because tokio changed what `Poll::Pending` means. Also if you write scheduler independent futures (you should if you can) then you can not opt-out of it. Generally in a multiplexed future you don't want to opt. out anyway, you want to have a budged per logical strange of execution, which tokio doesn't provide. Instead tokio assumes a task (which in the end is just a future polled by it) represents a single logical strange of execution. But that is simple NOT how rust futures are designed!! (Through often it happens to play out that way.) This doesn't mean that tokios idea is bad, it's just not compatible with rusts future design in subtle edge cases. I think it would be a nice thing to add it to the future design, but then you would need a standardized way to properly handle multiplexing. (Which as a side not tokio doesn't provide, it only provides opt. out `unconstrained`, but what you need is to snapshot and restore the budget or something similar, i.e. snapshot it when entering the multiplexer and restoring it after each multiplexed future or similar, the only solution futures unordered could take is speculative adding yields, but that's _not_ a proper solution as it subtle de-prioritize multiplexed futures compared to spawned futures and also can easily fall apart if you nest multiplexed futures....). Also I have no idea why they call it cooperative scheduling. Futures are cooperative scheduling. What they do is preemting futures (i.e. force full yield them), but only in places where they could have yielded in context of cooperative scheduling. So it's some in between solution.
    - psarna 5y ago
      wrt. preemption - in Seastar we have maybe_yield(), which gives up the cpu, but only if the task quota (more or less a semantic equivalent for Tokio's budget) has passed. Wouldn't it make sense to have a similar capability in Tokio? Then, if somebody is not a big fan of the default preemption, they could run their tasks under tokio::task::unconstrained and only check the budget in very specific, explicitly chosen places - where they call maybe_yield(). That could of course also be open-coded by trying to implement maybe_yield on top of yield_now and some time measurements, but since the whole budgeting code is already there... Do you think it's feasible?