3 ms·

The truth is you need almost everything to do push and pull. Just as pull alone does "forks" inefficiently, push alone does "joins" inefficiently. People shy a

The truth is you need almost everything to do push and pull. Just as pull alone does "forks" inefficiently, push alone does "joins" inefficiently. People shy away from this conclusion because doing everything both ways is quite hard and requires fancier abstractions to make nice than people are familiar with.

scott0129 6y ago
> The truth is you need almost everything to do push and pull. Just as pull alone does "forks" inefficiently, push alone does "joins" inefficiently. That's really is a simple and intuitive summary. Thank you for that insight!
- Ericson2314 6y ago
  Thanks, glad to share it!
Quekid5 6y ago
People have been vacillating on this forever and the truth is... (drum roll) It depends.
kjeetgill 6y ago
Mind explaining the fork vs joins inefficiency thing a bit further? I feel like there's some wisdom here, but I don't think I grocked it
- Ericson2314 6y ago
  With forks (in the article DAGs vs trees), pure pulling means you pull the same thing twice. With joins, pushing only works if both sides push at the same time. In general, joins work by a push on one side triggering a pull on the other side, and then both are combined. For forks there should be some subscribe mechanism to turn pulls into pushes that can cheaply fan out to all subscribers. (Side note, the mitigations ought to be completely symmetric. This is a sign there is more to the problem that I am missing.)
  - quantified 6y ago
    I believe joins work without simultaneous pushing. One side can build first, the other side can probe in a pushed fashion after but not necessarily simultaneously. “Trigger” is not necessarily the relationship, between build and probe sides though it could be. The DAG paths that produce/push the probe can be triggered by the completion of the build. “Happens after” is more strictly apt.
jadbox 6y ago
This is exactly why data patterns like Observables (https://github.com/tc39/proposal-observable https://github.com/tc39/proposal-observable) and ReactiveX (http://reactivex.io/ http://reactivex.io/) are so powerful. You can design your interface to be subjective to both. :)
- btown 6y ago
  It's a bit more complicated than this, because Observables in both frameworks are only required to implement push-style interfaces (that's the public interface of event listeners by design), and if a consumer wishes to pull at its own rate from an arbitrary source, the only general solution is to create an infinite buffer that materializes the pusher's outputs, which is the exact performance tradeoff described in the (excellent) OP article. http://reactivex.io/documentation/operators/backpressure.html http://reactivex.io/documentation/operators/backpressure.htm... describes ReactiveX's approach to the problem in great detail. To quote the section that essentially reaches the same conclusion as the grandparent post pointing out how "forks" (using the same pull-style data source twice) inherently require buffering/materialization or sampling: > When a cold Observable is multicast (when it is converted into a connectable Observable and its Connect method is called), it effectively becomes hot and for the purposes of backpressure and flow-control it should be treated as a hot Observable. Cold Observables are ideal for the reactive pull model of backpressure implemented by some implementations of ReactiveX (which is described elsewhere). Hot Observables typically do not cope well with a reactive pull model, and are better candidates for other flow control strategies, such as the use of the operators described on this page, or operators like Buffer, Sample, Debounce, or Window. There are great visuals if you expand the RxJS examples in the link above.

scott0129 6y ago

> The truth is you need almost everything to do push and pull. Just as pull alone does "forks" inefficiently, push alone does "joins" inefficiently. That's really is a simple and intuitive summary. Thank you for that insight!

Ericson2314 6y ago
Thanks, glad to share it!

Quekid5 6y ago

People have been vacillating on this forever and the truth is... (drum roll) It depends.

kjeetgill 6y ago

Mind explaining the fork vs joins inefficiency thing a bit further? I feel like there's some wisdom here, but I don't think I grocked it

Ericson2314 6y ago
With forks (in the article DAGs vs trees), pure pulling means you pull the same thing twice. With joins, pushing only works if both sides push at the same time. In general, joins work by a push on one side triggering a pull on the other side, and then both are combined. For forks there should be some subscribe mechanism to turn pulls into pushes that can cheaply fan out to all subscribers. (Side note, the mitigations ought to be completely symmetric. This is a sign there is more to the problem that I am missing.)
- quantified 6y ago
  I believe joins work without simultaneous pushing. One side can build first, the other side can probe in a pushed fashion after but not necessarily simultaneously. “Trigger” is not necessarily the relationship, between build and probe sides though it could be. The DAG paths that produce/push the probe can be triggered by the completion of the build. “Happens after” is more strictly apt.

jadbox 6y ago

This is exactly why data patterns like Observables (https://github.com/tc39/proposal-observable https://github.com/tc39/proposal-observable) and ReactiveX (http://reactivex.io/ http://reactivex.io/) are so powerful. You can design your interface to be subjective to both. :)

btown 6y ago
It's a bit more complicated than this, because Observables in both frameworks are only required to implement push-style interfaces (that's the public interface of event listeners by design), and if a consumer wishes to pull at its own rate from an arbitrary source, the only general solution is to create an infinite buffer that materializes the pusher's outputs, which is the exact performance tradeoff described in the (excellent) OP article. http://reactivex.io/documentation/operators/backpressure.html http://reactivex.io/documentation/operators/backpressure.htm... describes ReactiveX's approach to the problem in great detail. To quote the section that essentially reaches the same conclusion as the grandparent post pointing out how "forks" (using the same pull-style data source twice) inherently require buffering/materialization or sampling: > When a cold Observable is multicast (when it is converted into a connectable Observable and its Connect method is called), it effectively becomes hot and for the purposes of backpressure and flow-control it should be treated as a hot Observable. Cold Observables are ideal for the reactive pull model of backpressure implemented by some implementations of ReactiveX (which is described elsewhere). Hot Observables typically do not cope well with a reactive pull model, and are better candidates for other flow control strategies, such as the use of the operators described on this page, or operators like Buffer, Sample, Debounce, or Window. There are great visuals if you expand the RxJS examples in the link above.