3 ms·

Show HN: Coseq – high order functions for JavaScript async/sync iterators

9 points by manchagnu 9y ago

kernelsanderz 9y ago
Great library. I really can't wait until async iterators hit the language. I think they're much more elegant than node.js streams, but I'd like to understand better how critical concepts like back pressure are implemented with these constructs.
- manchagnu 9y ago
  thanks! I am also excited about async iterators. Back pressure is a very interesting point. Thinking about it from a nodejs perspective, node streams already handle back pressure. Every call to iterator.next will cause reading from the stream, which should trigger flow of information. If the stream/observable constructs you are using don't have back pressure built in, I would probably wrap them with a construct that has back pressure support and let generators interact with that instead. Buffering certainly has it's own limitations if you can't really pause the data provider (stream/observable). Rxjs has some good information on how it handles back pressure for its observables. http://reactivex.io/documentation/operators/backpressure.html http://reactivex.io/documentation/operators/backpressure.htm.... If worse comes to worse, add logic for buffering in the generator itself. Definitely something I have been pondering and to consider when dealing with async iterators that interact with data providers with no back pressure handling built in.
  - kernelsanderz 9y ago
    Thanks for the link. From what I can read it seems that there needs to be some concept of pausing or stopping and resuming a stream. This is one of the outstanding issues with promises too. That there is no standard way to cancel them. I hope TC39 are considering this as part of the spec. It's less of an issue in the client, because you are less likely to encounter big firehoses of data. But I have had the issue with webrtc data processing, and some flow control / back pressure mechanisms would be awesome. We were able to just use node streams, but would be nice if there was an elegant solution for async iterators too. Node streams are good, but also add a lot of complexity. And debugging back-pressure issues can be very tricky in my experience, as they can manifest themselves in a variety of different ways.

9 points by manchagnu 9y ago

Great library. I really can't wait until async iterators hit the language. I think they're much more elegant than node.js streams, but I'd like to understand better how critical concepts like back pressure are implemented with these constructs.

manchagnu 9y ago
thanks! I am also excited about async iterators. Back pressure is a very interesting point. Thinking about it from a nodejs perspective, node streams already handle back pressure. Every call to iterator.next will cause reading from the stream, which should trigger flow of information. If the stream/observable constructs you are using don't have back pressure built in, I would probably wrap them with a construct that has back pressure support and let generators interact with that instead. Buffering certainly has it's own limitations if you can't really pause the data provider (stream/observable). Rxjs has some good information on how it handles back pressure for its observables. http://reactivex.io/documentation/operators/backpressure.html http://reactivex.io/documentation/operators/backpressure.htm.... If worse comes to worse, add logic for buffering in the generator itself. Definitely something I have been pondering and to consider when dealing with async iterators that interact with data providers with no back pressure handling built in.
- kernelsanderz 9y ago
  Thanks for the link. From what I can read it seems that there needs to be some concept of pausing or stopping and resuming a stream. This is one of the outstanding issues with promises too. That there is no standard way to cancel them. I hope TC39 are considering this as part of the spec. It's less of an issue in the client, because you are less likely to encounter big firehoses of data. But I have had the issue with webrtc data processing, and some flow control / back pressure mechanisms would be awesome. We were able to just use node streams, but would be nice if there was an elegant solution for async iterators too. Node streams are good, but also add a lot of complexity. And debugging back-pressure issues can be very tricky in my experience, as they can manifest themselves in a variety of different ways.