6 ms·

JavaScript iterator patterns

104 points by loige 8y ago

jondubois 8y ago
Iterators and generators are powerful and can make code much cleaner/more declarative. Especially when used with async/await. If anyone is interested, I recently released a real-time WebSocket library (Node.js and front end) which only uses Async Iterators to stream data (no event listener callbacks): https://hackernoon.com/getting-started-with-asyngular-bbe3dd1c716c https://hackernoon.com/getting-started-with-asyngular-bbe3dd... Feedback welcome.
- sephoric 8y ago
  As a cool example of the practicality of generators, my son and I wrote a generator in PICO-8 (Lua calls it coroutines) to animate moving the tiles in a 2048 game we made, using simple a for-loop to animate the tile's pixel coordinates, but yielding at the end of the loop body, so that we could let _update() and _draw() continue. This allowed the program to still be able to respond to key presses, which would "skip" to the end of an animation by fast-forwarding the coroutine until it was dead, and then create and start the next animation coroutine. I think when he saw that it was just a for-loop but it that kept "pausing" (yielding) at the end of each iteration, he finally understood generators.
anonytrary 8y ago
One thing I dislike about the iterator protocol is that it creates garbage around every value you're iterating over: let set = new Set([1,2,3]), it = set.values(), n; while(!(n = it.next()).done) console.log(n); // {value: 1, done: false} // {value: 2, done: false} // {value: 3, done: false} I found this strange considering javascript has symbols now. Symbols would be perfect for the iterator protocol: while((n = it.next()) !== Symbol.iteratorDone) console.log(n) // 1 // 2 // 3 Not that this is too much of an issue, I'm sure using yield/yield*/of will eventually optimize this.
- duckerude 8y ago
  That would turn Symbol.iteratorDone into a dangerous value, that can't safely be yielded from a generator, or included in an array, or passed to a variadic function. I don't know if that would be much of a problem in practice but it feels wrong instinctively.
  - tonyg 8y ago
    Exactly. `Symbol.iteratorDone` would be a kind of "null" value. The existing approach is equivalent to returning Either<a,b> in a language with labelled sums; it carefully distinguishes between the envelope and the payload, if you like.
    - anonytrary 8y ago
      These are all great points. I didn't immediately consider: [1,2, Symbol.iteratorDone, 3] which would definitely break on something like that. It would seem the only real way to fix this is with a distinct wrapper.
      - duckerude 8y ago
        Python's solution is to throw a StopIteration exception when the iterator runs out. That works out ok as long as you don't manually throw it in weird places, but a naive implementation could have high overhead because exceptions are expensive. PHP's iterators have separate methods for advancing to the next element, checking if there's a current element, and getting the current element. That's cumbersome, and requires keeping extra state, but it's abstracted over so it doesn't typically matter. I think it mimics a bizarre old way of iterating through arrays. There are multiple ways to signal the end. Javascript's is pretty clean.
        anonytrary 8y ago
        Oof, I'd definitely take JavaScript's wrapper objects over an exception. Iteration complete isn't something that strikes me as an error, it's just the end of the finite list you're iterating over! Does the program expect the list to never end or something? Errors should be reserved for the unexpected. A list ending is very expected, unless you're dealing with an infinite generator.
guest124678 8y ago
The next step I would have shown if I wrote this article, would have been to tell reader that JS generators as they are now, do not work with async / wait. As long as you have some simple Fibonacci interview question they look nice to show off, but when you want to use them for real like return some paginated data from server as on going iterator, then current JS generators cannot be used. This makes js generators limited for any real usage, given JS IO is async.
- daxterspeed 8y ago
  Both async generator functions and async iterators have been in the pipeline for a while. The proposed, and currently implemented syntax in Firefox and Chrome is `for await ... of`[0]. Creating an async generator function is as easy as `async function* functionName`. [0]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... Example of this syntax in practice: https://jsbin.com/folotu/edit?js,console https://jsbin.com/folotu/edit?js,console
  - loige 8y ago
    I have another article in mind to approach this topic. Also event emitters and streams might be other useful tools of the trade when you have to deal with asynchronous stuff.
  - haskman 8y ago
    You might be interested in a slightly unusual UI framework I wrote that heavily focuses on async generators to define the UI - https://github.com/ajnsit/concur-js https://github.com/ajnsit/concur-js. The idea is that a UI "Widget" is a sequence of UIs (which themselves can be Widgets) generated asynchronously (say when a UI event fires). A sample widget - async function*() { // Show a clickable button. // All processing is async, this will conceptually block until button is clicked. yield* <button onClick>Click me</button> // Button was clicked, show text yield* <div>Button was clicked!</div> }) It uses React and VDOM to diff the UIs generated and update the page slightly more efficiently. As you can see, it also provides JSX support (by overriding createElement calls). The Widgets themselves can be easily composed, and the entire thing is designed to be extremely easy to get started with. The README on github has more information.
nicknaso 8y ago
Good article thanks.
- loige 8y ago
  Thank you, Nick I am really glad you liked it!
sktrdie 8y ago
If anybody is interested in coding in a way that is closer to requirements, generators are really helpful to switch towards a paradigm called Behavioral Programming: https://lmatteis.github.io/react-behavioral/ https://lmatteis.github.io/react-behavioral/
- bfrydl 8y ago
  Forgive me if I'm missing something but this looks needlessly complex. Could you explain how this could be helpful, especially for developing React applications? The page doesn't really explain it besides insisting that this is closer to how we think, but I don't agree at all.
  - sktrdie 8y ago
    One of the things it helps with is the idea that we can modify the behavior without having to actually see how old code has been implemented. All one needs is a trace of events (a particular behavior) and can decide with newly added code how to modify that trace to fit the new requirements (by blocking specific traces and have others happen instead). This is an extremely powerful way of programming because it more easily fits how we develop apps: requirements constantly change. Currently when behavior needs to be modified we are stuck with having to modify and understand old code which can be very tedious (in my opinion it's a crucial pain-point in software development). Behavioral Programming makes it easier to modify a system without having to understand how it was built, but by observing a particular behavior. Once observed, the behavior can be modified, removed or completely changed incrementally, without having to go back and refactor old code. Sorry if it's very abstract. I gave a talk about it in case it helps: https://www.youtube.com/watch?v=_BLQIE-_prc https://www.youtube.com/watch?v=_BLQIE-_prc
  - haskman 8y ago
    I referenced my framework further down the thread as well, but repeating it here since you specifically ask for ease-of-use and React. Please try out Concur-JS which is my async-gen based UI framework that is React based, and designed to be extremely easy to get started with. https://github.com/ajnsit/concur-js https://github.com/ajnsit/concur-js
_xgw 8y ago
> Consecutive calls to next() will always produce { done: true }. Not necessarily. You can make recursive generators which never "terminate". For example, here's a codegolfed version of a recursive generator which represents a n + 1 sequence: p=function*a(x){yield x;yield*a(x+1)}(0)
- loige 8y ago
  I meant there "[Once the generator has completed], Consecutive calls to `next()` will always produce `{ done: true }`". I should probably make it more explicit PS: I like your n+1 example.

104 points by loige 8y ago

Iterators and generators are powerful and can make code much cleaner/more declarative. Especially when used with async/await. If anyone is interested, I recently released a real-time WebSocket library (Node.js and front end) which only uses Async Iterators to stream data (no event listener callbacks): https://hackernoon.com/getting-started-with-asyngular-bbe3dd1c716c https://hackernoon.com/getting-started-with-asyngular-bbe3dd... Feedback welcome.

sephoric 8y ago
As a cool example of the practicality of generators, my son and I wrote a generator in PICO-8 (Lua calls it coroutines) to animate moving the tiles in a 2048 game we made, using simple a for-loop to animate the tile's pixel coordinates, but yielding at the end of the loop body, so that we could let _update() and _draw() continue. This allowed the program to still be able to respond to key presses, which would "skip" to the end of an animation by fast-forwarding the coroutine until it was dead, and then create and start the next animation coroutine. I think when he saw that it was just a for-loop but it that kept "pausing" (yielding) at the end of each iteration, he finally understood generators.

anonytrary 8y ago

One thing I dislike about the iterator protocol is that it creates garbage around every value you're iterating over: let set = new Set([1,2,3]), it = set.values(), n; while(!(n = it.next()).done) console.log(n); // {value: 1, done: false} // {value: 2, done: false} // {value: 3, done: false} I found this strange considering javascript has symbols now. Symbols would be perfect for the iterator protocol: while((n = it.next()) !== Symbol.iteratorDone) console.log(n) // 1 // 2 // 3 Not that this is too much of an issue, I'm sure using yield/yield*/of will eventually optimize this.

duckerude 8y ago
That would turn Symbol.iteratorDone into a dangerous value, that can't safely be yielded from a generator, or included in an array, or passed to a variadic function. I don't know if that would be much of a problem in practice but it feels wrong instinctively.
- tonyg 8y ago
  Exactly. `Symbol.iteratorDone` would be a kind of "null" value. The existing approach is equivalent to returning Either<a,b> in a language with labelled sums; it carefully distinguishes between the envelope and the payload, if you like.
  - anonytrary 8y ago
    These are all great points. I didn't immediately consider: [1,2, Symbol.iteratorDone, 3] which would definitely break on something like that. It would seem the only real way to fix this is with a distinct wrapper.
    - duckerude 8y ago
      Python's solution is to throw a StopIteration exception when the iterator runs out. That works out ok as long as you don't manually throw it in weird places, but a naive implementation could have high overhead because exceptions are expensive. PHP's iterators have separate methods for advancing to the next element, checking if there's a current element, and getting the current element. That's cumbersome, and requires keeping extra state, but it's abstracted over so it doesn't typically matter. I think it mimics a bizarre old way of iterating through arrays. There are multiple ways to signal the end. Javascript's is pretty clean.
      - anonytrary 8y ago
        Oof, I'd definitely take JavaScript's wrapper objects over an exception. Iteration complete isn't something that strikes me as an error, it's just the end of the finite list you're iterating over! Does the program expect the list to never end or something? Errors should be reserved for the unexpected. A list ending is very expected, unless you're dealing with an infinite generator.

guest124678 8y ago

The next step I would have shown if I wrote this article, would have been to tell reader that JS generators as they are now, do not work with async / wait. As long as you have some simple Fibonacci interview question they look nice to show off, but when you want to use them for real like return some paginated data from server as on going iterator, then current JS generators cannot be used. This makes js generators limited for any real usage, given JS IO is async.

daxterspeed 8y ago
Both async generator functions and async iterators have been in the pipeline for a while. The proposed, and currently implemented syntax in Firefox and Chrome is `for await ... of`[0]. Creating an async generator function is as easy as `async function* functionName`. [0]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... Example of this syntax in practice: https://jsbin.com/folotu/edit?js,console https://jsbin.com/folotu/edit?js,console
- loige 8y ago
  I have another article in mind to approach this topic. Also event emitters and streams might be other useful tools of the trade when you have to deal with asynchronous stuff.
- haskman 8y ago
  You might be interested in a slightly unusual UI framework I wrote that heavily focuses on async generators to define the UI - https://github.com/ajnsit/concur-js https://github.com/ajnsit/concur-js. The idea is that a UI "Widget" is a sequence of UIs (which themselves can be Widgets) generated asynchronously (say when a UI event fires). A sample widget - async function*() { // Show a clickable button. // All processing is async, this will conceptually block until button is clicked. yield* <button onClick>Click me</button> // Button was clicked, show text yield* <div>Button was clicked!</div> }) It uses React and VDOM to diff the UIs generated and update the page slightly more efficiently. As you can see, it also provides JSX support (by overriding createElement calls). The Widgets themselves can be easily composed, and the entire thing is designed to be extremely easy to get started with. The README on github has more information.

nicknaso 8y ago

Good article thanks.

loige 8y ago
Thank you, Nick I am really glad you liked it!

sktrdie 8y ago

If anybody is interested in coding in a way that is closer to requirements, generators are really helpful to switch towards a paradigm called Behavioral Programming: https://lmatteis.github.io/react-behavioral/ https://lmatteis.github.io/react-behavioral/

bfrydl 8y ago
Forgive me if I'm missing something but this looks needlessly complex. Could you explain how this could be helpful, especially for developing React applications? The page doesn't really explain it besides insisting that this is closer to how we think, but I don't agree at all.
- sktrdie 8y ago
  One of the things it helps with is the idea that we can modify the behavior without having to actually see how old code has been implemented. All one needs is a trace of events (a particular behavior) and can decide with newly added code how to modify that trace to fit the new requirements (by blocking specific traces and have others happen instead). This is an extremely powerful way of programming because it more easily fits how we develop apps: requirements constantly change. Currently when behavior needs to be modified we are stuck with having to modify and understand old code which can be very tedious (in my opinion it's a crucial pain-point in software development). Behavioral Programming makes it easier to modify a system without having to understand how it was built, but by observing a particular behavior. Once observed, the behavior can be modified, removed or completely changed incrementally, without having to go back and refactor old code. Sorry if it's very abstract. I gave a talk about it in case it helps: https://www.youtube.com/watch?v=_BLQIE-_prc https://www.youtube.com/watch?v=_BLQIE-_prc
- haskman 8y ago
  I referenced my framework further down the thread as well, but repeating it here since you specifically ask for ease-of-use and React. Please try out Concur-JS which is my async-gen based UI framework that is React based, and designed to be extremely easy to get started with. https://github.com/ajnsit/concur-js https://github.com/ajnsit/concur-js

_xgw 8y ago

> Consecutive calls to next() will always produce { done: true }. Not necessarily. You can make recursive generators which never "terminate". For example, here's a codegolfed version of a recursive generator which represents a n + 1 sequence: p=function*a(x){yield x;yield*a(x+1)}(0)

loige 8y ago
I meant there "[Once the generator has completed], Consecutive calls to `next()` will always produce `{ done: true }`". I should probably make it more explicit PS: I like your n+1 example.