5 ms·

What about generators? I've built coroutines based on generators. CSP has been implemented in JS using generators. Not sure why those projects went thru "stagge

What about generators? I've built coroutines based on generators. CSP has been implemented in JS using generators. Not sure why those projects went thru "staggering amount of overhead" to get what we get for free with generators. Please educate.

1propionyl 10y ago
Because generators may be implemented with delineated continuations but they are not themselves the same thing.
- ilostmykeys 10y ago
  Are you sure it's not because those frameworks/libraries were written before generators became available in the latest browsers? :)
- morenoh149 10y ago
  that's Delimited continuation
jpolitz 10y ago
The main difference is that all of the use cases I mentioned necessarily don't distinguish between calls/functions that may pause, and calls that don't (it's just the semantics of those languages that arbitrary calls might need to pause). So to use generators as a compilation target, every function has to be a generator, and every call a generator instantiation followed by yield*. I actually don't know if that qualifies as "staggering". Your sibling comment has some truth; I can only speak generally about Gopherjs and Doppio, because I know them less intimately, but I know that Pyret and Whalesong were definitely started before generators had widespread adoption. Compiling to generators, rather than to the handwritten stack unwinding we have, is on my list of things to try and measure. Maybe "significant" overhead would be more obviously true than "staggering," since I don't have clear numbers to back it up. Does that make sense?
- ilostmykeys 10y ago
  <<So to use generators as a compilation target, every function has to be a generator>> Sure, but in those cases the functions are transpiled to JS so it doesn't really matter, not like they're coded by hand, right...? I think I understand what you mean.
  - jpolitz 10y ago
    I don't know what you mean by "it doesn't really matter." Surely there's a cost to using generators instead of regular function calls and returns, right? That's where the overhead would come from, because generators aren't free. Right now, Pyret and GopherJS (the last time I checked in GopherJS's case) basically manually encode the `IteratorResult` type, and check for "stack unwind" vs "regular result" when each function call returns, if it might pause. The first question for generators is if they are less overhead than this manual process. There's a bunch of other details, too, but this is the main one. And generators are certainly going to cause _some_ overhead over regular calls and returns, just like the manual checking of return values has overhead. My original comment was about the lack of something like delimited continuations in JS, which would allow saving portions of the stack while intentionally minimizing the overhead of regular function calls. That's a well-fitting language-level solution to this issue.
    - ilostmykeys 10y ago
      Ah! You're right. Now I get what you mean. Thanks.
      - jpolitz 10y ago
        No problem. Thanks for asking bluntly about generators. It made me write some more experiments using them. Right now, the main thing that I think stands in the way of them being a good solution for Pyret is that they have limited stack depth – about 7000 frames on Chrome Canary, for instance. One thing we get out of our strategy, which reifies stack frames onto the heap when pausing, is that we can simulate a much deeper stack. See http://imgur.com/a/GaBg2 http://imgur.com/a/GaBg2 (I don't need to be able to do sum of ten million, but 10,000 would be nice! This is just so a non-tail-recursive map works over reasonable-sized lists; we'd rather not have the concept of tail recursion as a curricular dependency for working with that size of data.)

1propionyl 10y ago

Because generators may be implemented with delineated continuations but they are not themselves the same thing.

ilostmykeys 10y ago
Are you sure it's not because those frameworks/libraries were written before generators became available in the latest browsers? :)
morenoh149 10y ago
that's Delimited continuation

jpolitz 10y ago

The main difference is that all of the use cases I mentioned necessarily don't distinguish between calls/functions that may pause, and calls that don't (it's just the semantics of those languages that arbitrary calls might need to pause). So to use generators as a compilation target, every function has to be a generator, and every call a generator instantiation followed by yield*. I actually don't know if that qualifies as "staggering". Your sibling comment has some truth; I can only speak generally about Gopherjs and Doppio, because I know them less intimately, but I know that Pyret and Whalesong were definitely started before generators had widespread adoption. Compiling to generators, rather than to the handwritten stack unwinding we have, is on my list of things to try and measure. Maybe "significant" overhead would be more obviously true than "staggering," since I don't have clear numbers to back it up. Does that make sense?

ilostmykeys 10y ago
<<So to use generators as a compilation target, every function has to be a generator>> Sure, but in those cases the functions are transpiled to JS so it doesn't really matter, not like they're coded by hand, right...? I think I understand what you mean.
- jpolitz 10y ago
  I don't know what you mean by "it doesn't really matter." Surely there's a cost to using generators instead of regular function calls and returns, right? That's where the overhead would come from, because generators aren't free. Right now, Pyret and GopherJS (the last time I checked in GopherJS's case) basically manually encode the `IteratorResult` type, and check for "stack unwind" vs "regular result" when each function call returns, if it might pause. The first question for generators is if they are less overhead than this manual process. There's a bunch of other details, too, but this is the main one. And generators are certainly going to cause _some_ overhead over regular calls and returns, just like the manual checking of return values has overhead. My original comment was about the lack of something like delimited continuations in JS, which would allow saving portions of the stack while intentionally minimizing the overhead of regular function calls. That's a well-fitting language-level solution to this issue.
  - ilostmykeys 10y ago
    Ah! You're right. Now I get what you mean. Thanks.
    - jpolitz 10y ago
      No problem. Thanks for asking bluntly about generators. It made me write some more experiments using them. Right now, the main thing that I think stands in the way of them being a good solution for Pyret is that they have limited stack depth – about 7000 frames on Chrome Canary, for instance. One thing we get out of our strategy, which reifies stack frames onto the heap when pausing, is that we can simulate a much deeper stack. See http://imgur.com/a/GaBg2 http://imgur.com/a/GaBg2 (I don't need to be able to do sum of ten million, but 10,000 would be nice! This is just so a non-tail-recursive map works over reasonable-sized lists; we'd rather not have the concept of tail recursion as a curricular dependency for working with that size of data.)