7 ms·

correct me if I'm wrong but in Node you could use something like a Generator + Promise combination to achieve synchronous looking code with no callbacks https:

correct me if I'm wrong but in Node you could use something like a Generator + Promise combination to achieve synchronous looking code with no callbacks https://eladnava.com/write-synchronous-node-js-code-with-es6-generators/ https://eladnava.com/write-synchronous-node-js-code-with-es6...

drostie 10y ago
You need a wrapper to make it work, but yes, you can do this. The first thing you need is a wrapper to make a Node-style call into a Promise: function node(fn, ...args) { return new Promise((accept, reject) => { args.push((err, ...vals) => err? reject(err) : accept.apply(this, vals)); fn.apply(this, args); }); } What does this do? The argument of `new Promise` is a function which takes two functions, `accept` and `reject`, as its arguments. The function will be immediately run and is meant to do some stuff, possibly asynchronously, and then eventually call the first one of these on success or the second on failure. (One thing that takes a while to get used to about Promises is that generally the asynchronous thing starts when the Promise is created, so execution is not deferred ever.) We just take the args that it was called with, and append our own usual-Node-style callback to those arguments, asking it to pipe the Node error to the reject() function or the successful values to the accept() function. Then you need a sort of "trampoline" to convert a generator-yielding-promises into a single promise. That looks like this: function go(gen_maker) { var gen, goNext; gen = gen_maker(); goNext = val => { var item = gen.next(val); return item.done? item.value : item.value.then(goNext); }; return Promise.resolve(undefined).then(goNext); } Now what does this do? It defines a semirecursive function goNext which will hand itself to any new promises. The goNext function just runs the generator for one step with its argument, and looks at what it got out of that generator. If the generator has completed, it says "I'm done, great, return that last value!" and if it hasn't completed it says "okay, I have a Promise here, I need to tell it that when it's done computing what it's computing, it should goNext() with that value, to hand that value and control back to the generator." Finally after defining this magical function we begin the first gen.next(undefined) run by creating a Promise which immediately resolves to undefined, and telling it to goNext() with it. Of course, `gen` is the output `GeneratorFunction` that is generated by running a function* generator-maker on some arguments. With those two basics, you do eventually de-nest everything to just one layer of asynchronicity. You of course cannot do better than this to get truly synchronous code, but you might not need to if your last expression would have been undefined anyway. For example, here is how you then write something which accepts two arguments and appends the first to the second, using a 10MB buffer in memory. (Yes, I know this is just `fs.createReadStream(process.argv[1]).pipe(fs.createWriteStream(process.argv[2], {flags: 'a'})`, with some messing around with the high-water mark; it's just to prove a point.) // In this comment, we pretend node and go are defined as above, that we have // checked for exactly two args on process.argv and logged some usage // instructions if not, etc. var fs = require('fs'); const buff_size = 10 * 1024 * 1024; go(function* () { var in_file, out_file, buff, count; buff = new Buffer(buff_size); in_file = yield node(fs.openFile, process.argv[1], 'r'); do { out_file = yield node(fs.openFile, process.argv[2], 'a'); count = yield node(fs.read, in_file, buff, 0, buff_size, null); yield node(fs.write, out_file, buff, 0, count); } while (count === buff_size); yield node(fs.close, in_file); yield node(fs.close, out_file); }); In terms of larger libraries that do the same, so that you don't have to roll your own `node` and `go` functions, see e.g. http://taskjs.org/ http://taskjs.org/ and some other similar projects.
adamtulinius 10y ago
In practice a lot of libraries don't support promises, so in my experience you'll just end up with a codebase with callbacks and promises mixed. Not very pretty.
- SonicSoul 10y ago
  pretty sure they're part of ES6 spec[0]. If you're using a 3rd part library that supports callbacks only that sounds more like a choice rather than a limitation with Node. [0] http://www.ecma-international.org/ecma-262/6.0/#sec-promise-constructor http://www.ecma-international.org/ecma-262/6.0/#sec-promise-...
- bilalq 10y ago
  Any library that accepts a standard node callback can be easily promisified. See: http://bluebirdjs.com/docs/api/promise.promisify.html http://bluebirdjs.com/docs/api/promise.promisify.html http://bluebirdjs.com/docs/api/promise.promisifyall.html http://bluebirdjs.com/docs/api/promise.promisifyall.html
  - spriggan3 10y ago
    Now compare that kind of patching with writing Go code where non blocking I/O doesn't require any special syntax. I'm not fan of Go by a long shot, but it is undeniable the blue/red function problem is a pain [1] when dealing with JS. [1] : http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...
    - jerf 10y ago
      I love that blog post, because then this makes sense: Promises are basically an inner-platform [1] that replicate all the control flow constructs of the given language, except as a different "color". Go simply is that inner platform. Go does not use promises because all of its code is already running in the promise color. You don't need anything special to "deal with" a for loop that may end up waiting for something to be read from a socket, because the for loop is already a "promise", and so forth. You can rearrange your code as much as you like, and you are not constantly juggling with stuff the compiler ought to deal with. Manual promise transpilation is better than manual callback transpilation, but it's still manual transpilation. Why are you working so hard? Sometimes I get mistaken as an advocate for Go. I'm not really. What I am an advocate for is using languages where you don't need an inner-platform of promises because the language is already that color. Go just happens to be the most familiar Algol-esque of the bunch right now. Don't want to learn Go? Go learn Erlang or Elixir. Or maybe if you're feeling saucy, Haskell. Or some of the other choices popping up lately. If you're writing network servers you will not be able to understood how you ever functioned without this. [1]: https://en.wikipedia.org/wiki/Inner-platform_effect https://en.wikipedia.org/wiki/Inner-platform_effect
scoin 10y ago
Generators are sweet, but you can use just Promises or the async.js library to write nice flat code. No callback hell / play button shaped code.
- masklinn 10y ago
  While they allow limiting the rightsward drift of code, explicit promises are still event emitters and still require a significant amount of callbacks if you don't have generators or async functions.
- drostie 10y ago
  The nice thing about the generator is that (with a nontrivial amount of boilerplate!) you can pretend that the body of the function* is a synchronous zone even though it makes several calls to asynchronous promises. The basic idea is that you wrap the generator in a sort of trampoline that turns it into a promise. This 'trampoline' detects whenever the generator falls down to it (via some `yield` of some Promise) and then waits on that promise before resuming the generator-trampoline combo. The promise then returns its result directly into generator.next(), where it becomes the value of the `yield` expression, thus within the generator `x = yield myPromise()` creates a new Promise by calling myPromise, suspends execution until that Promise completes (by dropping down to the trampoline, which bounces us up into the myPromise().then()), and finally gets the value and assigns it to x. You have just one scope, the scope of the function* of the generator, in which all of the results of all of your Promises live.
lossolo 10y ago
Sure but you need to know that you will pay for this abstraction with some performance penalty (depends what your expectations are). Another thing is this workaround is still more complex because you need to create generator function and then you need to write reentry points, you don't need in Go. Anyway this is for sure good solution for Node but in context of comparing Go is still in my opinion easier to reason about.
- SonicSoul 10y ago
  this is interesting. could you show (or point to a good example) of how GO handles this w/out reentry points? I mean you have to tell it where to return the results somehow
  - lossolo 10y ago
    It really depends on your app logic. If you need to process data in waterfall like model and you have multiple tasks working in same time you still need to use asynchronous calls or you will lock the execution of code in node. In go you just lock the gorutine in which you make I/O call.
  - hellcow 10y ago
    Goroutines communicate data via channels, so you can just pass a channel a message (which can be anything). The channel will sit and wait for a message, processing the message when it's received. That said, for anything like a website or an API you probably won't need to worry much about channels or concurrency at all. The stdlib handles each web request on a different goroutine, so you write synchronous code, yet your server is concurrent.
    - SonicSoul 10y ago
      ah yes! This behavior is also supported in Javascript but not as natively as in GO. But there are CSP libraries out there :) http://jlongster.com/Taming-the-Asynchronous-Beast-with-CSP-in-JavaScript http://jlongster.com/Taming-the-Asynchronous-Beast-with-CSP-...

drostie 10y ago

You need a wrapper to make it work, but yes, you can do this. The first thing you need is a wrapper to make a Node-style call into a Promise: function node(fn, ...args) { return new Promise((accept, reject) => { args.push((err, ...vals) => err? reject(err) : accept.apply(this, vals)); fn.apply(this, args); }); } What does this do? The argument of `new Promise` is a function which takes two functions, `accept` and `reject`, as its arguments. The function will be immediately run and is meant to do some stuff, possibly asynchronously, and then eventually call the first one of these on success or the second on failure. (One thing that takes a while to get used to about Promises is that generally the asynchronous thing starts when the Promise is created, so execution is not deferred ever.) We just take the args that it was called with, and append our own usual-Node-style callback to those arguments, asking it to pipe the Node error to the reject() function or the successful values to the accept() function. Then you need a sort of "trampoline" to convert a generator-yielding-promises into a single promise. That looks like this: function go(gen_maker) { var gen, goNext; gen = gen_maker(); goNext = val => { var item = gen.next(val); return item.done? item.value : item.value.then(goNext); }; return Promise.resolve(undefined).then(goNext); } Now what does this do? It defines a semirecursive function goNext which will hand itself to any new promises. The goNext function just runs the generator for one step with its argument, and looks at what it got out of that generator. If the generator has completed, it says "I'm done, great, return that last value!" and if it hasn't completed it says "okay, I have a Promise here, I need to tell it that when it's done computing what it's computing, it should goNext() with that value, to hand that value and control back to the generator." Finally after defining this magical function we begin the first gen.next(undefined) run by creating a Promise which immediately resolves to undefined, and telling it to goNext() with it. Of course, `gen` is the output `GeneratorFunction` that is generated by running a function* generator-maker on some arguments. With those two basics, you do eventually de-nest everything to just one layer of asynchronicity. You of course cannot do better than this to get truly synchronous code, but you might not need to if your last expression would have been undefined anyway. For example, here is how you then write something which accepts two arguments and appends the first to the second, using a 10MB buffer in memory. (Yes, I know this is just `fs.createReadStream(process.argv[1]).pipe(fs.createWriteStream(process.argv[2], {flags: 'a'})`, with some messing around with the high-water mark; it's just to prove a point.) // In this comment, we pretend node and go are defined as above, that we have // checked for exactly two args on process.argv and logged some usage // instructions if not, etc. var fs = require('fs'); const buff_size = 10 * 1024 * 1024; go(function* () { var in_file, out_file, buff, count; buff = new Buffer(buff_size); in_file = yield node(fs.openFile, process.argv[1], 'r'); do { out_file = yield node(fs.openFile, process.argv[2], 'a'); count = yield node(fs.read, in_file, buff, 0, buff_size, null); yield node(fs.write, out_file, buff, 0, count); } while (count === buff_size); yield node(fs.close, in_file); yield node(fs.close, out_file); }); In terms of larger libraries that do the same, so that you don't have to roll your own `node` and `go` functions, see e.g. http://taskjs.org/ http://taskjs.org/ and some other similar projects.

adamtulinius 10y ago

In practice a lot of libraries don't support promises, so in my experience you'll just end up with a codebase with callbacks and promises mixed. Not very pretty.

SonicSoul 10y ago
pretty sure they're part of ES6 spec[0]. If you're using a 3rd part library that supports callbacks only that sounds more like a choice rather than a limitation with Node. [0] http://www.ecma-international.org/ecma-262/6.0/#sec-promise-constructor http://www.ecma-international.org/ecma-262/6.0/#sec-promise-...
bilalq 10y ago
Any library that accepts a standard node callback can be easily promisified. See: http://bluebirdjs.com/docs/api/promise.promisify.html http://bluebirdjs.com/docs/api/promise.promisify.html http://bluebirdjs.com/docs/api/promise.promisifyall.html http://bluebirdjs.com/docs/api/promise.promisifyall.html
- spriggan3 10y ago
  Now compare that kind of patching with writing Go code where non blocking I/O doesn't require any special syntax. I'm not fan of Go by a long shot, but it is undeniable the blue/red function problem is a pain [1] when dealing with JS. [1] : http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...
  - jerf 10y ago
    I love that blog post, because then this makes sense: Promises are basically an inner-platform [1] that replicate all the control flow constructs of the given language, except as a different "color". Go simply is that inner platform. Go does not use promises because all of its code is already running in the promise color. You don't need anything special to "deal with" a for loop that may end up waiting for something to be read from a socket, because the for loop is already a "promise", and so forth. You can rearrange your code as much as you like, and you are not constantly juggling with stuff the compiler ought to deal with. Manual promise transpilation is better than manual callback transpilation, but it's still manual transpilation. Why are you working so hard? Sometimes I get mistaken as an advocate for Go. I'm not really. What I am an advocate for is using languages where you don't need an inner-platform of promises because the language is already that color. Go just happens to be the most familiar Algol-esque of the bunch right now. Don't want to learn Go? Go learn Erlang or Elixir. Or maybe if you're feeling saucy, Haskell. Or some of the other choices popping up lately. If you're writing network servers you will not be able to understood how you ever functioned without this. [1]: https://en.wikipedia.org/wiki/Inner-platform_effect https://en.wikipedia.org/wiki/Inner-platform_effect

scoin 10y ago

Generators are sweet, but you can use just Promises or the async.js library to write nice flat code. No callback hell / play button shaped code.

masklinn 10y ago
While they allow limiting the rightsward drift of code, explicit promises are still event emitters and still require a significant amount of callbacks if you don't have generators or async functions.
drostie 10y ago
The nice thing about the generator is that (with a nontrivial amount of boilerplate!) you can pretend that the body of the function* is a synchronous zone even though it makes several calls to asynchronous promises. The basic idea is that you wrap the generator in a sort of trampoline that turns it into a promise. This 'trampoline' detects whenever the generator falls down to it (via some `yield` of some Promise) and then waits on that promise before resuming the generator-trampoline combo. The promise then returns its result directly into generator.next(), where it becomes the value of the `yield` expression, thus within the generator `x = yield myPromise()` creates a new Promise by calling myPromise, suspends execution until that Promise completes (by dropping down to the trampoline, which bounces us up into the myPromise().then()), and finally gets the value and assigns it to x. You have just one scope, the scope of the function* of the generator, in which all of the results of all of your Promises live.

lossolo 10y ago

Sure but you need to know that you will pay for this abstraction with some performance penalty (depends what your expectations are). Another thing is this workaround is still more complex because you need to create generator function and then you need to write reentry points, you don't need in Go. Anyway this is for sure good solution for Node but in context of comparing Go is still in my opinion easier to reason about.

SonicSoul 10y ago
this is interesting. could you show (or point to a good example) of how GO handles this w/out reentry points? I mean you have to tell it where to return the results somehow
- lossolo 10y ago
  It really depends on your app logic. If you need to process data in waterfall like model and you have multiple tasks working in same time you still need to use asynchronous calls or you will lock the execution of code in node. In go you just lock the gorutine in which you make I/O call.
- hellcow 10y ago
  Goroutines communicate data via channels, so you can just pass a channel a message (which can be anything). The channel will sit and wait for a message, processing the message when it's received. That said, for anything like a website or an API you probably won't need to worry much about channels or concurrency at all. The stdlib handles each web request on a different goroutine, so you write synchronous code, yet your server is concurrent.
  - SonicSoul 10y ago
    ah yes! This behavior is also supported in Javascript but not as natively as in GO. But there are CSP libraries out there :) http://jlongster.com/Taming-the-Asynchronous-Beast-with-CSP-in-JavaScript http://jlongster.com/Taming-the-Asynchronous-Beast-with-CSP-...