10 ms·

Why I am switching to promises

124 points by shawndumas 13y ago

beering 13y ago
Oh goodness... if only there was a way for the computer to manage all the bookkeeping of what happens after what, in a way where multiple things could run at approximately the same time, without the programmer having to thread "promises" or "callbacks" through every single piece of non-trivial code. We could even give a name to these sequences of operations... maybe call them "strands"... or "filaments"... Using libraries in Node.js seems to be a matter of 1) finding a library, 2) taking a hammer and smashing it into your existing choice of continuation-tracking mechanism. We end up with N x M different ways of using a package, depending on what style the package author chose vs what style the programmer chooses. "nodeify" is a bandage on top of a missing limb. Do people really put up with this? Just look at how many different things were benchmarked in this article---I counted 21. How many more ways of reinventing basic programming-language design is the JS community going to come up with before we realize that the emperor has no clothes?
- macspoofing 13y ago
  >How many more ways of reinventing basic programming-language design is the JS community going to come up with before we realize that the emperor has no clothes? Yes, JS was a rushed language that was then left to rot for 15-odd years. Unfortunately it's the language that powers the web, runs everywhere, and (along with HTML/CSS) is the only thing that all the big guys agree on. It's getting better but it is what it is.
  - thoradam 13y ago
    Which reminds me, wasn't ECMAScript 6 supposed to be finalised in December[1]? 1: http://en.wikipedia.org/wiki/ECMAScript#ECMAScript_Harmony_.286th_Edition.29 http://en.wikipedia.org/wiki/ECMAScript#ECMAScript_Harmony_....
    - cpeterso 13y ago
      The target date for ES6 has slipped to December 2014: https://github.com/rwaldron/tc39-notes/blob/48c5d285bf8bf0c4e6e8bb0c02a7c840c01cd2ff/es6/2013-03/mar-13.md#416-current-status-of-es6 https://github.com/rwaldron/tc39-notes/blob/48c5d285bf8bf0c4...
  - zaphar 13y ago
    Yes. JS runs everywhere that a modern browser runs and powers the web's client side. Those are excellent reasons to use javascript to power your web applications client side as opposed to say Flash or ActiveX. However, those have absolutely no bearing on writing your server code in JS. The only reason to run JS on your server is so you can write the server side and client side in the same language. That's a debatable reason for running JS on your server. Node incredible scalability around IO is also provided by a host of other not rushed and left to rot languages. Erlang, Go, C++, even Java has libraries for it. When it comes to server side you can make a pretty good argument that the "emperor has no clothes".
    - TylerE 13y ago
      Hardly the only reason. Look at something like Meteor, which is running not only the same language client and server side, but there is a large overlap in the actual code as well, which makes a ton of sense to me. For instance, you can run the actual, same, validation code against a form and before the data is inserted into the DB.
    - pfraze 13y ago
      You're underestimating NPM and the community of modules around it. Node is well-rounded, and it's capable of handling most programming tasks. I use it to write utility scripts, which it's pretty good at. When I make a service, Express is a nice DSL-like framework. It gets lots of stuff done, so I'd say it's pretty well-clothed.
      - cgag 13y ago
        Clojure/java also have lots of libraries and tools around them, as well as proper promises (where you can block and wait if you want), channels/goroutines, futures, threads, agents, reducers, etc. Erlang / Haskell / even Go are saner languages with better concurrency stories. I guess there's something to be said for not having to learn more than one language. Probably something vulgar.
      - zaphar 13y ago
        No I'm really not. NPM is a good package manager. I'm sure it has good modules as well. But NPM isn't the only package manager out there. Other languages have excellent package managers too. Clojars, CPAN, Ruby Gems, even erlang has a package manager now. Those are minimum reqs for a language and community. They aren't differentiators. Aside from their absence being a negative indicator.
- ebiester 13y ago
  How long? Until one of the other languages (say... clojureScript) develops a large enough ecosystem and best practices around it to allow portions of code to be shared between the client and server. Oh, that also means that this given system has to reach critical mass. Oh, and since everything's server and client side, the compilation has to be rock solid and be easy enough to understand without reverting to javascript in frustration. I miss programming in Node, for nothing else that I hate having to keep up with so many paradigms and switch between the two. I only have X amount of hours to keep up. When I move back to heavy development in a language after even a few days off, there is friction. And as someone who writes both front end and back end code, it gets frustrating. I'm not as good as I could be, given the hours in the day I have.
- JoshTriplett 13y ago
  Threads are quite useful, and Javascript has them. However, if what you have is a computation you have the inputs to start now and won't need the result of for a while, lazy evaluation is a far more natural representation for that than a thread. Ideally, a language ought to have one representation for lazy evaluation, not a dozen; perhaps at some point a new standard will come out that standardizes some subset of lazy evaluation that has won out. Welcome to Javascript.
  - zaphar 13y ago
    Javascript does not have threads. There are some Javascript environments that may or may not have threads in some form. Rhino, WebWorkers. But Javascript does not have a builtin concept of threads.
- gfodor 13y ago
  Yeah I just started working on a node project and can't help but feel like I have been transported to a time 40 years ago where my operating system didn't have a process scheduler and I have to do it all myself. I literally have to think about how asynchronous behavior works every 2 or 3 lines of code. It's sheer madness.
  - nawitus 13y ago
    Your comment is ironic because C# implemented async in 2012. Async thinking takes a while to get used, but that doesn't mean it's more difficult than threads or mutexes and whatnot. In fact, I think it's simpler. You don't have to learn anything properly new if you already know how functions work.
    - gfodor 13y ago
      we haven't needed complicated threading on the web server for the last 20 years, because there is a natural isolation model that works most of the time: the request. node turns this on its head and says now you will need to think about concurrency always, even if you don't have a problem that naturally warrants it. that's the issue. if i want to write a goddamn shell script I need to think about concurrency. wat? are callbacks vs mutexes vs actors vs STM the best way to handle concurrency, that's debatable. but what's not debatable is that comparing concurrent vs non-concurrent code the latter is always easier for people to write and reason about, so you should tend to want to work in environments that are non-concurrent by default and let you focus just on the parts that you need to deal with concurrency.
      - nawitus 13y ago
        Okay, now I understand your critique. I don't think it's a very good point though: Node.js is not designed for extremely simple synchronous web servers (and neither is JavaScript).
        caoilte 13y ago
        That's a generational perspective. For developers fresh out of college promises, async, continuations and so forth are the norm now.
- spion 13y ago
  Some of those "things" you counted are based on generators, which let you to do exactly what you want. Example: https://github.com/spion/async-compare/blob/master/examples/promises-bluebird-generator.js https://github.com/spion/async-compare/blob/master/examples/... With promises, generators are not really necessary -- especially if you add arrow function syntax into the mix. https://gist.github.com/spion/8254661 https://gist.github.com/spion/8254661 (Note: that particular method in doxbee has long since been refactored to something saner with better separation of concerns :)
- dustingetz 13y ago
  Promises are actually very simple - so fundamental that you can think of them as dual to function composition. By a dual to function composition i mean that you can mechanically convert async promise code to everyday synchronous function calls - including everyday exceptions. That means that literally the code can look exactly the same whether or not you are async or sync. The trick is you need either macros or monads to do this. (This is kind of the whole point of monads - you could write a piece of code by composing a few functions, and then make it asynchronous without breaking the function composition! No need to rewrite your code to use callbacks; the same code will just work.)
  - gcr 13y ago
    Is this currently possible with NodeJS? Javascript has none of the following: - Macros - Monads - call/cc
    - jonsterling 13y ago
      Monads are not a language feature, they are a library feature.
      - gcr 13y ago
        Ok, true. They're certainly much nicer to use if you have nice syntax for them though. :)
      - dustingetz 13y ago
        Only true in the most pedantic sense. For monads to work, all your code needs to be pure (in the FP sense). So if you want to do anything substantial with monads in an imperative language, you need to rewrite the entire ecosystem of libraries. That would cost quite a bit more money than your average organization can afford ;)
        jonsterling 13y ago
        Also has nothing to do with the language, mate. Your argument does nothing but agree with my point :)
- davedx 13y ago
  You mean like fibers? :) https://github.com/laverdet/node-fibers https://github.com/laverdet/node-fibers This is what Meteor uses on the server side, and it's a pleasure. Everything is suddenly synchronous.
- gcr 13y ago
  (Edit: Oops! Sorry, I see that you're already familiar with continuations, but let me explain them for those who aren't) Sometimes, fundamental switches in thinking are worth it. :) For example, continuations would be perfect here, and are a wonderful tool for simplifying "nested callback hell" type problems. But why should a web developer care about continuations? Consider Paul Graham's Arc challenge: Build a web page that presents a form to the user. After they submit the form, a new page presents them with a link. The link takes them to a third page where the submitted value is presented back to the user. The restriction is that the input field must not be passed in via the URL. Right now, the user must write in "continuation-passing style", which means threading callbacks, like this: function serve(req,res) { askUserForValue(req,res, function(value){ showLink(req, res, function(){ render(res, {"user-value": value}); }); }); } Or perhaps the user could somehow stuff the variable into a "session" library with three separate URL routes: function serve(req,res) { render the form with the POST action of /submit; } function submit(req,res, value) { req.session["uservalue"] = value; render the link that sends the user to /view; } function view(req,res) { render the value of req.session["uservaule"]; } This one is even more fragmented since you have three separate functions that model one user interaction flow! I certainly like it less than the callback example -- at least I can keep one in my head. We can do better though. Continuations let you write this: function serve(req, res) { var value = askUserForValue(); showLink(); render(res, {"user-value": value}); } If node.js supported continuations, the askUserForValue() function would respond with the form and suspend the serve function right then and there. After the user made a new HTTP call, the suspended askUserForValue() call would return to serve() and its return value would be placed into our variable. Then, the showLink() function could render the link and suspend the function again. Once the user makes another HTTP request, the function resumes, and so on. The server would have to store the state of the computation itself to disk so it could continue later. With continuations, we can pretend the user's web browser is the program counter. This frees my mind up. First, I don't have to keep callbacks in my head since there aren't any callbacks anymore. Second, stack traces work the way I expect because functions explicitly call each other. Third, if askForValue() throws an exception if the user inputs a string where I entered a number, no problem! Just wrap it in a try{} block that asks the user to enter a valid value, as if I'm writing a text program that calls readline(). Also note that this is different from just letting each user interaction be in its own thread! Consider: 1. Continuations should be serializable to disk. You wouldn't want blocked threads to just pile up as more users visit your website, and if your server crashes, you certainly wouldn't want to forget their state. 2. The browser "back" button would have to step a program backwards in time; something that is unsupported in a naïve thread-equals-user-interaction system. (E.g. how many of you have booked an airplane ticket with Travelocity/Expedia/whatever, pressed "Back", and then got a big error because the server didn't know how to step backwards through its user stack?) But the ability to save continuations to disk is the ability to save the current state of the program to disk, which makes stepping back in time easy: you can simply reload what was about to happen, just like savestates in a video game emulator. This is not a new idea. Many Common Lisp and Smalltack web frameworks work well with continuations and have done so for years (the first Seaside release was 2002, I'm not kidding you!) Racket, a great dialect of Scheme, ships with such a web framework out of the box. Here's my implementation of the Arc challenge in Racket. Note how the 'start' function is implemented: https://gist.github.com/gcr/ad116f565e105c8b2e0d https://gist.github.com/gcr/ad116f565e105c8b2e0d It's super easy to try this out and you don't need to fiddle with dependencies or anything. Just download Racket from http://racket-lang.org/ http://racket-lang.org/ , copy+paste the code into DrRacket's giant text box on the top, press Run, and your web browser will pop up with the example.
- bbllee 13y ago
  I find one of your premises very thin: browser implementors are implementing DOM promises, so promises were clearly not invented solely to make up for the difficulties of Node.
- SimHacker 13y ago
  The emperor may have no clothes, but what's so great about clothes? (If God meant people to be naked, they'd be born that way! Same goes for programming languages.) The naked Emperor JavaScript runs everywhere, and runs fast, and isn't going away. He's streaking at lightning speed all over the kingdom, everybody's seen him, and he doesn't stay still. The emperor is naked. Long run the emperor.
ilaksh 13y ago
I hope that everyone who writes JavaScript will do themselves a favor and try out ToffeeScript. This offshoot of CoffeeScript simplifies things. For example: if fs.exists! thefile e, datastr = fs.readFile! thefile, 'utf8' else datastr = 'not found' success = db.update! 'somekey', datastr res.end success So in that example even though there are three callbacks it looks like a normal synchronous flow.
- lucaspiller 13y ago
  I haven't heard of ToffeeScript before, but this is pretty much how Erlang works. Whenever you make a blocking call, all of the async stuff happens under the hood, so that you just deal with sequential code. Erlang's concurrency model deals with lots of lightweight processes. Rather than having callbacks it uses a message system. Whenever a process waits for a message it yields so other processes can be executed. In your example there would be a process separate from the main[0] process that handles the database connection. When you request the update to the database, you are just sending a message to the database connection process. This will receive the message and handle the update when it can[1], and once complete, send you a message back with the result of the update. During the waiting period the VM will continue executing other processes. No callbacks needed :) [0] Erlang concurrency is based upon the actor model. There isn't really a main process, but for the sake of explanation... [1] The database connection process will actually block while making the update, so if you have 10,000 processes calling this, you will have a bottleneck. As such you should have a pool of database connection processes, rather than just one.
  - ilaksh 13y ago
    Yeah Erlang looks awesome and I should definitely try to learn it at some point. To me Node.js and ToffeeScript seem a lot more straightforward and have the advantage of the whole package system which I believe is more advanced, as well as the npm ecosystem with 54,000 packages. But anyway it would be good to actually learn Erlang and use it in some projects so that I can really evaluate it. I'm not sure but I think it might be a lot more resource efficient for one thing.
- 13y ago
OhHeyItsE 13y ago
JavaScript promises. Got it. Wait - crashing processes? Denial of service? Wha? Why would JavaScript?... Oh, this is about Node. Goodbye.
Patrick_Devine 13y ago
I'm not a js guy, but this concept sounds a lot like Twisted's (python) concept of "deferreds". The idea is that you "defer" caring about some asynchronous process, and then attach "callbacks" and "errbacks" to the "deferred". You essentially create a chain of callbacks or errbacks which gets called depending on if a callback is fired or an exception is raised. Overall it's a pretty great concept, but debugging can be a real pain.
- domenicd 13y ago
  Yup, from what I recall of history both Twisted Deferreds and JavaScript promises have their heritage in E's promises.
- davedx 13y ago
  Yup. jQuery: http://api.jquery.com/category/deferred-object/ http://api.jquery.com/category/deferred-object/
- spion 13y ago
  Debugging with Bluebird (mentioned in the article) is not a pain because it stitches stack traces across events for you when you run it in development mode :)
eldude 13y ago
The problem with discussions of promises vs ... is there's usually a great deal of misunderstanding about what problem promises are solving: enforcement of the callback contract. I could go on for hours about the subtleties in the conversation, but the gist of it boils down to this: Node callbacks have a strict but entirely unenforced (in userland) contract: The contract: * Callbacks should be called OAOO * Never throw (explicitly or implicitly) after origin tick (aka pass post-tick errors to callback) There is also an increasingly common informal contract: * Never throw ever (aka pass ALL errors to the callback) But there are a couple of issues that get in the way of successful enforcement of this contract: * When external libraries violate this contract (library A calls library B, but library B violates the contract in a way that makes it difficult for library A to easily maintain the contract) * Asynchronous errors * Crash-worthy exceptions Inevitably, this all continually comes back to error handling over and over again, with the core issue being that node.js has 2 divergent methods for error handling: exceptions (try/catch/throw) and error passing (callback(err)). Exceptions requires opt-in error handling (crash by default) while error passing requires opt-out error handling (crash on demand) "and never the twain shall meet." Promises attempt to resolve this by coercing all exceptions to error passing, but not all exceptions are safe to be passed. Specifically, any exception originating from core that is not an invalid argument exception (think 4XX vs 5XX class errors) cannot safely be caught / coerced / continued on. Also, not all errors passed to callbacks are even theoretically passable as the unofficial policy of core is that core callback errors are only distinguishable from exceptions in that they occur after the origin tick. In other words, they are equally crash-worthy and equally non-crash-worthy. So promises are an improvement because when used pervasively, they enforce the formal and informal callback contracts, while providing a continuable-like representation of the value that can be passed around. This benefit of contract enforcement is entirely independent of the specific API implementations. The stepup[1] library trivially accomplishes contract enforcement by integrating the async trycatch[2] module without using promises or requiring pervasive usage. I've used them both for years now in various professional projects, with trycatch in use here at LinkedIn. Additionally, stepup could easily return a continuable to support a passable value representation. Async generators will solve most of this though, allowing node.js to ditch the error passing for exception handling, but since Error subclassing and typed catches are basically not supported in node.js the language keeps getting in the way of a satisfactory complete solution. This is a whole other discussion. In conclusion, error handling in node.js is an undesigned mess of which the core contributors don't even seem to fully understand or be aware[3,4]. FWIW, node.js' crash on error design is a DoS liability, which spion addresses in the article, and which LinkedIn uses trycatch to avoid. [1] https://npmjs.org/package/stepup https://npmjs.org/package/stepup [2] https://npmjs.org/package/trycatch https://npmjs.org/package/trycatch [3] https://github.com/joyent/node/issues/5114 https://github.com/joyent/node/issues/5114 [4] https://github.com/joyent/node/issues/5149 https://github.com/joyent/node/issues/5149
nadaviv 13y ago
> if (err) return callback(err) > That line is haunting me in my dreams now. What happened to the DRY principle? iferr = (succfn, errfn) -> (err, a...) -> if err? then errfn err else succfn a... show_user = (id, cb) -> db.query ..., iferr cb, (user) -> ... FTFY, no promises needed. (not to say that promises don't have other advantages, but that specific complaint is easily fixable with an higher-order function to abstract it away) (JavaScript version: http://coffeescript.org/#try:%20%20iferr%20%3D%20(succfn%2C%20errfn)%20-%3E%20(err%2C%20a...)%20-%3E%0A%20%20%20%20if%20err%3F%20then%20errfn%20err%20else%20succfn%20a...%0A%0A%20%20show_user%20%3D%20(id%2C%20cb)%20-%3E%0A%20%20%20%20db.query%20foo%2C%20iferr%20cb%2C%20(user)%20-%3E%0A%20%20%20%20%20%20bar http://coffeescript.org/#try:%20%20iferr%20%3D%20(succfn%2C%...)
- spion 13y ago
  No its not. I address this immediately afterwards (below "But spion, why don't you wrap your calbacks?"), but basically, this simple wrapper makes stack traces unusable. Bluebird (and also Q) stitches together stack traces from multiple previous events. This is amazingly awesome.
rtpg 13y ago
Slightly off topic, but I've been working in Haskell, and recently realized how the laziness of the language means everything is basically asynchronous by default, but you can still write your code in a synchronous fashion. No real need for promises or callbacks because lazy values are the promises. Well, when using lazy IO, at least. I'd actually much prefer lazy values be put into Javscript than almost any other feature (real module support is another big one).
- ilaksh 13y ago
  See my comment below about ToffeeScript which compiles to JavaScript.
- rapala 13y ago
  Then again, Simon Peyton Jones has said that were he to make a new Haskell, it would have strict evaluation. As far as I know, lazy IO is considered a pain in the butt. That beeing said, laziness does have its merits and some ideas can be elegantly stated in a lazy language.
  - rtpg 13y ago
    do you have a source on this? For me laziness is the single strongest thing going for Haskell.
    - rapala 13y ago
      It's from a retrospective on Haskell: www.cs.nott.ac.uk/~gmh/appsem-slides/peytonjones.ppt There seems to be two arguments against laziness. Firstly, it makes reasoning about performance hard. Secondly, forcing strict evaluation is tricky, maybe harder than it would be to do optional lazy evaluation in a strict language.
  - audreyt 13y ago
    The default Lazy IO issue has been addressed in pipes-bytestring: http://www.haskellforall.com/2013/09/perfect-streaming-using-pipes-bytestring.html http://www.haskellforall.com/2013/09/perfect-streaming-using... The pipes ecosystem ( http://www.haskell.org/haskellwiki/Pipes http://www.haskell.org/haskellwiki/Pipes ) is working well to offer {effects, streaming, composability} without the resource leak issues introduced by lazy IO.
moron4hire 13y ago
I see stuff like: promise.when(function(value){ //blah blah blah }) .error(function(err){ //handle it }) and I think to myself I'm seeing text replacements: "promise.when(function(value)" = "try" ".error(function" = "catch" "})" = "}" So how is this actually different from the code we're supposedly too lazy to write or too stupid to figure out or something in the first place, thus requiring this slightly more verbose form? And where is this all going? Like, what's the right granularity of lines-of-code to when chains? If programmers are too lazy to try/catch, then why aren't we too lazy to when/then/error a gigantic block, or you know, not at all? And if we were to instead [w|t]hen/error every line, then I think about DRY and and it occurs to me the most likely next step is a language that "implicitly whens your code", that every line is written like regular procedural code, but automagically has "when" inserted in the middle. Or in other words, you know, continuations. But it occurs to me that the only reason any of this stuff is of any interest is because JavaScript, and therefore Node, still lacks real MT support, so you have to jump through these infinitely fractal spilling callback tangos to fake it. Web Workers was 2009, people. 5 YEARS MAN. Five years that we've been waiting to hear anything about having sane primitives for MT.
- spion 13y ago
  Nope, `doStuff().then(function(value)" != try` A closer representation would be `try { var value = doStuff()` which has similar verbosity. ES6 will have generators and "arrow" function syntax. Meaning: doThing().then(function(value){ //blah blah blah }) .catch(function(err){ //handle it }) becomes either: try { var value = yield doThing(); //blah blah blah } catch (err) { //handle it } or doThing().then(value => /* blah blah blah */) .catch(err => /* handle it */); meaning you can pick your paradigm :) Except that the functional one composes slightly better, perhaps. doThing().then(value => /* blah blah blah */) .catch(only(ParseError, err => /* handle it */)); vs try { var value = yield doThing(); //blah blah blah } catch (err) { only(err, ParseError); // argument repetition //handle it }
  - moron4hire 13y ago
    Uhhh, you're not convincing me that you've said anything meaningful towards improving software quality.
pornel 13y ago
Promises have two awesome things: * With callbacks interface of the function hardcodes whether the function is strictly synchronous or may use any async code. Change of one little helper function from sync to async may have ripple effect throughout the entire codebase (you have to add callback to all its callers, and all their callers, and their callers...). Promises fix that. Sync and async functions take arguments and return values the same way, so when you change sync to async you only need to change source and consumer of the value, and nothing in between (e.g. same code can cache results of (memoize) promise-returning functions and sync functions). * With ES6 yield async calls look and behave almost like sync calls. No callbacks. You don't have to use weird error handling schemes — just use try/catch! You don't have to have a library to iterate over things — just use for()! A call that used to look like: use(compute(arg)) was twisted by async to be: compute(arg, use) Promises change it to: compute(arg).then(use) and yield changes it back to nearly original form: use(yield compute(arg))
hatchoo 13y ago
Aren't they made to be broken? (Sorry, couldn't help it)
SimHacker 13y ago
And what's wrong with reinventing basic programming-language design a million different ways? Are you complaining that it's too much like Lisp?

124 points by shawndumas 13y ago

Oh goodness... if only there was a way for the computer to manage all the bookkeeping of what happens after what, in a way where multiple things could run at approximately the same time, without the programmer having to thread "promises" or "callbacks" through every single piece of non-trivial code. We could even give a name to these sequences of operations... maybe call them "strands"... or "filaments"... Using libraries in Node.js seems to be a matter of 1) finding a library, 2) taking a hammer and smashing it into your existing choice of continuation-tracking mechanism. We end up with N x M different ways of using a package, depending on what style the package author chose vs what style the programmer chooses. "nodeify" is a bandage on top of a missing limb. Do people really put up with this? Just look at how many different things were benchmarked in this article---I counted 21. How many more ways of reinventing basic programming-language design is the JS community going to come up with before we realize that the emperor has no clothes?

macspoofing 13y ago
>How many more ways of reinventing basic programming-language design is the JS community going to come up with before we realize that the emperor has no clothes? Yes, JS was a rushed language that was then left to rot for 15-odd years. Unfortunately it's the language that powers the web, runs everywhere, and (along with HTML/CSS) is the only thing that all the big guys agree on. It's getting better but it is what it is.
- thoradam 13y ago
  Which reminds me, wasn't ECMAScript 6 supposed to be finalised in December[1]? 1: http://en.wikipedia.org/wiki/ECMAScript#ECMAScript_Harmony_.286th_Edition.29 http://en.wikipedia.org/wiki/ECMAScript#ECMAScript_Harmony_....
  - cpeterso 13y ago
    The target date for ES6 has slipped to December 2014: https://github.com/rwaldron/tc39-notes/blob/48c5d285bf8bf0c4e6e8bb0c02a7c840c01cd2ff/es6/2013-03/mar-13.md#416-current-status-of-es6 https://github.com/rwaldron/tc39-notes/blob/48c5d285bf8bf0c4...
- zaphar 13y ago
  Yes. JS runs everywhere that a modern browser runs and powers the web's client side. Those are excellent reasons to use javascript to power your web applications client side as opposed to say Flash or ActiveX. However, those have absolutely no bearing on writing your server code in JS. The only reason to run JS on your server is so you can write the server side and client side in the same language. That's a debatable reason for running JS on your server. Node incredible scalability around IO is also provided by a host of other not rushed and left to rot languages. Erlang, Go, C++, even Java has libraries for it. When it comes to server side you can make a pretty good argument that the "emperor has no clothes".
  - TylerE 13y ago
    Hardly the only reason. Look at something like Meteor, which is running not only the same language client and server side, but there is a large overlap in the actual code as well, which makes a ton of sense to me. For instance, you can run the actual, same, validation code against a form and before the data is inserted into the DB.
  - pfraze 13y ago
    You're underestimating NPM and the community of modules around it. Node is well-rounded, and it's capable of handling most programming tasks. I use it to write utility scripts, which it's pretty good at. When I make a service, Express is a nice DSL-like framework. It gets lots of stuff done, so I'd say it's pretty well-clothed.
    - cgag 13y ago
      Clojure/java also have lots of libraries and tools around them, as well as proper promises (where you can block and wait if you want), channels/goroutines, futures, threads, agents, reducers, etc. Erlang / Haskell / even Go are saner languages with better concurrency stories. I guess there's something to be said for not having to learn more than one language. Probably something vulgar.
    - zaphar 13y ago
      No I'm really not. NPM is a good package manager. I'm sure it has good modules as well. But NPM isn't the only package manager out there. Other languages have excellent package managers too. Clojars, CPAN, Ruby Gems, even erlang has a package manager now. Those are minimum reqs for a language and community. They aren't differentiators. Aside from their absence being a negative indicator.
ebiester 13y ago
How long? Until one of the other languages (say... clojureScript) develops a large enough ecosystem and best practices around it to allow portions of code to be shared between the client and server. Oh, that also means that this given system has to reach critical mass. Oh, and since everything's server and client side, the compilation has to be rock solid and be easy enough to understand without reverting to javascript in frustration. I miss programming in Node, for nothing else that I hate having to keep up with so many paradigms and switch between the two. I only have X amount of hours to keep up. When I move back to heavy development in a language after even a few days off, there is friction. And as someone who writes both front end and back end code, it gets frustrating. I'm not as good as I could be, given the hours in the day I have.
JoshTriplett 13y ago
Threads are quite useful, and Javascript has them. However, if what you have is a computation you have the inputs to start now and won't need the result of for a while, lazy evaluation is a far more natural representation for that than a thread. Ideally, a language ought to have one representation for lazy evaluation, not a dozen; perhaps at some point a new standard will come out that standardizes some subset of lazy evaluation that has won out. Welcome to Javascript.
- zaphar 13y ago
  Javascript does not have threads. There are some Javascript environments that may or may not have threads in some form. Rhino, WebWorkers. But Javascript does not have a builtin concept of threads.
gfodor 13y ago
Yeah I just started working on a node project and can't help but feel like I have been transported to a time 40 years ago where my operating system didn't have a process scheduler and I have to do it all myself. I literally have to think about how asynchronous behavior works every 2 or 3 lines of code. It's sheer madness.
- nawitus 13y ago
  Your comment is ironic because C# implemented async in 2012. Async thinking takes a while to get used, but that doesn't mean it's more difficult than threads or mutexes and whatnot. In fact, I think it's simpler. You don't have to learn anything properly new if you already know how functions work.
  - gfodor 13y ago
    we haven't needed complicated threading on the web server for the last 20 years, because there is a natural isolation model that works most of the time: the request. node turns this on its head and says now you will need to think about concurrency always, even if you don't have a problem that naturally warrants it. that's the issue. if i want to write a goddamn shell script I need to think about concurrency. wat? are callbacks vs mutexes vs actors vs STM the best way to handle concurrency, that's debatable. but what's not debatable is that comparing concurrent vs non-concurrent code the latter is always easier for people to write and reason about, so you should tend to want to work in environments that are non-concurrent by default and let you focus just on the parts that you need to deal with concurrency.
    - nawitus 13y ago
      Okay, now I understand your critique. I don't think it's a very good point though: Node.js is not designed for extremely simple synchronous web servers (and neither is JavaScript).
      - caoilte 13y ago
        That's a generational perspective. For developers fresh out of college promises, async, continuations and so forth are the norm now.
spion 13y ago
Some of those "things" you counted are based on generators, which let you to do exactly what you want. Example: https://github.com/spion/async-compare/blob/master/examples/promises-bluebird-generator.js https://github.com/spion/async-compare/blob/master/examples/... With promises, generators are not really necessary -- especially if you add arrow function syntax into the mix. https://gist.github.com/spion/8254661 https://gist.github.com/spion/8254661 (Note: that particular method in doxbee has long since been refactored to something saner with better separation of concerns :)
dustingetz 13y ago
Promises are actually very simple - so fundamental that you can think of them as dual to function composition. By a dual to function composition i mean that you can mechanically convert async promise code to everyday synchronous function calls - including everyday exceptions. That means that literally the code can look exactly the same whether or not you are async or sync. The trick is you need either macros or monads to do this. (This is kind of the whole point of monads - you could write a piece of code by composing a few functions, and then make it asynchronous without breaking the function composition! No need to rewrite your code to use callbacks; the same code will just work.)
- gcr 13y ago
  Is this currently possible with NodeJS? Javascript has none of the following: - Macros - Monads - call/cc
  - jonsterling 13y ago
    Monads are not a language feature, they are a library feature.
    - gcr 13y ago
      Ok, true. They're certainly much nicer to use if you have nice syntax for them though. :)
    - dustingetz 13y ago
      Only true in the most pedantic sense. For monads to work, all your code needs to be pure (in the FP sense). So if you want to do anything substantial with monads in an imperative language, you need to rewrite the entire ecosystem of libraries. That would cost quite a bit more money than your average organization can afford ;)
      - jonsterling 13y ago
        Also has nothing to do with the language, mate. Your argument does nothing but agree with my point :)
davedx 13y ago
You mean like fibers? :) https://github.com/laverdet/node-fibers https://github.com/laverdet/node-fibers This is what Meteor uses on the server side, and it's a pleasure. Everything is suddenly synchronous.
gcr 13y ago
(Edit: Oops! Sorry, I see that you're already familiar with continuations, but let me explain them for those who aren't) Sometimes, fundamental switches in thinking are worth it. :) For example, continuations would be perfect here, and are a wonderful tool for simplifying "nested callback hell" type problems. But why should a web developer care about continuations? Consider Paul Graham's Arc challenge: Build a web page that presents a form to the user. After they submit the form, a new page presents them with a link. The link takes them to a third page where the submitted value is presented back to the user. The restriction is that the input field must not be passed in via the URL. Right now, the user must write in "continuation-passing style", which means threading callbacks, like this: function serve(req,res) { askUserForValue(req,res, function(value){ showLink(req, res, function(){ render(res, {"user-value": value}); }); }); } Or perhaps the user could somehow stuff the variable into a "session" library with three separate URL routes: function serve(req,res) { render the form with the POST action of /submit; } function submit(req,res, value) { req.session["uservalue"] = value; render the link that sends the user to /view; } function view(req,res) { render the value of req.session["uservaule"]; } This one is even more fragmented since you have three separate functions that model one user interaction flow! I certainly like it less than the callback example -- at least I can keep one in my head. We can do better though. Continuations let you write this: function serve(req, res) { var value = askUserForValue(); showLink(); render(res, {"user-value": value}); } If node.js supported continuations, the askUserForValue() function would respond with the form and suspend the serve function right then and there. After the user made a new HTTP call, the suspended askUserForValue() call would return to serve() and its return value would be placed into our variable. Then, the showLink() function could render the link and suspend the function again. Once the user makes another HTTP request, the function resumes, and so on. The server would have to store the state of the computation itself to disk so it could continue later. With continuations, we can pretend the user's web browser is the program counter. This frees my mind up. First, I don't have to keep callbacks in my head since there aren't any callbacks anymore. Second, stack traces work the way I expect because functions explicitly call each other. Third, if askForValue() throws an exception if the user inputs a string where I entered a number, no problem! Just wrap it in a try{} block that asks the user to enter a valid value, as if I'm writing a text program that calls readline(). Also note that this is different from just letting each user interaction be in its own thread! Consider: 1. Continuations should be serializable to disk. You wouldn't want blocked threads to just pile up as more users visit your website, and if your server crashes, you certainly wouldn't want to forget their state. 2. The browser "back" button would have to step a program backwards in time; something that is unsupported in a naïve thread-equals-user-interaction system. (E.g. how many of you have booked an airplane ticket with Travelocity/Expedia/whatever, pressed "Back", and then got a big error because the server didn't know how to step backwards through its user stack?) But the ability to save continuations to disk is the ability to save the current state of the program to disk, which makes stepping back in time easy: you can simply reload what was about to happen, just like savestates in a video game emulator. This is not a new idea. Many Common Lisp and Smalltack web frameworks work well with continuations and have done so for years (the first Seaside release was 2002, I'm not kidding you!) Racket, a great dialect of Scheme, ships with such a web framework out of the box. Here's my implementation of the Arc challenge in Racket. Note how the 'start' function is implemented: https://gist.github.com/gcr/ad116f565e105c8b2e0d https://gist.github.com/gcr/ad116f565e105c8b2e0d It's super easy to try this out and you don't need to fiddle with dependencies or anything. Just download Racket from http://racket-lang.org/ http://racket-lang.org/ , copy+paste the code into DrRacket's giant text box on the top, press Run, and your web browser will pop up with the example.
bbllee 13y ago
I find one of your premises very thin: browser implementors are implementing DOM promises, so promises were clearly not invented solely to make up for the difficulties of Node.
SimHacker 13y ago
The emperor may have no clothes, but what's so great about clothes? (If God meant people to be naked, they'd be born that way! Same goes for programming languages.) The naked Emperor JavaScript runs everywhere, and runs fast, and isn't going away. He's streaking at lightning speed all over the kingdom, everybody's seen him, and he doesn't stay still. The emperor is naked. Long run the emperor.

ilaksh 13y ago

I hope that everyone who writes JavaScript will do themselves a favor and try out ToffeeScript. This offshoot of CoffeeScript simplifies things. For example: if fs.exists! thefile e, datastr = fs.readFile! thefile, 'utf8' else datastr = 'not found' success = db.update! 'somekey', datastr res.end success So in that example even though there are three callbacks it looks like a normal synchronous flow.

lucaspiller 13y ago
I haven't heard of ToffeeScript before, but this is pretty much how Erlang works. Whenever you make a blocking call, all of the async stuff happens under the hood, so that you just deal with sequential code. Erlang's concurrency model deals with lots of lightweight processes. Rather than having callbacks it uses a message system. Whenever a process waits for a message it yields so other processes can be executed. In your example there would be a process separate from the main[0] process that handles the database connection. When you request the update to the database, you are just sending a message to the database connection process. This will receive the message and handle the update when it can[1], and once complete, send you a message back with the result of the update. During the waiting period the VM will continue executing other processes. No callbacks needed :) [0] Erlang concurrency is based upon the actor model. There isn't really a main process, but for the sake of explanation... [1] The database connection process will actually block while making the update, so if you have 10,000 processes calling this, you will have a bottleneck. As such you should have a pool of database connection processes, rather than just one.
- ilaksh 13y ago
  Yeah Erlang looks awesome and I should definitely try to learn it at some point. To me Node.js and ToffeeScript seem a lot more straightforward and have the advantage of the whole package system which I believe is more advanced, as well as the npm ecosystem with 54,000 packages. But anyway it would be good to actually learn Erlang and use it in some projects so that I can really evaluate it. I'm not sure but I think it might be a lot more resource efficient for one thing.
13y ago

OhHeyItsE 13y ago

JavaScript promises. Got it. Wait - crashing processes? Denial of service? Wha? Why would JavaScript?... Oh, this is about Node. Goodbye.

Patrick_Devine 13y ago

I'm not a js guy, but this concept sounds a lot like Twisted's (python) concept of "deferreds". The idea is that you "defer" caring about some asynchronous process, and then attach "callbacks" and "errbacks" to the "deferred". You essentially create a chain of callbacks or errbacks which gets called depending on if a callback is fired or an exception is raised. Overall it's a pretty great concept, but debugging can be a real pain.

domenicd 13y ago
Yup, from what I recall of history both Twisted Deferreds and JavaScript promises have their heritage in E's promises.
davedx 13y ago
Yup. jQuery: http://api.jquery.com/category/deferred-object/ http://api.jquery.com/category/deferred-object/
spion 13y ago
Debugging with Bluebird (mentioned in the article) is not a pain because it stitches stack traces across events for you when you run it in development mode :)

eldude 13y ago

The problem with discussions of promises vs ... is there's usually a great deal of misunderstanding about what problem promises are solving: enforcement of the callback contract. I could go on for hours about the subtleties in the conversation, but the gist of it boils down to this: Node callbacks have a strict but entirely unenforced (in userland) contract: The contract: * Callbacks should be called OAOO * Never throw (explicitly or implicitly) after origin tick (aka pass post-tick errors to callback) There is also an increasingly common informal contract: * Never throw ever (aka pass ALL errors to the callback) But there are a couple of issues that get in the way of successful enforcement of this contract: * When external libraries violate this contract (library A calls library B, but library B violates the contract in a way that makes it difficult for library A to easily maintain the contract) * Asynchronous errors * Crash-worthy exceptions Inevitably, this all continually comes back to error handling over and over again, with the core issue being that node.js has 2 divergent methods for error handling: exceptions (try/catch/throw) and error passing (callback(err)). Exceptions requires opt-in error handling (crash by default) while error passing requires opt-out error handling (crash on demand) "and never the twain shall meet." Promises attempt to resolve this by coercing all exceptions to error passing, but not all exceptions are safe to be passed. Specifically, any exception originating from core that is not an invalid argument exception (think 4XX vs 5XX class errors) cannot safely be caught / coerced / continued on. Also, not all errors passed to callbacks are even theoretically passable as the unofficial policy of core is that core callback errors are only distinguishable from exceptions in that they occur after the origin tick. In other words, they are equally crash-worthy and equally non-crash-worthy. So promises are an improvement because when used pervasively, they enforce the formal and informal callback contracts, while providing a continuable-like representation of the value that can be passed around. This benefit of contract enforcement is entirely independent of the specific API implementations. The stepup[1] library trivially accomplishes contract enforcement by integrating the async trycatch[2] module without using promises or requiring pervasive usage. I've used them both for years now in various professional projects, with trycatch in use here at LinkedIn. Additionally, stepup could easily return a continuable to support a passable value representation. Async generators will solve most of this though, allowing node.js to ditch the error passing for exception handling, but since Error subclassing and typed catches are basically not supported in node.js the language keeps getting in the way of a satisfactory complete solution. This is a whole other discussion. In conclusion, error handling in node.js is an undesigned mess of which the core contributors don't even seem to fully understand or be aware[3,4]. FWIW, node.js' crash on error design is a DoS liability, which spion addresses in the article, and which LinkedIn uses trycatch to avoid. [1] https://npmjs.org/package/stepup https://npmjs.org/package/stepup [2] https://npmjs.org/package/trycatch https://npmjs.org/package/trycatch [3] https://github.com/joyent/node/issues/5114 https://github.com/joyent/node/issues/5114 [4] https://github.com/joyent/node/issues/5149 https://github.com/joyent/node/issues/5149

nadaviv 13y ago

> if (err) return callback(err) > That line is haunting me in my dreams now. What happened to the DRY principle? iferr = (succfn, errfn) -> (err, a...) -> if err? then errfn err else succfn a... show_user = (id, cb) -> db.query ..., iferr cb, (user) -> ... FTFY, no promises needed. (not to say that promises don't have other advantages, but that specific complaint is easily fixable with an higher-order function to abstract it away) (JavaScript version: http://coffeescript.org/#try:%20%20iferr%20%3D%20(succfn%2C%20errfn)%20-%3E%20(err%2C%20a...)%20-%3E%0A%20%20%20%20if%20err%3F%20then%20errfn%20err%20else%20succfn%20a...%0A%0A%20%20show_user%20%3D%20(id%2C%20cb)%20-%3E%0A%20%20%20%20db.query%20foo%2C%20iferr%20cb%2C%20(user)%20-%3E%0A%20%20%20%20%20%20bar http://coffeescript.org/#try:%20%20iferr%20%3D%20(succfn%2C%...)

spion 13y ago
No its not. I address this immediately afterwards (below "But spion, why don't you wrap your calbacks?"), but basically, this simple wrapper makes stack traces unusable. Bluebird (and also Q) stitches together stack traces from multiple previous events. This is amazingly awesome.

rtpg 13y ago

Slightly off topic, but I've been working in Haskell, and recently realized how the laziness of the language means everything is basically asynchronous by default, but you can still write your code in a synchronous fashion. No real need for promises or callbacks because lazy values are the promises. Well, when using lazy IO, at least. I'd actually much prefer lazy values be put into Javscript than almost any other feature (real module support is another big one).

ilaksh 13y ago
See my comment below about ToffeeScript which compiles to JavaScript.
rapala 13y ago
Then again, Simon Peyton Jones has said that were he to make a new Haskell, it would have strict evaluation. As far as I know, lazy IO is considered a pain in the butt. That beeing said, laziness does have its merits and some ideas can be elegantly stated in a lazy language.
- rtpg 13y ago
  do you have a source on this? For me laziness is the single strongest thing going for Haskell.
  - rapala 13y ago
    It's from a retrospective on Haskell: www.cs.nott.ac.uk/~gmh/appsem-slides/peytonjones.ppt There seems to be two arguments against laziness. Firstly, it makes reasoning about performance hard. Secondly, forcing strict evaluation is tricky, maybe harder than it would be to do optional lazy evaluation in a strict language.
- audreyt 13y ago
  The default Lazy IO issue has been addressed in pipes-bytestring: http://www.haskellforall.com/2013/09/perfect-streaming-using-pipes-bytestring.html http://www.haskellforall.com/2013/09/perfect-streaming-using... The pipes ecosystem ( http://www.haskell.org/haskellwiki/Pipes http://www.haskell.org/haskellwiki/Pipes ) is working well to offer {effects, streaming, composability} without the resource leak issues introduced by lazy IO.

moron4hire 13y ago

I see stuff like: promise.when(function(value){ //blah blah blah }) .error(function(err){ //handle it }) and I think to myself I'm seeing text replacements: "promise.when(function(value)" = "try" ".error(function" = "catch" "})" = "}" So how is this actually different from the code we're supposedly too lazy to write or too stupid to figure out or something in the first place, thus requiring this slightly more verbose form? And where is this all going? Like, what's the right granularity of lines-of-code to when chains? If programmers are too lazy to try/catch, then why aren't we too lazy to when/then/error a gigantic block, or you know, not at all? And if we were to instead [w|t]hen/error every line, then I think about DRY and and it occurs to me the most likely next step is a language that "implicitly whens your code", that every line is written like regular procedural code, but automagically has "when" inserted in the middle. Or in other words, you know, continuations. But it occurs to me that the only reason any of this stuff is of any interest is because JavaScript, and therefore Node, still lacks real MT support, so you have to jump through these infinitely fractal spilling callback tangos to fake it. Web Workers was 2009, people. 5 YEARS MAN. Five years that we've been waiting to hear anything about having sane primitives for MT.

spion 13y ago
Nope, `doStuff().then(function(value)" != try` A closer representation would be `try { var value = doStuff()` which has similar verbosity. ES6 will have generators and "arrow" function syntax. Meaning: doThing().then(function(value){ //blah blah blah }) .catch(function(err){ //handle it }) becomes either: try { var value = yield doThing(); //blah blah blah } catch (err) { //handle it } or doThing().then(value => /* blah blah blah */) .catch(err => /* handle it */); meaning you can pick your paradigm :) Except that the functional one composes slightly better, perhaps. doThing().then(value => /* blah blah blah */) .catch(only(ParseError, err => /* handle it */)); vs try { var value = yield doThing(); //blah blah blah } catch (err) { only(err, ParseError); // argument repetition //handle it }
- moron4hire 13y ago
  Uhhh, you're not convincing me that you've said anything meaningful towards improving software quality.

pornel 13y ago

Promises have two awesome things: * With callbacks interface of the function hardcodes whether the function is strictly synchronous or may use any async code. Change of one little helper function from sync to async may have ripple effect throughout the entire codebase (you have to add callback to all its callers, and all their callers, and their callers...). Promises fix that. Sync and async functions take arguments and return values the same way, so when you change sync to async you only need to change source and consumer of the value, and nothing in between (e.g. same code can cache results of (memoize) promise-returning functions and sync functions). * With ES6 yield async calls look and behave almost like sync calls. No callbacks. You don't have to use weird error handling schemes — just use try/catch! You don't have to have a library to iterate over things — just use for()! A call that used to look like: use(compute(arg)) was twisted by async to be: compute(arg, use) Promises change it to: compute(arg).then(use) and yield changes it back to nearly original form: use(yield compute(arg))

hatchoo 13y ago

Aren't they made to be broken? (Sorry, couldn't help it)

SimHacker 13y ago

And what's wrong with reinventing basic programming-language design a million different ways? Are you complaining that it's too much like Lisp?