4 ms·

Node.js brings callback hell to the serverside world of threads and we're supposed to be grateful? It's like you don't know the history of computing. Callback-b

by sreque 9y ago

Node.js brings callback hell to the serverside world of threads and we're supposed to be grateful? It's like you don't know the history of computing. Callback-based code was the original programming model in UNIX dating back decades! Threaded code came about because it's far easier to write it than callback-based code. The ONLY reason to do callback-based code is for performance reasons: for network-IO-heavy apps there is less OS-level overhead from managing thousands of connections with a few threads using the OS-provided non-blocking I/O operations than managing a connection per thread. Even then, if you actually want to do better than connection-per-thread, you have to avoid the POSIX API for non-blocking I/O and use whatever OS-specific API is available for better performance (epoll, kqueue, or completion ports). The greatest joke of all though is that Node.js is dirt slow because Javascript is dirt-slow. Yes, you may be able to invent some microbenchmark where Node.js looks good, but the only reason it does in that benchmark is because the Node.js app is spending 99.9% of it's time executing code in a C library. Any real-world Node.js app is going to be slow because Javascript in general, even on V8, is anywhere from 10-100x slower than C or even well-written Java. And worst of all, Node.js is single-threaded! If you want to scale onto all your cores you have to go multi-process, which is an incredible pain in the general case. So, you switch to a runtime and framework that forces a more difficult way of programming on you (callback hell) but which is better for general networking performance, but you use a dirt-slow, single-threaded runtime while doing so. You are basically ending up with the worst of both worlds, and any decent development team will run circles around you in performance and productivity with just about any other backend stack.

tmp124567 9y ago
You don't know what you are talking about. Code some simple server on c using epoll I think you could learn a lot from that. And JS has async await now, so no callbacks.
- sreque 9y ago
  I have written servers in C with epoll that were used in production, thank you. And yes, I'm aware of async/await and its limitations. State-machine-style transformations from blocking-looking code to non-blocking code are definitely helpful, but in the end still fall short in terms of usability to a full blocking style of programming. To even begin to approach the ease-of-use of blocking-style programming you need something akin to delimited continuations in your programming language.
WA 9y ago
So if I don't like Java and C, what's your suggestion I should use instead to get the best of both worlds and not the worst?
- sreque 9y ago
  C# is fairly efficient while still having managed memory. And, of course, golang was designed to hit the sweet spot of automatic memory management, decent performance, and non-blocking I/O. Beyond that, there's things like Erlang and Elixer, which I am less familiar with. My understanding is that the Erlang VM is significantly slower than JVM or CLR implementations but generally faster than scripting language runtimes.
- jswizzy 9y ago
  Any language that uses the actor model for concurrency which is basically every language created in the last decade. Elixir, GO, Akka framework for Java and C#, Pony, Dart, Erlang, ELM, SCALA, ...
  - bonesss 9y ago
    F# with it's built in actor model, robust OCaml based syntax, great support for functional Akka[.net] actors, simplified concurrency and asynch programming support, and very own ELM implementation is a great contextual answer for anyone who wants to be on the front lines of lightweight non-blocking concurrent & parallel web development in .Net land.
- lackbeard 9y ago
  Go, Erlang, Elixer, Haskell, Clojure, Scala.
ralusek 9y ago
I should definitely make a macro at this point: Callback paradigm has not been used in node development for 3-4 years at this point. return someHttpCall() .then(response => someOtherHttpCall(response)) // Do 2 async operations in Parallel: .then(response => Promise.join( someCacheThing(response), someDBThing(response) )) .spread((cacheResponse, dbResponse) => sendBackToClient()); This isn't callback hell. Without creating or managing threads, I have established serial as well as an example of branched concurrent logic that is entirely nonblocking. Every promise is handled as a try catch, so error handling can be done at the end of the chain with a single `.catch`. Clustered express application puts the performance well within what you would expect from Go/Phoenix, all of which are operating at an order of magnitude above Python/Ruby.
- sreque 9y ago
  I'm well aware of promises, thank you. I'm also well aware that there's very little difference in reality between a promise-based style of coding and a callback one. You still lose stack continuity, it's still much harder to chain operations that hop stacks, it's still a nightmare to debug compared to stack-based programming, etc. If you've ever actually implemented a promise it's even easier to see how little promises actually get you. Given a callback-based API, it's absolutely trivial to wrap it in a Promise-based API. Fundamentally this is because promises give you very little in the end on top of callbacks beyond some syntactic sugar.
  - ralusek 9y ago
    First of all, I have absolutely implemented promises from scratch, and it IS trivial. However, "callback hell" is a reference to writing code with deeply nested callbacks, particularly with manual error handling at each level, both of which promises completely negate. If you want to change the topic from "callback hell" to debugging across stacks, then we can do that. There was some work with "domains," since moved to AsyncListener/Continuation Local Storage to keep context across stack calls, but I don't disagree that improvements in this department are the place that NodeJS paradigm would currently benefit from most.
brondeau 9y ago
As with any tool, there is a right and wrong way/time to use it. Node.js would not be the monster success and influencer it is today if it was the 'worst of both worlds'. For example, using a single threaded environment can solve so many issues that are present in a multi-threaded environment, such as reducing possibility of race conditions (though you can still write crappy single threaded code that introduces race conditions). When used in the right way, it can be and is best of class. In other instances, it is a boat anchor. Just my 2 cents
disordinary 9y ago
V8 is not dirt slow. You may dismiss benchmarks where Node looks good as "invented" but there are countless real world examples of people moving from other languages to node and experiencing a huge speed increase. I know of one startup that had a fortune 500 company sign up to their service and their dotnet APIs couldn't hold the load and the cost of scaling them wasn't sustainable. They rewrote the hardest hit APIs in node over a weekend and were shocked at the improvements. I also know another startup that had a similar experience, they moved off of Ruby to node and were shocked at the speed increase that they were getting. They couldn't be competitive with their competition with their old codebase. Despite what I just said above speed isn't necessarily the most important factor. Infrastructure is cheap and developers are expensive. If I'm asked what the best programing language or platform is my answer is usually the language and platform that your team is most productive with. Only a few companies get successful enough that speed is the deciding factor in platform choice, if we say that productivity is the most important factor then the JS/Node combination is right up there as one of the more understood and productive systems.
- deleted 9y ago
  [deleted]
- jonnycoder 9y ago
  You are spot on as I have been developing in node for the past 2 months. Coming from doing python and c# in the past, I picked up the asynchronous nature + promises and their "gotchas" within a few days of writing and rewriting an activemq module. All of our other requirements/stories have been developed at such a high speed relative to my past projects. Combined with docker and a QA team that writes automated tests with postman and protractor, our continuous integration is at a high maturity level.

by sreque 9y ago

tmp124567 9y ago

You don't know what you are talking about. Code some simple server on c using epoll I think you could learn a lot from that. And JS has async await now, so no callbacks.

sreque 9y ago
I have written servers in C with epoll that were used in production, thank you. And yes, I'm aware of async/await and its limitations. State-machine-style transformations from blocking-looking code to non-blocking code are definitely helpful, but in the end still fall short in terms of usability to a full blocking style of programming. To even begin to approach the ease-of-use of blocking-style programming you need something akin to delimited continuations in your programming language.

WA 9y ago

So if I don't like Java and C, what's your suggestion I should use instead to get the best of both worlds and not the worst?

sreque 9y ago
C# is fairly efficient while still having managed memory. And, of course, golang was designed to hit the sweet spot of automatic memory management, decent performance, and non-blocking I/O. Beyond that, there's things like Erlang and Elixer, which I am less familiar with. My understanding is that the Erlang VM is significantly slower than JVM or CLR implementations but generally faster than scripting language runtimes.
jswizzy 9y ago
Any language that uses the actor model for concurrency which is basically every language created in the last decade. Elixir, GO, Akka framework for Java and C#, Pony, Dart, Erlang, ELM, SCALA, ...
- bonesss 9y ago
  F# with it's built in actor model, robust OCaml based syntax, great support for functional Akka[.net] actors, simplified concurrency and asynch programming support, and very own ELM implementation is a great contextual answer for anyone who wants to be on the front lines of lightweight non-blocking concurrent & parallel web development in .Net land.
lackbeard 9y ago
Go, Erlang, Elixer, Haskell, Clojure, Scala.

ralusek 9y ago

I should definitely make a macro at this point: Callback paradigm has not been used in node development for 3-4 years at this point. return someHttpCall() .then(response => someOtherHttpCall(response)) // Do 2 async operations in Parallel: .then(response => Promise.join( someCacheThing(response), someDBThing(response) )) .spread((cacheResponse, dbResponse) => sendBackToClient()); This isn't callback hell. Without creating or managing threads, I have established serial as well as an example of branched concurrent logic that is entirely nonblocking. Every promise is handled as a try catch, so error handling can be done at the end of the chain with a single `.catch`. Clustered express application puts the performance well within what you would expect from Go/Phoenix, all of which are operating at an order of magnitude above Python/Ruby.

sreque 9y ago
I'm well aware of promises, thank you. I'm also well aware that there's very little difference in reality between a promise-based style of coding and a callback one. You still lose stack continuity, it's still much harder to chain operations that hop stacks, it's still a nightmare to debug compared to stack-based programming, etc. If you've ever actually implemented a promise it's even easier to see how little promises actually get you. Given a callback-based API, it's absolutely trivial to wrap it in a Promise-based API. Fundamentally this is because promises give you very little in the end on top of callbacks beyond some syntactic sugar.
- ralusek 9y ago
  First of all, I have absolutely implemented promises from scratch, and it IS trivial. However, "callback hell" is a reference to writing code with deeply nested callbacks, particularly with manual error handling at each level, both of which promises completely negate. If you want to change the topic from "callback hell" to debugging across stacks, then we can do that. There was some work with "domains," since moved to AsyncListener/Continuation Local Storage to keep context across stack calls, but I don't disagree that improvements in this department are the place that NodeJS paradigm would currently benefit from most.

brondeau 9y ago

As with any tool, there is a right and wrong way/time to use it. Node.js would not be the monster success and influencer it is today if it was the 'worst of both worlds'. For example, using a single threaded environment can solve so many issues that are present in a multi-threaded environment, such as reducing possibility of race conditions (though you can still write crappy single threaded code that introduces race conditions). When used in the right way, it can be and is best of class. In other instances, it is a boat anchor. Just my 2 cents

disordinary 9y ago

V8 is not dirt slow. You may dismiss benchmarks where Node looks good as "invented" but there are countless real world examples of people moving from other languages to node and experiencing a huge speed increase. I know of one startup that had a fortune 500 company sign up to their service and their dotnet APIs couldn't hold the load and the cost of scaling them wasn't sustainable. They rewrote the hardest hit APIs in node over a weekend and were shocked at the improvements. I also know another startup that had a similar experience, they moved off of Ruby to node and were shocked at the speed increase that they were getting. They couldn't be competitive with their competition with their old codebase. Despite what I just said above speed isn't necessarily the most important factor. Infrastructure is cheap and developers are expensive. If I'm asked what the best programing language or platform is my answer is usually the language and platform that your team is most productive with. Only a few companies get successful enough that speed is the deciding factor in platform choice, if we say that productivity is the most important factor then the JS/Node combination is right up there as one of the more understood and productive systems.

deleted 9y ago
[deleted]
jonnycoder 9y ago
You are spot on as I have been developing in node for the past 2 months. Coming from doing python and c# in the past, I picked up the asynchronous nature + promises and their "gotchas" within a few days of writing and rewriting an activemq module. All of our other requirements/stories have been developed at such a high speed relative to my past projects. Combined with docker and a QA team that writes automated tests with postman and protractor, our continuous integration is at a high maturity level.