13 ms·

Disclaimer: I worked with Node.js full-time for about 14 months, ending about 9 months ago. I'm willing to admit that the following complaints may or may not be

• zaidf 10y ago
  I think the idea of having a powerful scripting language based on JS is a godsend in many ways. But like you, I also feel that async is a bad default for a language that has such widespread use. I love being able to use one syntax to write 95% of my code(instead of having to switch between JS / python / php etc.) but some of NodeJS's programming and ecosystem realities are annoying to say the least.
• jomamaxx 10y ago
  Your comments about the annoyingness of async are real. BUT Just because you want to do ABCDE does not mean it's feasible. You can access a file, read it, compute something, depending on that access another file, read it, write something via Http Post somewhere else. BUT the nature of i/o is such that the problem is 'intrinsic' - it's not a 'node.js' problem so much. Of course you know that you can do everything syncrhonously if you want, but then you get hangups and what not waiting for underlying resources, networking calls etc.. The nature of a distributed system is inherently async, and so we have to live with it. That said - maybe you have some examples of where async definitely should not be used? The other huge problem with Node.js is the vast, crazy expansion of tiny libraries, supported by ghosts, that change all the time. A simple lib can depend on a dozen others, with varying versions, everything changes almost daily. A small change somewhere can really jam things up. This is by far my #1 concern, although it can kind of be avoided if you're careful.
  • marcosdumay 10y ago
    > Just because you want to do ABCDE does not mean it's feasible. Every single language is able to do ABCDE, except for JavaScript. Yes, I guess that means it's impossible. > The nature of a distributed system is inherently async, and so we have to live with it. The nature of a distributed system is the one you impose onto it by design. Computer systems are not something that appear at nature and we just harvest. There are plenty of synchronous architectures for distributed systems, from distributed logical clock ones, for clock domain ones. People tend to avoid async ones anyway.
• formula1 10y ago
  Async isnt about ordered operations its about waiting. When you make a database call, you thread can sit there for 50 ms or do something useful. You can outsource rendering work to a child process. You dont have to let infinite loops block events.
  • echelon 10y ago
    This seems absurd to me. What exactly are we going to sit there and do while waiting on the data we need to fulfill a query? I work on services where any given endpoint may handle many thousands of requests per second. We don't care about a slight penalty to query the database, because these are very short, and our services return responses on the order of 100 ms. Maybe these are things you care about in a language like Javascript, but in something statically typed, they just don't seem like a problem.
    • mahyarm 10y ago
      In a client world, you definitely care about using all of your resources and never halting the rendering/main thread. There is often many things going on at once and you usually need to keep main thread halts under 16ms
    • mplewis 10y ago
      You could use the same thread and process to do something else instead of having to spin up another thread or process. If you're running a web server, you can handle another request, then come back to the first one when the data arrives.
      • jtrtoo 10y ago
        So shouldn't the only question ever be: * Would we rather burn computing resources (i.e. on new threads) or calendar time+human resources (i.e. on code refactoring)? Followed closely by: * And does it even matter for our situation? (i.e. since for many situations getting every last drop of performance out of a system isn't the real obstacle; it's making it do something useful for the end-user or business)
• lacker 10y ago
  If you think promise-based code with a lot of "then" is annoying, perhaps ES7 async/await will make you happier.
  • nurettin 10y ago
    This is exactly what I was about to write. Also, the amount of "callbacks, that's how it's supposed to be" and "hardware is like that" and "programming is hard, deal with it" kind of replies for a readily solved problem are staggering.
    • lootsauce 10y ago
      I don't know about others but the imposition of try catch for error handling in async/await seems to cut against the syntax gains over promises.
      • nurettin 10y ago
        If by promises you mean the `some.fun(had).catch(had, function(){handle stuff here})` syntax compared to async/await, that is actually a very fair point and it does make two syntaxes similarly verbose. However, it is still easier to follow a ladder of conditional statements using async/await compared to escher-style stairs of callbacks inside callbacks.
    • edwinjm 10y ago
      Interesting. In which language is this "readily solved"?
      • iends 10y ago
        Erlang, Elixir, & Go have nicer models.
      • nurettin 10y ago
        ES7 http://rossboucher.com/await/#/ http://rossboucher.com/await/#/
• athenot 10y ago
  > In the real world, almost everything is synchronous, and only occasionally do you really want async behavior. From an Ops perspective, I disagree. At a systems level, everything already is async, whether you like it or not. So then the options are either to embrace the admittedly hectic nature of async, or pretent that things are synchronous and impose those constraints to maintain a logical purity from a programmer's standpoint. For example, in a synchronous setup, it's perfectly logical to service an incoming http request with a few database queries followed by some more logic and some more DB queries or API calls; wrap all the result and send it back. The same in node.js forces the code author to think about those initial DB queries (which may run simultaneously), about how to roll up the result when they are all done and pass it to callbacks. But while thinking about those callback interactions, there's a motivation to reduce the number of steps, or to factor them out into their own module or microservice. This promotes thinking at a systems level instead of just at the level of "1 request serviced by 1 handler". Perhaps that big request can be broken up into several API calls and the web page can load those asynchronously. Perhaps the smaller microservices end up building up less memory usage per request (or spread it out in a way that makes clustering more efficient). One may see async as a hurdle in the way of logical abstraction, or as a way to align thinking with the realities of network programming.
  • knucklesandwich 10y ago
    The advantage you're describing is concurrency, which doesn't necessarily require an asynchronous/event-driven programming model. Synchronous concurrency is built around threads to achieve concurrent execution (i.e. instead of waiting on events for several requests to finish in parallel, you fork off a thread for each to process each request synchronously). Node came of age about a decade after epoll was introduced, when not having access to nonblocking IO was considered a big liability for a couple of dominant web programming languages, and they built their concurrency model around the semantics of epoll. However, there are languages like Haskell, Erlang, and Go that IMO did the right thing by building a synchronous programming model for concurrency and offering preemptable lightweight processes to avoid the overhead associated with OS thread per connection concurrency models. These languages offer concurrent semantics to programmers, yet still are able to use nonblocking IO underneath the covers by parking processes waiting on I/O events. It's not the right tradeoff for every language, particularly I think lower level languages like Rust are better off not inheriting all the extra baggage of a runtime like this, but for higher level languages I think its probably the most convenient model to programmers.
• marssaxman 10y ago
  Well.... no, not really. Processors really are fundamentally asynchronous. They are state machines which react when external signals impinge upon them. Synchronous behavior is a useful illusion, a convenient reasoning tool for some kinds of tasks, and our traditional notations for describing computational processes are all based on this model of how computing might be done - but it hasn't actually been done that way since the batch-processing era. I learned this as I dove into low-level hardware work - drivers and embedded systems. When you're dealing with hardware, life is fundamentally driven by interrupts. Processors spend most of their time asleep. When something happens, the signal level of an input pin changes, and the processor reacts: it wakes up, it starts executing the interrupt handler for that signal, it does things, it sends messages, and then - its job done - it goes back to sleep. So what, we're talking about normal computers here, you say, surely they're different, right? But no, they really aren't; it's the same process of signal, interrupt, handler, return, all the time. Every context switch, every packet receipt or transmit completion, every timer tick, all of it, it's all driven asynchronously; the system is a hierarchy of event handlers waiting for their opportunity to respond to some external signal. If you're trying to build an IO system, then, the synchronous model is an expensive illusion. You can force it, but it isn't reality. You spend time waiting, you spend time blocking, you build complex APIs that pretend to poll arrays of connections in parallel; and all you're doing is forcing that illusion of synchrony onto a fundamentally asynchronous world. It's backwards, all right, but it's backwards because our traditional way of describing computation is backwards, especially the languages which follow the imperative tradition. Functional languages have gained popularity in part because the mental model works better for asynchrony; instead of describing a series of steps to follow, and treating the interrupt as the exception, you describe the processes that should be undertaken in certain circumstances; you effectively construct a state-machine instead of describing a process, and then you leave that state-machine to be poked and prodded by whatever external signals come along to drive it.
  • tigershark 10y ago
    Mainstream processors are synchronous. For this reason there is a global clock needed to synchronise all (most) the chip parts. I remember there was some research on asynchronous CPUs but I don't think it ever yielded anything commercial. EDIT: Wikipedia link: https://en.m.wikipedia.org/wiki/Asynchronous_circuit https://en.m.wikipedia.org/wiki/Asynchronous_circuit What you are saying is that they react to interrupts asynchronously, but this has nothing to do with the synchronous nature of a CPU. Moreover I think that if you have a CPU that doesn't respect the barriers and reorders the instructions to execute them in parallel, then it's a bugged CPU.
• ohyes 10y ago
  I think what this really gets at is that callbacks are a shitty concurrency primitive. They certainly make sense from an implementation perspective (I've got this interpreter and I want to do concurrency... I know, I'll just use it to trampoline callbacks). But callbacks are too low level to reason about easily to some extent, it's kind of like writing everything as a series of goto statements. Sure you can do it, but good luck following it. The advantages of asynchronous execution are lost. This isn't limited to nodejs, I've experienced similar problems with Python twisted... Channels (such as in clojure/core.async) or lightweight processes and messaging (such as in erlang/beam), are much more enjoyable to work with. This shouldn't really be a shock, however, they're higher level abstractions based on real computer science research by people who really thought about the problem.
  • azernik 10y ago
    On the subject of nice higher-level concurrency abstractions, I'm very taken with the current fad for Promises/Futures (a la ES6 or Scala), which are basically a translation of the Haskell IO monad to plain English.
    • gclaramunt 10y ago
      Isn't closer to the Async monad?
  • int_19h 10y ago
    Callbacks are ugly when you have to deal with them directly. But a little bit of syntactic sugar to capture the current continuation as a callback - such as C# await, or the now-repurposed yield in Python - makes it very easy and straightforward to write callback-based async code.
    • davidjnelson 10y ago
      Babel and regenerator let you use await in es/js too.
    • flamedoge 10y ago
• savanaly 10y ago
  Maybe it's because I learned to write code in javascript (browser and node) -- does that make me an async native? -- that I feel the reverse of you. Synchronous just doesn't feel right, and I'm happy with the way node is assumes async. I'm one of those who finds the new Await syntax sort of unsettling at a gut level.
• Illniyar 10y ago
  Doing synchronious is much clearer and easier, most of the async patterns (like yield/await and promises) really just try to simulate a sync paradigm. But there is one great reason why node's async first mentality is superior- when you want to do async in node, you don't have to worry that some library you are using is going to lock up your thread. In any other language, you have to painstaikingly make sure everything you use isn't doing sync, or try and monkeypatch all io operations (python has something like that). Frankly, for me, when I'm dealing with webservers I find myself needing to use async quite a lot.
  • icebraining 10y ago
    In any other language No; for example, in Go, every time the code does IO, the scheduler will re-assign another goroutine, and they asynchronously get back to the other when the IO finishes. It doesn't need any special support by the library.
    • Illniyar 10y ago
      That sounds like an implicit async/await, which sounds rather awesome. I haven't really gotten around to trying Go yet, but there are definitely great things going on there.
      • zigzigzag 10y ago
        It's just green threading, which has been around for a long time. You can run millions of threads on a single machine with modern Linux kernels if you either have lots of memory or simply minimize your stack sizes and usage. The kernel doesn't care much. You don't really need any special language features. The main reason fibers/green threads/goroutines/etc have become popular lately is that common language runtimes and standard libraries don't resize thread stacks on the fly and tend to like very deep call stacks, so minimising your memory usage can be rather hard.
        • soulbadguy 10y ago
          > he main reason fibers/green threads/goroutines/etc have become popular lately is that common language runtimes and standard libraries don't resize thread stacks on the fly and tend to like very deep call stacks, so minimising your memory usage can be rather hard. Green-thread, coroutine have been around for a long time, the recent popular is probably due to increase prevalence of people trying to solve the C10K problem. Green thread, are more efficient both in term of memory consumption and context switching than kernel thread (even more so with compiler support). The linux kernel might be able to handle millions of threads, but thruth is one can do it much more efficiently in userland
• bitwize 10y ago
  1) All data has a type. 2) All I/O (if you're doing it right) is asynchronous. Node does things the right way around; it's just a shame that Linux (unlike Windows) doesn't have async I/O baked into the heart of the OS and well supported with useful kernel primitives. Synchronous I/O by default means you're wasting your CPU cores. When running at Web scale, you ALWAYS want to leverage async I/O in order to not have the CPU idle blocking on some I/O operation, and you don't necessarily want the synchronization and memory overhead of multiple threads. That's why Node is designed the way it is. By using the 'co' library and generators you can write async code almost as if it were sync. It works like the inline-callbacks feature of Twisted.
  • icebraining 10y ago
    Synchronous I/O by default means you're wasting your CPU cores. But that's the thing: synchronous code doesn't mean synchronous execution. The language/platform should be able to make it async for you, without forcing you to re-shape your code. And it's not a rare system: even plain old threads work like that - in fact, they're often a good solution: https://www.mailinator.com/tymaPaulMultithreaded.pdf https://www.mailinator.com/tymaPaulMultithreaded.pdf
    • dahauns 10y ago
      The code itself is not the problem. Data, specifically mutable state and dependencies are. Relying on the platform to figure those out for efficient async execution hasn't worked out so well in the past...
  • jcrites 10y ago
    > Synchronous I/O by default means you're wasting your CPU cores When a process becomes blocked (like due to synchronously waiting for IO to complete), the kernel will context switch away to another runnable process. The process that becomes blocked essentially goes to sleep, and then "wakes up" and becomes runnable again once the operation completes. The CPU is free during this time to run other processes. If you need to do multiple different of these things concurrently, then you can run multiple processes. Writing a single process with async code won't make that process faster. To do more things at the same time you can run multiple processes. Context switching between different processes is what the kernel scheduler is designed to do, and it does so very efficiently. There isn't much overhead per thread. If I recall correctly, Linux kernel stacks per thread are 8 kilobytes (with efforts under way to reduce that further [4] - also discussed in [1]), and the user stack space is something the application can control and tune. The memory use per thread needn't be much. Using all available cores to perform useful work is the most important thing to achieve in high-throughput code, and both async and sync can achieve it. Async doesn't become necessary for high performance unless you're considering very high performance which is beyond the reach of NodeJS anyway [2]. Asynchronous techniques win on top performance benchmarks, but typical multithreaded blocking synchronous Java can still handily beat async NodeJS, since Java will use all available CPU cores while Node's performance is blocked on one (unless you use special limited techniques). There's some good discussion about this in the article and thread about "Zero-cost futures in Rust" [1]. The article includes a benchmark which compares Java, Go, and NodeJS performance. These benchmarks suggest that the other tested platforms provide 10-20x better throughput than Node (they're also asynchronous, so this benchmark isn't about sync/async). Folks might also be interested in the TechEmpower web framework benchmark [2]. The top Java entry ("rapidoid") is #2 on the benchmark and achieves 99.9% of the performance of the #1 entry (ulib in C++). These frameworks both achieve about 6.9 million requests per second. The top Java Netty server (widely deployed async/NIO server) is about 50% of that, while the Java Jetty server, which is regular synchronous socket code, clocks in at 10% of the best or 710,000 R/s. NodeJS manages 320,000 R/s which is 4.6% of the best. In other words, performance-focused async Java achieves 20x, regular asynchronous Java achieves 10x, and boring old synchronous Jetty is still 2x better than NodeJS throughput. NodeJS does a pretty good job given that it's interpreted while Java is compiled, though Lua with an Nginx frontend can manage about 4x more. I agree that asynchronous execution can provide an advantage, but it's not the only factor to consider while evaluating performance. If throughput is someone's goal, then NodeJS is not the best platform due to its concurrency and interpreter handicap. If you value performance then you'll chose another platform that offers magnitude better requests-per-second throughput, such as C++, Java, Rust, or Go according to [1] and [2]. Asynchronous execution also does not necessarily require asynchronous programming. Other languages have good or better async support -- for example, see C#'s `await` keyword. [3] explores async in JavaScript, await in C#, as well as Go, and makes the case that Go handles async most elegantly of those options. Java has Quasar, which allows you to write regular code that runs as fibers [5]. The code is completely normal blocking code, but the Quasar runtime handles running it asynchronously with high concurrency and M:N threading. Plus these fibers can interoperate with regular threads. Pretty gnarly stuff (but requires bytecode tampering). If async is your preference over Quasar's sync, then Akka might be up your alley instead [6]. > By using the 'co' library and generators you can write async code almost as if it were sync. For an interesting and humorous take on the difficulties of NodeJS's approach to async, and where that breaks down in the author's opinion, see "What color is your function?" [3]. [1] https://news.ycombinator.com/item?id=12268988 https://news.ycombinator.com/item?id=12268988 [2] https://www.techempower.com/benchmarks/#section=data-r11&hw=peak&test=plaintext https://www.techempower.com/benchmarks/#section=data-r11&hw=... [3] http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... [4] https://lwn.net/Articles/692208/ https://lwn.net/Articles/692208/ [5] http://docs.paralleluniverse.co/quasar/ http://docs.paralleluniverse.co/quasar/ [6] http://akka.io/ http://akka.io/
• adrianratnapala 10y ago
  With due caution about @marssaxman's point that software is fundemantally interrupt driven, I will agree with this. I work an a real-time system that is controlled by RPC from a desktop. We claim we want to keep the real-time code small and simple. Yet I find people pushing complexity down onto the RT system even when the real-time guarantees aren't needed because the PC code responds to it using a callback driven async model. It's easier to put a non-real time for loop into the RT thread than to emulate a for loop on the PC.
• regularfry 10y ago
  NPM would be hugely improved if: 1) `npm install` on two different machines at the same point in time was guaranteed to install the same set of dependency versions. 1b) shrinkwrap was standard. 2) it was faster. 3) you could run a private repo off a static file server.
  • jdlshore 10y ago
    Check in your modules.
• metafunctor 10y ago
  In the past, I was involved with building a Scheme interpreter which was automatically asynchronous, without need for manually handling it with callbacks. Basically, it was lightweight threads which would yield execution to other threads whenever waiting for I/O, or explicitly yielding. It allowed for a very straightforward linear programming style, no callback hell. Coupled with a functional programming style, there was rarely a need for mutexes or other synchronization between threads. When a thread yielded, the current continuation was captured and resumed when the thread was ready to continue. At the bottom of it all was a simple event loop where the Scheme interpreter dispatched I/O and timeouts. Scheme happens to have support for continuations built in to the language, so the implementation was actually quite simple.
  • ambrop7 10y ago
    I've built something similar but a completely new programming language with backtracking as the major feature. The intended use is dynamic OS configuration especially network interface configuration. The interpreter is single-threaded but there is simple concurrency between the "processes", that is a process yields when it has nothing more to immediately do. Old online demo build through Emscripten (it misses many new features such as functions): https://rawgit.com/ambrop72/badvpn-googlecode-export/wiki/emncd.html https://rawgit.com/ambrop72/badvpn-googlecode-export/wiki/em... Docs currently still dying on Google Code archive: https://code.google.com/archive/p/badvpn/wikis/NCD.wiki https://code.google.com/archive/p/badvpn/wikis/NCD.wiki Source: https://github.com/ambrop72/badvpn https://github.com/ambrop72/badvpn
  • umanwizard 10y ago
    > Basically, it was lightweight threads which would yield execution to other threads whenever waiting for I/O, or explicitly yielding. It allowed for a very straightforward linear programming style, no callback hell. How is this different from a normal multithreaded C program on a POSIX kernel? When I call read(), my thread is suspended until the data comes back, and another thread can run.
• jroseattle 10y ago
  > The NPM stuff... well, I think all ecosystems have their pros and cons. I'm not a huge fan of NPM, but it does the job for the most part, and I'm curious as to how people would actually improve it, rather than just complain about it all the time. I don't really have any good ideas (knowing nothing about how package management actually works under the hood). - Remove the Couchbase dependency - Make login authentication a plugin or API-based - Integration with Github/Gitlab for automated publishes to name a few.
• jupp0r 10y ago
  The universe is inherently concurrent. It only makes sense for programming languages (which are used to model problems in this universe) to follow along.
• amelius 10y ago
  Yes, and on top of that, it is asynchronicity without real concurrency. Imagine one part of the code doing a long CPU-bound computation; the rest has to wait! It's like the worst of two worlds.
• catnaroek 10y ago
  > make pretty much everything async. This isn't intrinsically bad. It just makes Node.js biased towards use cases that benefit from asynchronous I/O, which is a perfectly valid design choice. Sensible programmers already know that computationally intensive tasks that benefit from actual parallelism are best served by a different tool. What's absolutely horrible about Node.js is how they make everything asynchronous: explicit continuation passing everywhere. In other words, the user is responsible for manually multiplexing multiple logical threads through a single-OS-threaded event loop. If it sounds low level, it's because it is low-level. A high-level programming language ought to to better than this. Other languages get this right: Erlang, Go, Clojure, Scala, Haskell, Rust, etc.
• sesm 10y ago
  Everything is async in node, because doing sync operations in event loop is terribly slow.
  • zigzigzag 10y ago
    No. That's nonsense. Everything is async in node because node is based on the V8 Javascript engine which is not thread safe. V8 is not thread safe because it was designed for web browsers, and browser rendering engines are also not thread safe. People have tried to retcon node's choices as being based on some sort of deep technical justification, but that's not the case: its limits are the limits of the material it's made of, not a specific design choice. If you look at more advanced VMs like the CLR or JVM then they support all modes of operation: synchronous/blocking (which is often what you want), synchronous/blocking with many threads, async with callbacks, async with continuations ... node only supports one. How is that better?
    • edwinjm 10y ago
      JavaScript was explicitly designed to use one thread and prevent all kinds of concurrency problems. It requires getting used to, but it's very powerful. Look at this page and tell me the JVM is better: https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/package-summary.html https://docs.oracle.com/javase/7/docs/api/java/util/concurre...
      • zigzigzag 10y ago
        Javascript was never explicitly designed to use one thread: it used one thread because Netscape used one thread. This is back in the 1990s when multicore on the desktop didn't exist and JS execution speed was a non issue. The page you linked to shows a full and complete library of concurrency tools, which you don't have to use if you don't want to. How is this evidence of being worse?
      • dahauns 10y ago
        Sure, powerful. As long as you're mainly I/O-bound and shared-nothing. Oh, and your link actually DOES show that the JVM is better. It can give you everything node.js has and then some. If you want to see something like node.js for grown-ups, take a look at Vert.x or Akka (& Frameworks based on it), for example. (Sorry for the snark, it's been a long day...)
• hacknat 10y ago
  I was a node developer back when it hadn't been packaged yet (yes, I'm a hipster). I left a few years ago for many reasons, including the fact that I think the community sucks. I agree with everything you say, and I want to add to it. People compare node to things like Go, PHP, Python, etc, which is a mistake. As someone who had to write a fairly complex native module for node that a lot of people still use (5000 downloads this month, according to npm), nodeJS is a framework not a language. Node is not in control of the underlying code that implements the ES spec. Node is a bunch of C/C++ middleware wrapping V8 and implementing various server and process utilities. That is all it is. Why is this important? Because it biases the community towards piling on feature after feature and moving at an insane pace, sometimes in ways that are outside of the community's control. Node is biased against stability, because they feel the need to keep up with V8. Counter this with Go, which has been frozen as a language for 8 releases now, with each release fine tuning the stability of the compiler/language, runtime, and libs. That native module that I mentioned earlier, I have a bunch of open requests to make it work with nodeJS 6 (no idea what this is), which is the fourth breaking V8 interface change I've had to deal with. The native module clocks in at >2000 lines of C++. I implemented the same library in Go and it clocks in at 500 lines. I've only had to modify the Go version of this library once, and it was to take advantage of a new feature (i.e. I didn't have to update it). Node's continual quest for features is going to keep it in the niche web development space it has come to dominate. There is simply no way I would architect a system with such an unstable platform.