5 ms·

I went from a company that used Elixir in the backend to one that uses Nodejs. I had gone in neutral about Nodejs, having never really used it much. These pro

• winrid 2y ago
  The problem with Node is observability. They've optimized away observability to where it's hard to find performance problems compared to the JVM to Beam.
• mm263 2y ago
  I'd appreciate an in-depth write-up about deficiencies you found in Node and how Erlang fixes them
  • hosh 2y ago
    Starting with the most general: Nodejs suffers in the same way that other async systems do -- the lack of preemption means that certain async threads can starve other async threads. You can see this on GUI desktop apps when the GUI freezes because it wasn't written in a way to take that into account. In other words, the runtime feature that Nodejs is the most proud of and markets to the world as its main advantage does not scale well in a reliable way. The BEAM runtime has preemption and will degrade in performance much more gracefully. In most situations, because of preemption (and hot code reloading) you still have a chance for attaching a REPL to the live runtime while under load. That allows someone to understand the live environment and maybe even hot patch the live code until a the real fix can run through the continuous delivery system. I'm not going to go into the bad Javascript syntax bloopers that still haunts us, and only partially mitigated by Typescript. That is documented in "Javascript: The Good Parts". Or how the "async" keyword colors function calls, forcing everything in a call chain to also be async, or forcing you to use the older callbacks. Most people I talk to who love Typescript don't consider those as issues. The _main_ problems are: 1. Async threads can easily get orphaned in Nodejs. This doesn't happen when using OTP on BEAM because you typically start a gen_server (or a gen_*) under a supervisor. Even processes that are not supervised can be tracked. Because pids (identifiers to processes) are first-class primitives, you can always access the scheduler which will tell you _all_ of the running processes. If you were to attach a Nodejs REPL, you can't really tell. This is because there is no encapsulation of the process, no way to track when something went async, no way to send control messages to those async processes. 2. Because async threads are easily orphaned, errors that get thrown gets easily lost. The response I get from people who love Typescript on Nodejs tells me that is what the linter is for. That is, we're going to use an external tool to enforce all errors gets handled, rather than having the design of the language and the runtime handle the error. In the BEAM runtime, unhandled errors within the process crashes the process, without crashing anything else; processes that are monitoring that process that crashed gets notified by the runtime that it has crashed. The engineer can then define the logic for handling that crash (retry? restart? throw an error?). 3. The gen_server behavior in OTP defines ways to send control messages. This allows more nuanced approaches to managing subsystems than just restarting when things crash. I'm pretty much at the point where I would not really want to work on deploying Nodejs on the backend. I don't see how something like Deno would fix anything. Typescript is incapable of fixing this, because these are design flaws in the runtime itself.
    • roblh 2y ago
      Just to further hammer point 2 and how it’s a problem in the real world, Express, probably the go to server library for close to a decade, has only within the last couple months sorted out not completely swallowing any error that happens in async middleware by default. And only because some new people came in to finally fix it! It’s absolutely insane how long that took and how easy it was to get stung by that issue.
    • RossBencina 2y ago
      I'm not a Nodejs developer. Could someone please explain, in technical terms, what it means for a thread to "get orphaned"?
      • melodyogonna 2y ago
        Node is single-threaded, so maybe OP means a coroutine, or async functions; Since you can run async functions without awaiting for the result
        • lenkite 2y ago
          Wouldn't orphaned async tasks be garbage collected ?
      • hosh 2y ago
        An invocation of a Nodejs async function is automatically tracked within the code as a locally-scoped promise. The runtime will track it, but unless you then register that Promise elsewhere, it can only be accessed within that local scope. You better hope that you immediately chain it with the success callback or capture errors from it. Spawning a lightweight process in BEAM returns a first-class primitive called a pid. That pid is recorded by the scheduler, so even if it gets lost by the code, you can still find out if it has been taking up resources (when debugging problems live in production). Supervisor behavior is written in a way so that any gen_server-behavior-complying processes will be linked. That means any crashes of the spawned process will notify the supervisor. That’s not something we are doing with Nodejs async — there is no mailbox to notify, just either awaiting completion, or make sure you add the error handling … which is where people write linters to check.
  • jlouis 2y ago
    Bandwidth vs latency. Erlang is designed to keep low latency under load with graceful degradation.
• elcritch 2y ago
  Elixir never performs all to well in microbenchmarks. Yet in every application I've seen Elixir/Erlang projects compared to more standard Node, Python, or even C# projects and the Elixir one generally has way better performance and feels much faster even under load. Personally I think much of it is due to async being predominant in Node and python. Async seems much harder than actor or even threading for debugging performance issues. Sure it feels easier to do async at first. But async leads to small bloat adding up and makes it very difficult to debug and track down. It makes profiling harder, etc. In BEAM, every actor has its own queue. It's trivial to inspect and analyze performance blockages. Async by contrast puts everything into one giant processing queue. Plus every function call in async gets extra overhead added. It all adds up.
  • hosh 2y ago
    This has to do with how async works without preemption and resource limits. There's a counter-intuitive thing when trying to balance load across resources: applying resource limits helps the system run better overall. One example: when scaling a web app, there comes a point when scaling up the database doesn't seem to help. So we're tempted to increase the connection pool because that looks like a bottleneck. Increasing the pool can make the overall system perform worse, because often times, it is slow queries and poorly performing queries that is stopping up the system. Another example: one of the systems I worked on has over 250 node runtimes running on a single, large server. It used pm2 and did not apply cgroups to limit CPU resources. The whole system was a hog, and I temporarily fixed it by consolidating things to run on about 50 node runtimes. When I moved them over to Kubernetes, I also applied CPU resource limit, each in its own pod. I set the limits based on what I measured when they were all running on PM2 ... but the same code running on Kubernetes ran with 10x less CPU overall. Why? Because the async code were not allowed to just run grabbing as much CPU as it can for as long as it can, and the kernel scheduler was able to fairly run. That allowed the entire system to run with less resources overall. There's probably some math that folks who know Operations Research can prove all this.
    • sapiogram 2y ago
      > When I moved them over to Kubernetes, I also applied CPU resource limit, each in its own pod. I set the limits based on what I measured when they were all running on PM2 ... but the same code running on Kubernetes ran with 10x less CPU overall. Why? Because the async code were not allowed to just run grabbing as much CPU as it can for as long as it can, and the kernel scheduler was able to fairly run. That allowed the entire system to run with less resources overall. As someone who has advocated against Kubernetes CPU limits everywhere I've worked, I'm really struggling to see how they helped you here. The code used 10x less CPU with CPU limits, with no adverse effects? What were all those CPU cycles going before?
• javcasas 2y ago
  I have been looking for an Erlang thing akin to Apache Airflow or Argo Workflows. Something that allows me to define a DAG of processes, so that they run one after the other. How would you implement something like that?
  • goosejuice 2y ago
    Have a look at GenStage, flow, Broadway, Oban pro in elixir land. But OTP alone can get you pretty far.
• coldtea 2y ago
  >Node.js is unreliable by design Well, it's just a hack and some C libraries on top of a browser Javascript engine. No big thought went into it, either before or after it got big.
  • jemiluv8 2y ago
    That’s not fair.
    • coldtea 2y ago
      Don't know about fair, but it's accurate. Check on its history. Nothing much has changed, except more money thrown at it.