8 ms·
Maybe "we" should not be writing our server code using a terrible ecosystem? Just an idea.
by LeoNatan25 7y ago
Maybe "we" should not be writing our server code using a terrible ecosystem? Just an idea.
- kotxig 7y agoIndeed. It's hard to avoid that conclusion, but you can't deprecate things overnight.
- halfmatthalfcat 7y agoNode has a great server-side ecosystem. I'd go as far as saying Node lends itself to certain layers of the backend stack because of it's history being more CPU-efficient then other server-side runtimes. Can you provide any examples where Node does not provide a good ecosystem for backend programming? Genuinely curious.
- zaarn 7y agoNode isn't great when you need not only CPU-efficient but also multithreaded. The multi-threading or even multi-processing story in node is a clusterfuck. Other than running two full fat Node.js processes, your options are limited to a few APIs that have more chance to break your kneecaps than run the code.
- halfmatthalfcat 7y agoSure but that's the whole premise of Node, there doesn't _need_ to be multithreading. Whats wrong with spinning up multiple workers (in something like Docker/Swarm/K8S) that split up the work and report back to whatever collector or service that wants the end result? So many problems have spawned out of the desire to thread everywhere that the safety of single-threadedness is a feature not a bug.
- zaarn 7y agoMultithreaded (or Multiprocess) can be done safely. I write a lot of stuff in Rust these days, where (unless you use unsafe) it doesn't even let you do anything that would break under multi-threading, even if you don't use it. As a result, it was incredibly easy to write a light-weight crawler that delivery results into a main worker queue and the data is processed by <cpu_cores> threads. The crawler largely uses async for IO advantages but the worker threads rely heavily on data crunching where your throughput is limited by CPU not by IO and async suffers a bit. And since crawler and worker are on the same process, the latency between a page being grabbed and starting to be processed is incredibly low, any kind of networking would likely cut throughput by a factor of 10 or more. Not even mentioning having to serialize the data instead of relying on references to get read-only copies and only making minimal copies on the stack.
- halfmatthalfcat 7y agoOk, fair point Rust makes multithreading easier but one could easily make the argument that fighting Rust's borrow checker is a huge burden to development. My point is Node isn't as bad as a server-side language as people claim. There are certain use cases where you can leverage Node's strengths and use another language somewhere else where its shines more. I use Node heavily in my backend stack and have been extremely pleased with it's performance and library ecosystem.
- zaarn 7y agoYou can do the things you do in Node, Async, in Rust too. Futures and friends are in nightly and soon in Stable, which enable you to mix the best of both world; Multithreaded and Async. Once you get used to Rust, you fight the borrow checker a lot less.
- hombre_fatal 7y agoYou can level "just use Rust" at any non-Rust ecosystem. It's kind of tired advice. You might as well be recommending Haskell or any other hobby horse language and then argue from a standpoint of technical superiority as if that's the only concern when choosing tools. I use Rust all the time yet I'll still use Node for most HTTP-based applications and tools I start, and not because I'm an idiot or beginner, as many comments here will suggest. They are completely different languages and ecosystems and workflows that satisfy different goals.
- zaarn 7y agoThis isn't "just use rust". This is "just use something more mature". And there are plenty of more mature ecosystems developing Async capability or that have a good Threading story.
- marktangotango 7y agoWhats wrong with spinning up multiple workers (in something like Docker/Swarm/K8S) that split up the work and report back to whatever collector or service that wants the end result? The "spinning up multiple workers" is what's wrong with that. Specifically a language/runtime that does support multi-processing handles it for you; ie gomaxprocs. Another example would be servers like apache forking off several workers during start up (depending on the mpm module). With node this has to be managed manually, as you describe, with Docker or whatever. With a node async server, you're generally not cpu bound, so there's really no reason to try to utilize all the cores on the machine. Except if you are cpu bound, then you'd want to do that.
- HereBeBeasties 7y agoIf you think there's nothing wrong with all the complexity that comes with running a distributed system for something that could be done trivially in a single process in a sensible language, I would like to sell you this hammer - you will find it very useful when you next need to screw something together.
- justinmeiners 7y ago> two full fat Node.js processes Do you know about copy on write?
- zaarn 7y agoStarting up a second Node.js process gives you no CoW advantages, that would occur if you fork off the original node process. But Multiprocess support in Node is piss-poor at best.
- spion 7y agohttps://nodejs.org/dist/latest-v12.x/docs/api/worker_threads.html https://nodejs.org/dist/latest-v12.x/docs/api/worker_threads...
- zaarn 7y agoHave you considered that workers in JS absolutely suck?
- spion 7y agoHave considered you that suck conditions race more with threads and shared memory parallelism? If your counter-suggestion is Rust, have you considered.unwrap() that.unwrap() thinking.unwrap() about.unwrap() managing.unwrap() memory.unwrap() all the time might not be the wisest choice and that a GC is a useful abstraction to have?
- zaarn 7y agounwrap() can very very often be avoided in favor of a simple ? operation, the amount of nesting you suggest does occur... but not in a single function so it's spread over many lines and functions. There are GC libraries for Rust if you need them. Or Python/Lua bindings that have it too. Have you considered that avoiding race conditions isn't hard and you can't avoid having to avoid them even in JS?
- novaleaf 7y agoassuming your code doesn't actually need to be multithreaded, but you want to take advantage of multicore: 1) you can use a launcher like pm2 to auto-fork your node app 2) you can horizontally scale (put your servers behind a load-balancer) if you do need multithreading for computations, you are right that node probably isn't a very good choice, but you have some options: 1) use worker threads 2) offload your intensive computations to a separate process written in a different language.
- zaarn 7y agoEven without multithreading, Rust will soon regain advantages with Async being stabilized. You get the best of both worlds.
- rhinoceraptor 7y agoIt's also a lot harder to write, and in most cases developer time costs a lot more than another instance or two.
- zaarn 7y agoIt's not that much harder to write if you use modern practises and languages.
- smt88 7y agoPeople have written about this at length many times, but I'll rehash some of the arguments. Node is missing very basic features, leading to a million tiny modules like leftpad. That's annoying by itself because there's overhead in finding the best package for everything. The other consequences of that issue are much worse. You have lots of easy opportunities for misleadingly-named packages with malware in them. Package hijacking is easier to miss. Security updates, if packages even get them, have to be applied seemingly every hour. It's just a major problem to have a simple project with 500 dependencies. And that doesn't even get me started on issues with the mess that module importing was/is.
- halfmatthalfcat 7y agoI agree that JS is missing a lot of standard library features that libraries like lodash or underscore have tried to backfill and the proliferation of libraries in npm is a cause for concern but is that enough to call an entire runtime's ecosystem obsolete or unwanted? I just find it hard (however biased I may be) that we can just say "Node has all these problems, let's throw it out" when it's helped lower the barrier to entry for new people programming at-large.
- krapp 7y ago>I just find it hard (however biased I may be) that we can just say "Node has all these problems, let's throw it out" when it's helped lower the barrier to entry for new people programming at-large. This seems to assume that it would be impossible to design a better package manager which also provides a low barrier to entry. Bear in mind, however, that all you actually need to write javascript and learn programming with it at a basic level is a text editor. The low barrier that NPM provides is for publishing to NPM, which needn't be synonymous with "javascript development." Your comment also seems to assume that the barrier to entry everywhere else is prohibitively high, but plenty of people still learn with Ruby or Python or other languages. There are whole industries for teaching new programmers.
- smt88 7y agoI actually disagree that it's lowered the barrier. JS is a hard language with lots of "wtf" behavior and weird runtime errors. You need lots of discipline to write code without tons of runtime errors, and beginners don't even know what they need to be disciplined about. I'd argue that JS raises the barrier because lots of beginners now think it's normal to have a bizarre type system where null is an object, to have a complex Webpack/Babel setup just to use new-ish language features, and to have to constantly hit "run" to debug code. Most previous-gen web languages also have lots of gotchas, but something like Go, Kotlin, or Dart would be a much more sane beginner language.
- rubber_duck 7y ago>Can you provide any examples where Node does not provide a good ecosystem for backend programming? A decent URL parsing/matching lib not tied to a full web framework ? Disclosure - haven't done serious node work in a while but ~2 years ago when I was building something in with it all the published libs for this were complete garbage - looking at the code for the packages made me realize what sort of crap gets published to NPM and then actually used/referenced by community. I think it's better than PHP (took on a PHP integration job to help out a friend in trouble recently - that was going back to the dark ages with package management and module system). But it's the same tier in my eyes - filled with clueless newbies trying to lead other newbies - a community of blind leading the blind. In my eyes a sane/mature language like Python beats Node any day of the week quality/productivity wise and so does stuff built on top of JVM/CLR. As much as C#/Java devs like to obfuscate code with useless patterns I'd take a random C#/Java package over a random Node package any time - at least the developer managed to convince the type system to compile his code - wouldn't say that the authors could do the same for some of the NPM packages I saw.
- jeffschofield 7y agoFYI a URL parser was actually added around 2 years ago in v8.0.0. It's based on the WHATWG standard and matches the implementation in modern browsers. https://nodejs.org/api/url.html#url_url_parse_urlstring_parsequerystring_slashesdenotehost https://nodejs.org/api/url.html#url_url_parse_urlstring_pars...
- rubber_duck 7y agoGood to know. Maybe if I got back to it the core is more functional now. My impression is that it's soo easy to publish to NPM and the ecosystem is newbie friendly - this is good but it also creates a lot of noise and popularity/usage is not a valid indicator of reliability/quality (compared to other platforms). And because the core was so barebones you need to use third party solutions a lot more often so it compounds the issues.
- toastal 7y agoAny type system that allows null or any shouldn't be trusted any more than a dynamic language
- kotxig 7y ago"Backend programming" is a really broad class. It's safe to say that computationally intensive problems aren't best solved in node. For many server side "web service" type applications node is sufficient but there are a lot of tradeoffs being made that you may or may not be aware of. By choosing node you are choosing single-threaded async IO as your primitive. You have to be ok without a threading model that supports shared memory (v8 objects don't pass between isolates, only certain types pass between web workers). And what's more, you have to be certain that your problem will fit these choices in the future. Is it ever a good idea to hamstring a backend service with one narrow concurrency model? By comparison, golang offers you all of those things. If you need to write a backend service that performs well and can respond well to changing requirements, you have a much better chance of doing it with golang than node. I just can't promise any individual developer that they will enjoy writing the golang service more than the node one. I don't want you to think I'm saying golang is better than node, we haven't even started to talk about the module system. I'm just saying fundamentally it is more flexible and can do everything that node does with at least the same level of performance, and generally it is "faster". Most things out of the box are faster. Benchmarking and performance is in the DNA of the go community and this isn't true for the node community.
- justinmeiners 7y ago> By choosing node you are choosing single-threaded async IO as your primitive In node is it very easy to fork processes and have them communicate. Scripting languages for serving HTML and JSON aren't the place for threads. Node now offers sync versions of most IO functions as well.
- kotxig 7y agoYou just don't have shared memory at your disposal by forking processes. For example, having a rather large dictionary of translation strings which are read-only and static, you have to load these in every process as there is no copy on write property to these "forks". If your JSON payloads are large, you can't have a forked process do the deserialization of it (because you would need to re-serialize across the communication boundary, which defeats the point). I think it would be reasonable to have immutable data structures shared with web workers. I also think it would be reasonable to make it possible to pass complex objects (all primitive types supported by JSON) between workers without a serialization step.
- novaleaf 7y agoAgree! There's a package for just about anything you could want. If you are in a small shop (I'm a 1 person team) it's so invaluable. NPM's module selection is so wide that it's sometimes hard to find the best one. Too many choices! I would rather have that problem than lack of any choice. Overall I'd guess to get a functional product, NPM's modules saved me about 50% of my time. Personally I love C#, but can't use that for backend because of the lack of module ecosystem.
- lmm 7y ago> I'd go as far as saying Node lends itself to certain layers of the backend stack because of it's history being more CPU-efficient then other server-side runtimes. If you're talking about async I/O, TCL (among others) has offered that for more than 20 years (from AOLServer on), and in a much better-designed language than JS.
- spion 7y agoIs the design better in some fundamental way, or is it just about equality comparison and other shallow "wtfs"?
- blacksqr 7y agoI'm not a Node expert, but my understanding is that if any code throws an error, the server crashes and has to be restarted. That prompted a serious WTF from me when I read about it. While Tcl is running as a server in an event loop, if an error is thrown while processing an event it is reported via a background error API, and the server keeps going.
- spion 7y agoThat is true, and node's official advice in this regard is bad. It comes from previous bad designs of error handling and leaking resources (file descriptors mostly) caused by domains. However, modern node code typically uses promises and you can use slightly more old-fashioned methods of managing resources: https://stackoverflow.com/a/19540907/110271 https://stackoverflow.com/a/19540907/110271 You can go a step further and fully isolate "crash-assuming" code from "non-crash-assuming" (promise based code): https://gist.github.com/spion/ed8deb7a3b4add0a6d727dc78fe635d8 https://gist.github.com/spion/ed8deb7a3b4add0a6d727dc78fe635...
- lmm 7y agoI don't know what you consider a deep part of programming language design. E.g. I would consider prototypal inheritance and non-lexical scope to be deep language design mistakes in JavaScript. But even beyond that TCL is a particularly elegant design (rather lisplike in a way) - e.g. Guido van Rossum cites it as an inspiration.
- pjmlp 7y agoNode being more CPU efficient than Java, .NET, C++, Go?!?