3 ms·
Multithreaded (or Multiprocess) can be done safely. I write a lot of stuff in Rust these days, where (unless you use unsafe) it doesn't even let you do anythin
by zaarn 7y ago
Multithreaded (or Multiprocess) can be done safely.
I write a lot of stuff in Rust these days, where (unless you use unsafe) it doesn't even let you do anything that would break under multi-threading, even if you don't use it.
As a result, it was incredibly easy to write a light-weight crawler that delivery results into a main worker queue and the data is processed by <cpu_cores> threads. The crawler largely uses async for IO advantages but the worker threads rely heavily on data crunching where your throughput is limited by CPU not by IO and async suffers a bit.
And since crawler and worker are on the same process, the latency between a page being grabbed and starting to be processed is incredibly low, any kind of networking would likely cut throughput by a factor of 10 or more. Not even mentioning having to serialize the data instead of relying on references to get read-only copies and only making minimal copies on the stack.
- halfmatthalfcat 7y agoOk, fair point Rust makes multithreading easier but one could easily make the argument that fighting Rust's borrow checker is a huge burden to development. My point is Node isn't as bad as a server-side language as people claim. There are certain use cases where you can leverage Node's strengths and use another language somewhere else where its shines more. I use Node heavily in my backend stack and have been extremely pleased with it's performance and library ecosystem.
- zaarn 7y agoYou can do the things you do in Node, Async, in Rust too. Futures and friends are in nightly and soon in Stable, which enable you to mix the best of both world; Multithreaded and Async. Once you get used to Rust, you fight the borrow checker a lot less.
- hombre_fatal 7y agoYou can level "just use Rust" at any non-Rust ecosystem. It's kind of tired advice. You might as well be recommending Haskell or any other hobby horse language and then argue from a standpoint of technical superiority as if that's the only concern when choosing tools. I use Rust all the time yet I'll still use Node for most HTTP-based applications and tools I start, and not because I'm an idiot or beginner, as many comments here will suggest. They are completely different languages and ecosystems and workflows that satisfy different goals.
- zaarn 7y agoThis isn't "just use rust". This is "just use something more mature". And there are plenty of more mature ecosystems developing Async capability or that have a good Threading story.