3 ms·
I started using worker threads for some batch processing and development tools recently. We have a couple of tricky bits that pull data from different environme
by hinkley 4y ago
I started using worker threads for some batch processing and development tools recently. We have a couple of tricky bits that pull data from different environments to compare them, so the code has to run 'as' two or more environments and forcing a state change was bug ridden as hell, and spawning a child process was slow enough that we limited the scope of how much we scanned (running n worker threads for n environments dropped a lookup from 1.5s to 350ms).
Armed with those successes I turned to thinking about applying something similar to a production Node app, and every time I think about it I realize what I really want is OTP. Each type of processing we do has its own caches, and its own CPU uses, which could dovetail well with worker threads, but I have a chicken and egg problem with trying to move any of that work out of process because I don't get any benefit until 1) none of the workers can silently crash and leave the application nonfunctional and 2) before I'm handling half a dozen concurrent requests per Node process instead of one or two. and 3) before message passing overhead is cheaper
It's either too late or too early to rewrite this whole thing in Elixir.
- orf 4y agoPerhaps Node is the problem, rather than Elixir being the solution?
- toolz 4y agoIf node is the issue here that would make elixir a solution, no? Changing languages (i.e. node is the issue) to get the features described would put you looking at elixir (among other languages that can fill the described properties). Maybe what you mean is that elixir isn't the only solution?
- orf 4y agoYeah, that’s what I meant: “elixir being the (only) solution”. Guess it was worded ambiguously.
- hinkley 4y agoSometimes your problem is that you're using a reciprocating saw when you should be using an angle grinder, sometimes your problem is that you bought Ryobi instead of Milwaukee or Makita.
- orf 4y agoIndeed, but sometimes you just need to use the right tool for the job. That could be a precision guided flying laser chainsaw axe. Usually it’s a bog standard trusty dumb hammer that everyone else manages to bash similar looking nails with quite effectively.
- hosh 4y agoWould running Nodejs in WebAssembly along with something like Firefly or Lunatic help make the transition smoother?
- nesarkvechnep 4y agoOr… use the right tool - Elixir. Monoglotism is bad, people.