5 ms·

Every single "Why we switched from Node.js to Go" article I've ever read is rooted in misunderstanding of Node.js - This one is no exception. The problem was th

Every single "Why we switched from Node.js to Go" article I've ever read is rooted in misunderstanding of Node.js - This one is no exception. The problem was that the codebase was bloated and probably time to scale up or scale out. I bet that if an experienced Node.js developer rewrote the entire service again in Node, you would also get 2x speedup. I can't believe the CTO of digg bought this but I guess ultimately, it's also about making engineers happy. Eventually Go will also reach a point where it will start lagging and then some smart engineer will conclude that you need to switch to C++! Why can't developers just admit that they don't know how to scale a service?! It's not the language that's the problem, it's you; it's always you!

skizm 10y ago
I'm not a node expert, but here is what seems like the crux of the problem: > Consequently, when any request timeouts happened, the event and its associated callback was put on an already overloaded message queue. While the timeout event might occur at 1 second, the callback wasn’t getting processed until all other messages currently on the queue, and their corresponding callback code, were finished executing (potentially seconds later). Seems pretty specific to node, no? I mean, just looking at what the article says (again, I'm not a Node or Go expert) it seems like this problem was directly alleviated by Go and the speedup wasn't because Go is closer to the metal, but ultimately had a different method of handling many async network calls.
- chadscira 10y ago
  Yep thats what it looks like. So the take away is Go is better at handling massive amounts of open outgoing network calls. In Node.js you want to restrict this to prevent the server from choking. If they did that I bet performance would be very similar.
- wwwigham 10y ago
  I don't think it was alleviated directly by Go, I think it was alleviated directly by improved parallelism... By the descriptions given, their go implementation uses a worker pool to help split active work across threads, while their Node implementation did not (though in Node you'd shard across processes rather than threads) - or at least it doesn't seem like it did (despite admitting to spending a bunch of time doing processing in JS between s3 requests). Another nit: The article also implies that the height of the flame graph would indicate a performance issue, which I argue is inaccurate: Since the height of a flame graph measures stack depth, I argue it also represents abstraction. So long as most of those stacks are are equal-width, then the abstraction is low overhead (and probably has value to justify it). Generally with a flame graph you want to be making comparisons within the graph, comparing the time spent in each call (so the relative widths); not commenting on its shape in-general (though it does always make for a pretty blog post).
  - aikah 10y ago
    > I don't think it was alleviated directly by Go, Yes it is, Go is truly concurrent, NodeJS is not. Go has green threads in userland, NodeJS does not. A Go program can actually perform multiple operations at the same time since Go coroutines are implemented at the language level, NodeJS cannot. Obviously Javascript is more expressive but Go programs are easier to read and maintain. With Javascript anything goes.
    - WayneBro 10y ago
      Go has multithreaded concurrency. Node JS has multiprocess concurrency.
      - aikah 10y ago
        > Go has multithreaded concurrency. Node JS has multiprocess concurrency. Go also has "multi-process" concurrency. Go is both concurrent and parallel, Go coroutines will spawn on every processor available by default. NodeJS needs an external tool like Redis to enable "multi-process" communication. Go does not.
        wwwigham 10y ago
        Nonsense. You don't need redis for multiprocess communication in Node - child processes spawned have a communication channel with the parent whereby they can efficiently send messages (in the form of json objects) back and forth. (Or you could open your own shared file descriptor, like how you could do multiprocess message passing in any lower-level language.) There is little difference in usability between message-passing goroutines and message-passing child processes in node - just less syntactic sugar. Go makes it more frictionless to write parallel code than in Node, but its certainly not impossible to write good parallel code in Node. I wouldn't blame you nor the article author for not understanding the distinction - most people hear "Node is single threaded" and just accept that at face value. Coming from Ruby or Python, where the fork-process-to-be-parallel scheme is common, I am familiar with this variety of parallelism.
- jondubois 10y ago
  I guess you're probably not familiar with the whole "Why I switched from Node.js to Go" rhetoric - This article is just one of many. I've heard this "The Node.js event loop gets saturated" argument so many times - It's the #1 argument used in all articles which promote switching from Node.js to Go. The real reason why the event loop is saturated is because your Node.js process is doing too much work! The solution is to split your process into two (use the Cluster module or a load balancer - It's super easy). With goroutines and the like, all Go actually does is delay the inevitable point when you have to split your main process into two... It doesn't solve the problem, it just delays it... I don't know how Go handles the situation where a process gets too many requests... Maybe it just drops requests (instead of adding latency like Node.js)? I don't know... But once a program maxes out on total available CPU, bad things will happen! It doesn't matter what language/engine is being used; whether requests will start lagging + timing out or that they will start to get rejected outright; programs cannot transcend CPU constraints. I don't mind people switching to Go from Node.js because "they feel like it", but I don't think it's fair to write an article about it claiming that Node.js is unfit for technical reasons. There is no meat behind this argument at all and I'm just tired of seeing it resurface again and again and again on HN. It's starting to look like a propaganda campaign.
  - rounce 10y ago
    > With goroutines and the like, all Go actually does is delay the inevitable point when you have to split your main process into two... It doesn't solve the problem, it just delays it... Yes, but Go's concurrency model is far easier to reason about. "I _want_ to do this thing", and the runtime handles the _how_ and the doing. No need to draw up some IPC messaging protocol to handle trivial tasks safely. It's a language feature (ugh sounds so jingoistic). The cluster module is nothing like goroutines in concept or in practice. You're making some mighty big assertions there, all the while disregarding the obvious weakness of a _single_threaded_ event-loop. I'm a total Node.JS fanboy since 0.2, but as was mentioned elsewhere in the comments, every language is making trade-offs: substituting shiny new concerns for it's own achilles heel.
  - sotojuan 10y ago
    You're giving Node devs too much credit. Most Node devs know nothing beyond `fs` and `express`. For some strange reason they refuse to learn about the standard library beyond the very basics.
vesak 10y ago
Go and Javascript/Node.js are quite different languages in their feature lists and main implementations. To mention a big one, Go is mostly statically and strongly typed, in stark contrast to Javascript. Also it has significantly fewer misdesigns (such as the equality operator being weird). Are you claiming that such features don't matter for the effectivity of the platform?
- watson 10y ago
  The syntax of your language doesn't change its performance when running. I can appreciate that not everyone likes JavaScript - it's all a matter of personal preference I guess - but saying that Go is a better language because of the syntax is like saying bananas taste better because of their color. They are two very different languages with each their strengths and weaknesses. Does JavaScript have a few design flaws? Absolutely! More so than many other languages? Quite possibly! Does it make me write worse software if I'm aware of those quirks? Most certainly not.
  - vesak 10y ago
    > The syntax of your language doesn't change its performance when running. No need to bring up syntax. It's a superficial feature and has nothing to do with the actually important features I listed. > Does it make me write worse software if I'm aware of those quirks? Most certainly not. Is there some limit where good language features stop mattering? I guess we would obviously agree that creating web software with x86 assembler wouldn't be a good idea -- even if we could imagine a comparable library/framework situation.
  - aikah 10y ago
    > The syntax of your language doesn't change its performance when running. No but the difference between static typing and dynamic typing actually does. Go compiler performs a lot of optimizations that no Javascript engine can.
- gotofritz 10y ago
  Those you mention no. There are tools for typechecking (flow) and to be honest in years of JS programming only rarely have I had bugs caused by a variable of the wrong type being passed into a function. As for the "weird" equality operator, again, not an issue, any linter will force you to use === and then you are good to go. If you are talking about the callback queue getting too full and causing delays you may have a point, but the objections that you have raised are quite petty.
lazyjones 10y ago
> Eventually Go will also reach a point where it will start lagging and then some smart engineer will conclude that you need to switch to C++! Go is much better suited for large codebases than Node.js though, so when it becomes bloated again, it'll be easier to fix problems with/optimize the existing code and rewrites can be avoided.
heisenbit 10y ago
I'm also not getting it. Long flame graphs in node vs. golang is probably 50% refactor. The other 50% is the async calling pattern in node that leads to deep stack frames. The overloaded event queue? If that is true the not the queue is overloaded but there are too many entries that are ready to run and at least one takes a lot of time running to completion. So there could be system level problems (flood incoming data) or a code problem. There could also be a low level i/o problem in node showing up under high load. It would have been interesting to learn the true root cause. Considering the time already spent and the results they got and the cost of investment of switching to golang I would say the decision is rationale. The whole point of micro-services is to maintain the tooling freedom. You are trading off the need of expensive support contracts with the bounded risk of re-engineering a small service.
commentzorro 10y ago
> I bet that if an experienced Node.js developer rewrote the entire service again in Node, you would also get 2x speedup. Maybe, but why not try Go on a small piece and see how it works. > I can't believe the CTO of digg bought this... You make it seem like the CTO was somehow being conned. Like Go was a bad choice. Maybe it was as simple as wanting to see what another language offered and this was a good time to try it out. > Why can't developers just admit that they don't know how to scale a service?! It's not the language that's the problem, it's you; it's always you! Oddly enough, this is a language feature built into Go from the get-Go. The language is designed such that a mid level programmer can ramp up and write performant Go code in a short amount of time without needing years (or even months) of experience, theory, classes, third party libraries, tooling, etc. It's a bold experiment and it may not succeed. Has it for Digg?
xiaoma 10y ago
Building in C++ is completely rational after you've hit such scale that server costs outstrip the engineering costs of using a low level language. It's not like fb, google and amazon don't know how to scale a service.
algesten 10y ago
Exactly my thought! I was also thinking that Octo sounds way too much like a monolith. I wouldn't let retries tax the same instance. I bet they could use something like RabbitMQ to sort work distribution and retries between instances without relying on nested callbacks.
deleted 10y ago
[deleted]
blacksmythe 10y ago
>> I can't believe the CTO of digg bought this Sounds like the project was a success: >> Two weeks later, after my initial crash course introduction to Golang, we had a brand new Octo service up and running. >> you would also get 2x speedup. The win wasn't the 2x speedup, it was not choking under heavy load.
- 69mlgsniperdad 10y ago
  Success..? I'm assuming this woman is a payed employee. She claims to have worked with nodejs for months on end, she attempts and fails many times to fix a bug, or possibly refactor a simple utility, blames the technology and decides she should spend her companies time and money learning a new language and rebuilding the service, slowly, with a brand new language. In other words: She couldn't solve the issue because she couldn't find a tutorial. The only tutorial she could find used Go, so she switched to Go. Brilliant success.
tjholowaychuk 10y ago
As someone who wrote Node for roughly 5 years, it's VERY hard to write robust Node code, I would never trust it to mission critical work personally. Go isn't perfect, and it'll likely remain that way but it's much better suited to this sort of thing.
- dmitri1981 10y ago
  What are biggest issues with Node in your opinion?

skizm 10y ago

I'm not a node expert, but here is what seems like the crux of the problem: > Consequently, when any request timeouts happened, the event and its associated callback was put on an already overloaded message queue. While the timeout event might occur at 1 second, the callback wasn’t getting processed until all other messages currently on the queue, and their corresponding callback code, were finished executing (potentially seconds later). Seems pretty specific to node, no? I mean, just looking at what the article says (again, I'm not a Node or Go expert) it seems like this problem was directly alleviated by Go and the speedup wasn't because Go is closer to the metal, but ultimately had a different method of handling many async network calls.

chadscira 10y ago
Yep thats what it looks like. So the take away is Go is better at handling massive amounts of open outgoing network calls. In Node.js you want to restrict this to prevent the server from choking. If they did that I bet performance would be very similar.
wwwigham 10y ago
I don't think it was alleviated directly by Go, I think it was alleviated directly by improved parallelism... By the descriptions given, their go implementation uses a worker pool to help split active work across threads, while their Node implementation did not (though in Node you'd shard across processes rather than threads) - or at least it doesn't seem like it did (despite admitting to spending a bunch of time doing processing in JS between s3 requests). Another nit: The article also implies that the height of the flame graph would indicate a performance issue, which I argue is inaccurate: Since the height of a flame graph measures stack depth, I argue it also represents abstraction. So long as most of those stacks are are equal-width, then the abstraction is low overhead (and probably has value to justify it). Generally with a flame graph you want to be making comparisons within the graph, comparing the time spent in each call (so the relative widths); not commenting on its shape in-general (though it does always make for a pretty blog post).
- aikah 10y ago
  > I don't think it was alleviated directly by Go, Yes it is, Go is truly concurrent, NodeJS is not. Go has green threads in userland, NodeJS does not. A Go program can actually perform multiple operations at the same time since Go coroutines are implemented at the language level, NodeJS cannot. Obviously Javascript is more expressive but Go programs are easier to read and maintain. With Javascript anything goes.
  - WayneBro 10y ago
    Go has multithreaded concurrency. Node JS has multiprocess concurrency.
    - aikah 10y ago
      > Go has multithreaded concurrency. Node JS has multiprocess concurrency. Go also has "multi-process" concurrency. Go is both concurrent and parallel, Go coroutines will spawn on every processor available by default. NodeJS needs an external tool like Redis to enable "multi-process" communication. Go does not.
      - wwwigham 10y ago
        Nonsense. You don't need redis for multiprocess communication in Node - child processes spawned have a communication channel with the parent whereby they can efficiently send messages (in the form of json objects) back and forth. (Or you could open your own shared file descriptor, like how you could do multiprocess message passing in any lower-level language.) There is little difference in usability between message-passing goroutines and message-passing child processes in node - just less syntactic sugar. Go makes it more frictionless to write parallel code than in Node, but its certainly not impossible to write good parallel code in Node. I wouldn't blame you nor the article author for not understanding the distinction - most people hear "Node is single threaded" and just accept that at face value. Coming from Ruby or Python, where the fork-process-to-be-parallel scheme is common, I am familiar with this variety of parallelism.
jondubois 10y ago
I guess you're probably not familiar with the whole "Why I switched from Node.js to Go" rhetoric - This article is just one of many. I've heard this "The Node.js event loop gets saturated" argument so many times - It's the #1 argument used in all articles which promote switching from Node.js to Go. The real reason why the event loop is saturated is because your Node.js process is doing too much work! The solution is to split your process into two (use the Cluster module or a load balancer - It's super easy). With goroutines and the like, all Go actually does is delay the inevitable point when you have to split your main process into two... It doesn't solve the problem, it just delays it... I don't know how Go handles the situation where a process gets too many requests... Maybe it just drops requests (instead of adding latency like Node.js)? I don't know... But once a program maxes out on total available CPU, bad things will happen! It doesn't matter what language/engine is being used; whether requests will start lagging + timing out or that they will start to get rejected outright; programs cannot transcend CPU constraints. I don't mind people switching to Go from Node.js because "they feel like it", but I don't think it's fair to write an article about it claiming that Node.js is unfit for technical reasons. There is no meat behind this argument at all and I'm just tired of seeing it resurface again and again and again on HN. It's starting to look like a propaganda campaign.
- rounce 10y ago
  > With goroutines and the like, all Go actually does is delay the inevitable point when you have to split your main process into two... It doesn't solve the problem, it just delays it... Yes, but Go's concurrency model is far easier to reason about. "I _want_ to do this thing", and the runtime handles the _how_ and the doing. No need to draw up some IPC messaging protocol to handle trivial tasks safely. It's a language feature (ugh sounds so jingoistic). The cluster module is nothing like goroutines in concept or in practice. You're making some mighty big assertions there, all the while disregarding the obvious weakness of a _single_threaded_ event-loop. I'm a total Node.JS fanboy since 0.2, but as was mentioned elsewhere in the comments, every language is making trade-offs: substituting shiny new concerns for it's own achilles heel.
- sotojuan 10y ago
  You're giving Node devs too much credit. Most Node devs know nothing beyond `fs` and `express`. For some strange reason they refuse to learn about the standard library beyond the very basics.

vesak 10y ago

Go and Javascript/Node.js are quite different languages in their feature lists and main implementations. To mention a big one, Go is mostly statically and strongly typed, in stark contrast to Javascript. Also it has significantly fewer misdesigns (such as the equality operator being weird). Are you claiming that such features don't matter for the effectivity of the platform?

watson 10y ago
The syntax of your language doesn't change its performance when running. I can appreciate that not everyone likes JavaScript - it's all a matter of personal preference I guess - but saying that Go is a better language because of the syntax is like saying bananas taste better because of their color. They are two very different languages with each their strengths and weaknesses. Does JavaScript have a few design flaws? Absolutely! More so than many other languages? Quite possibly! Does it make me write worse software if I'm aware of those quirks? Most certainly not.
- vesak 10y ago
  > The syntax of your language doesn't change its performance when running. No need to bring up syntax. It's a superficial feature and has nothing to do with the actually important features I listed. > Does it make me write worse software if I'm aware of those quirks? Most certainly not. Is there some limit where good language features stop mattering? I guess we would obviously agree that creating web software with x86 assembler wouldn't be a good idea -- even if we could imagine a comparable library/framework situation.
- aikah 10y ago
  > The syntax of your language doesn't change its performance when running. No but the difference between static typing and dynamic typing actually does. Go compiler performs a lot of optimizations that no Javascript engine can.
gotofritz 10y ago
Those you mention no. There are tools for typechecking (flow) and to be honest in years of JS programming only rarely have I had bugs caused by a variable of the wrong type being passed into a function. As for the "weird" equality operator, again, not an issue, any linter will force you to use === and then you are good to go. If you are talking about the callback queue getting too full and causing delays you may have a point, but the objections that you have raised are quite petty.

lazyjones 10y ago

> Eventually Go will also reach a point where it will start lagging and then some smart engineer will conclude that you need to switch to C++! Go is much better suited for large codebases than Node.js though, so when it becomes bloated again, it'll be easier to fix problems with/optimize the existing code and rewrites can be avoided.

heisenbit 10y ago

I'm also not getting it. Long flame graphs in node vs. golang is probably 50% refactor. The other 50% is the async calling pattern in node that leads to deep stack frames. The overloaded event queue? If that is true the not the queue is overloaded but there are too many entries that are ready to run and at least one takes a lot of time running to completion. So there could be system level problems (flood incoming data) or a code problem. There could also be a low level i/o problem in node showing up under high load. It would have been interesting to learn the true root cause. Considering the time already spent and the results they got and the cost of investment of switching to golang I would say the decision is rationale. The whole point of micro-services is to maintain the tooling freedom. You are trading off the need of expensive support contracts with the bounded risk of re-engineering a small service.

commentzorro 10y ago

> I bet that if an experienced Node.js developer rewrote the entire service again in Node, you would also get 2x speedup. Maybe, but why not try Go on a small piece and see how it works. > I can't believe the CTO of digg bought this... You make it seem like the CTO was somehow being conned. Like Go was a bad choice. Maybe it was as simple as wanting to see what another language offered and this was a good time to try it out. > Why can't developers just admit that they don't know how to scale a service?! It's not the language that's the problem, it's you; it's always you! Oddly enough, this is a language feature built into Go from the get-Go. The language is designed such that a mid level programmer can ramp up and write performant Go code in a short amount of time without needing years (or even months) of experience, theory, classes, third party libraries, tooling, etc. It's a bold experiment and it may not succeed. Has it for Digg?

xiaoma 10y ago

Building in C++ is completely rational after you've hit such scale that server costs outstrip the engineering costs of using a low level language. It's not like fb, google and amazon don't know how to scale a service.

algesten 10y ago

Exactly my thought! I was also thinking that Octo sounds way too much like a monolith. I wouldn't let retries tax the same instance. I bet they could use something like RabbitMQ to sort work distribution and retries between instances without relying on nested callbacks.

deleted 10y ago

[deleted]

blacksmythe 10y ago

>> I can't believe the CTO of digg bought this Sounds like the project was a success: >> Two weeks later, after my initial crash course introduction to Golang, we had a brand new Octo service up and running. >> you would also get 2x speedup. The win wasn't the 2x speedup, it was not choking under heavy load.

69mlgsniperdad 10y ago
Success..? I'm assuming this woman is a payed employee. She claims to have worked with nodejs for months on end, she attempts and fails many times to fix a bug, or possibly refactor a simple utility, blames the technology and decides she should spend her companies time and money learning a new language and rebuilding the service, slowly, with a brand new language. In other words: She couldn't solve the issue because she couldn't find a tutorial. The only tutorial she could find used Go, so she switched to Go. Brilliant success.

tjholowaychuk 10y ago

As someone who wrote Node for roughly 5 years, it's VERY hard to write robust Node code, I would never trust it to mission critical work personally. Go isn't perfect, and it'll likely remain that way but it's much better suited to this sort of thing.

dmitri1981 10y ago
What are biggest issues with Node in your opinion?