6 ms·
OP is complaining about Goroutine stack size at 4kb per connection, his test shows that Node8 is taking up to 150MB of memory with 5k users, 4kb*5k = 20MB for G
by Thaxll 9y ago
OP is complaining about Goroutine stack size at 4kb per connection, his test shows that Node8 is taking up to 150MB of memory with 5k users, 4kb*5k = 20MB for Go memory, I don't understand how Nodejs can take less memory than Go, and without real numbers / test I'm pretty sure he's doing something wrong somewhere.
From my experience on some large prod deployment, Nodejs app takes much more memory and CPU vs Go app for doing similar work.
- bryanlarsen 9y agoIt's 2 per connection, so that's 40MB of RAM. The OP also said that was minor compared to the channel overhead -- it seems the number of channels was exponential to the number of users in a room.
- anonymoushn 9y agoSeems like it should be linear??
- iends 9y agoHopefully it's linear, or it's implemented wrong.
- bryanlarsen 9y agoHopefully, but all he said was "fanout" which could be either. He knows it's implemented wrong, that's the point of the article: that Go doesn't make it easy for him to implement it right. The only question is the degree of wrongness. :)
- iends 9y agoIt being implemented incorrectly is not a property of the Go language, though.
- KirinDave 9y agoIt is a property of the Go language that the most recommended and explained concurrency primitives are slower than even 20 year old implementations.
- weberc2 9y agoI don't think this criticism holds water since he opted for JavaScript. If he was having problems with goroutines and channels, he could have picked a Node-like architecture (single-threaded, callback-driven) and still likely enjoyed better performance (and optimization tooling!) than with Node. If his issue was Go's type system, then he shouldn't have chosen a language with a strictly worse type system. In other words, in the worst case, some scenarios may require you to drop into `interface{}` in Go, which has the same safety guarantees as any type in JavaScript. This way only ~3% of your code is type-unsafe instead of JavaScript's 100%. While I think there are lots of valid criticisms of Go (concurrency correctness is still hard and its type system is not very good for certain tasks), none of these are reasons to switch to JavaScript, as the author did. If he was struggling with goroutines and channels, he could still elect for a Node-like architecture (even making it single-threaded by setting GOMAXPROCS=1). If he needed generics and unions (as he cited), he switched to a language without them and without any static types at all (or a single static type, if you will); in Go he could have downgraded to `interface{}` which is analogous to JavaScript's one static type, and he would have only gave up type safety in the bits of code where Go's type system was lacking instead of everywhere. I don't want to give the impression of overselling Go here; it's just that for the cited criteria, Go is strictly better than JavaScript.
- bryanlarsen 9y agoSure, if you don't mind writing your own websocket and pub/sub libraries.
- deleted 9y ago[deleted]
- adamweston 9y agoNode makes the Node-like architecture much easier than it would be in Go, sort of by definition. Also, wouldn't Node's JIT fare much better in the face of dynamic typing than using reflection (interface {}) everywhere?
- 9y ago
- Thaxll 9y agoGoroutine takes 2kb, so 4kb for read / write per conection.
- JepZ 9y agoAnd I thought the memory usage of channels depends on the buffer size.
- weberc2 9y agoThis is correct.
- todd8 9y agoA fully connected graph of n users contains n*(n-1) links if users don't connect to themselves (I would describe this as polynomial growth in the number of channels, a lot better than exponential). A chat broker that acts as a switchboard between users could, I suppose, reduce this to a linear relationship between users and channels.
- adamweston 9y agoLike a bus?
- amelius 9y agoYou have to divide that by 2. It's "half the matrix".
- todd8 9y agoThanks, my mistake. You're right, there are n(n-1)/2 arcs in the fully connected graph (K_n).
- sebcat 9y ago> it seems the number of channels was exponential to the number of users in a room I do not see this in the code linked to from the article.
- sebcat 9y agoI count three goroutines per connection: One for ChatHandler.Loop (when invoked from ChatService), one for ChatHandler.socketReaderLoop and one for socketWriterLoop. Each ChatHandler has three (buffered) channels: outgoingInfo.channel, sockChannel, readErrorChannel (not buffered) There are some things that could cause bottle necks that are not related to goroutines and channels: Every published message calls GroupInfoManager.GetUsers to get a list of all the users in a group. That can be expensive (I don't know if it is or isn't). And then for every user in a channel, their userOutGoingInfo is retrieved. I would suggest profiling before switching languages. EDIT: changed "benchmarking" to "profiling" EDIT2: changed "are likely to" to "could" (... cause bottle necks) since I don't know
- tootie 9y agoJava or C# solved all of this like 10 years ago but people like making this hard on themselves.