3 ms·
The point to understand here is that Go uses lightweight concurrency and has a layer of indirection between goroutines and hardware threads. Goroutines allocat
by wscott 11y ago
The point to understand here is that Go uses lightweight concurrency and has a layer of indirection between goroutines and hardware threads. Goroutines allocate a tiny stack and Go uses a software scheduler to jump between routines. Then that Go scheduler starts hardware threads as workers to start executing the routines.
The net result is that Go is relatively efficient for naive programs that start hundreds or thousands of threads and let them all run.
I don't know Scala or Erlang, but based on the results I would assume both are allocating a posix thread for each thread in the software and so it is using the kernel scheduler to switch between them with all the overhead that entails.
But a program can be refactored in all the languages to have a smaller number of workers and a queue of work to be consumed. This would be faster than the Go version, but require more code and complexity.
I have been considering threading in Rust vs Go and that is the biggest difference I see between the approaches. However it should be possible to implement Go style goroutines in Rust using custom dispatch code like old C coroutines. I think I have already seen libraries like that, but I haven't dug in the details to know for sure.
- atemerev 11y agoNo, Scala and Erlang don't allocate pthreads. Their threads are also lightweight and in-system.
- shikhar 11y agoJVM threads map to native threads
- kasey_junk 11y agoThat's true but there is some nuance here because you can use the JVM fork/join executor service & see behavior similar to coroutine's.
- masklinn 11y agoUnclear wrt Scala, jakozaur asserts the bench does use pthreads. Erlang does use lightweight processes, but the configuration isn't tuned for high creation/destruction throughput: on my system, the smallest possible process is 333 words, or 2.5Kb (control structures, default stack and default private heap)
- dtech 11y agoAre you sure about scala? I would assume it just uses JVM threads, which map to OS threads on most modern implementations.
- atemerev 11y agoYes, I am sure. M:N dispatcher is used here. http://doc.akka.io/docs/akka/2.4.1/scala/dispatchers.html http://doc.akka.io/docs/akka/2.4.1/scala/dispatchers.html
- emergentcypher 11y agoScala runs on JVM and uses the same OS-level threads. What's going on here is the actors are being scheduled to run on a thread pool.
- emergentcypher 11y agoScala does not have lightweight threading. It uses the same thread pools from Java. All your millions of Actors are scheduled onto one of these thread pools.
- rozap 11y agoWhich means you need to program with futures in order to not do blocking IO on your limited thread pool that the actors are multiplexed across. So at that point you need to write node.js style code. Coming from an Erlang world to akka, this blew my mind.
- dozzie 11y ago> I don't know Scala or Erlang, but based on the results I would assume both are allocating a posix thread for each thread No. Erlang is only ~five times slower than Go, according to the benchmark run, which is a marvellous result given the Erlang's VM is slow as hell, even with HiPE. Scala/Akka is slower by thirty times under this workload. > But a program can be refactored in all the languages to have a smaller number of workers and a queue of work to be consumed. This would be faster than the Go version, but require more code and complexity. Oh, I bet it wouldn't be faster, but it surely would add tons of complexity. Erlang was specifically designed for programmer to spawn a thread for each activity. New client connected? Spawn. Need a background computation? Spawn. Need a connection to external entity? Spawn. Need to do cleanup (reliably) after this computation terminates? Spawn. Go shares this approach to some extent.
- deleted 11y ago[deleted]
- donatj 11y agoIf you do file system related things in a goroutine it sometimes spins up a new OS thread, and if you are not careful you'll have thousands of them in no time and they never seem to die.
- pcwalton 11y ago> I don't know Scala or Erlang, but based on the results I would assume both are allocating a posix thread for each thread in the software and so it is using the kernel scheduler to switch between them with all the overhead that entails. Incorrect. > I have been considering threading in Rust vs Go and that is the biggest difference I see between the approaches. However it should be possible to implement Go style goroutines in Rust using custom dispatch code like old C coroutines. I think I have already seen libraries like that, but I haven't dug in the details to know for sure. There is https://github.com/dpc/mioco https://github.com/dpc/mioco for I/O-based coroutines. For CPU bound tasks you want something like https://github.com/nikomatsakis/rayon https://github.com/nikomatsakis/rayon.