4 ms·
How does gevent handle spawning kernel threads? Last time I tried Go I found that it spawned unlimited kernel threads. For example, a thousand goroutines callin
by millstone 8y ago
How does gevent handle spawning kernel threads? Last time I tried Go I found that it spawned unlimited kernel threads. For example, a thousand goroutines calling stat() would result in as many kernel threads.
I ended up tracking down which goroutines were likely to spawn kernel threads (gross global reasoning), and applied a rate-limiter to that set.
- meowface 8y agoUnfortunately, Python is still bound by the GIL, so to my understanding, gevent will never spawn any kernel threads. This effectively means there is no true parallelism possible; context switches between greenthreads pretty much only occur when waiting on I/O. I believe this is the case for all other Python concurrency libraries as well, including the threading library in Python's standard lib. Most people, including myself, use gevent for I/O bound applications (where it excels), so this usually doesn't pose any issues. For CPU bound tasks where performance is important, I'd probably use something other than Python (likely Go, personally).