3 ms·
> Cloud computing (i.e. datacenter software) requires zillions of concurrent "fibers" on a much smaller number of OS threads. As a guy who actually wrote "data
by otabdeveloper2 7y ago
> Cloud computing (i.e. datacenter software) requires zillions of concurrent "fibers" on a much smaller number of OS threads.
As a guy who actually wrote "datacenter software" for over a decade: you don't speak from experience and what you said is false.
Basically, OS threads are the smallest and lightest form of concurrency currently in existence. Any userspace 'fiber' will be heavier and slower than actually spawning a pthread.
The problem is when you couple pthreads with interpreter runtimes. (Python, Lua, Ruby, etc.) These runtimes (and especially the GC engines they use!) don't play nice with OS threads, so you're forced to invent various rube-goldberg fiber-like contraptions on async primities.
The end result is slower, heavier and more brittle than any pthread-only solution, but interpreter runtimes necessarily impose overhead, you learn to deal with it.
- networkimprov 7y agoA "fiber" could be a specific state in an async state machine (e.g. a Node.js program) or a distinct stack on a M:N threads mapping (e.g. a Go program). Either is far lighter weight than an OS thread. What userspace fibers are you referring to? I don't have experience with Python, Ruby, or Lua, but IIRC Python has an async mode not unlike Node.js.
- the_why_of_y 7y agoIncorrect. If you don't have to provide POSIX thread semantics, then a user space thread can be much smaller than a OS thread; for example, an Erlang process can be as small as 1.2 KB [1], and a thread in GHC's runtime can be even smaller than that [2, section 4]. With both of these, the programmer writes synchronous code that blocks on IO, and the runtime is responsible for turning that into async IO operations at the OS level. [1] http://erlang.org/doc/efficiency_guide/processes.html http://erlang.org/doc/efficiency_guide/processes.html [2] https://www.microsoft.com/en-us/research/wp-content/uploads/2009/09/multicore-ghc.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
- sriram_malhar 7y agoYou have basically written datacenter software for a decade that may be a few orders of magnitude slower or more inefficient than it needed to be. pthreads is not fibers. Context switching between fibers can be extraordinarily fast because the kernel is not involved. I have some experience in the matter.