3 ms·

Wow, thank you for this. Good work! Some thoughts: I've often thought that unbuffered channels would cause scheduling thrashing - higher latency and lower thr

Wow, thank you for this. Good work! Some thoughts: I've often thought that unbuffered channels would cause scheduling thrashing - higher latency and lower throughput because you're swapping between stopping and starting processes blocked on a channel frequently. If you're sending just a small piece of data like 64 bit integer at a time, or any kind of pattern where you're using threads to break up work into tasks, this is too small breakdown of task to really scale multithreading. Want to communicate something that causes a LARGE AMOUNT of work on the other thread, to keep the processor busy. But there's balance between latency and throughput, if you send a big task you get higher latency to react to the next task but better throughput. Walking through my thinking and help me understand: If you have multiple channels that you could read from in a select call but none are ready, could you block that select instance and process a different process where other selects are potentially waiting? This is similar to blocking a goroutine or a Rust async Future task in Tokio that needs to be waked. I think this would need a scheduler. EDIT: Your scheduler to switch between "select instances" or what is running in a thread is the OS.

Galanwe 3y ago
> If you have multiple channels that you could read from in a select call but none are ready, could you block that select instance and process a different process where other selects are potentially waiting? From my experience, the most versatile approach for the wait on channels is best implemented as a spin lock with exponential backoff, up to a threshold that yields - effectively letting other threads perform their own select. Note that most of the time, if you are targeting latency, you would try not to get more workers than cores, meaning you would pin each worker to a core, thus not really benefiting from yielding on select. > But there's balance between latency and throughput, if you send a big task you get higher latency to react to the next task but better throughput. Agree, but that's mainly a client concern, the queue can propose multiple APIs. Most queues handle that with a "burst" mode, where you pop or push multiple messages at once instead of 1 by 1, to balance throughput vs latency.
- jpc0 3y ago
  > a spin lock with exponential backoff Also measure before you implement this yourself, to my knowledge this is infact what the linux mutex does anyway.

Galanwe 3y ago

> If you have multiple channels that you could read from in a select call but none are ready, could you block that select instance and process a different process where other selects are potentially waiting? From my experience, the most versatile approach for the wait on channels is best implemented as a spin lock with exponential backoff, up to a threshold that yields - effectively letting other threads perform their own select. Note that most of the time, if you are targeting latency, you would try not to get more workers than cores, meaning you would pin each worker to a core, thus not really benefiting from yielding on select. > But there's balance between latency and throughput, if you send a big task you get higher latency to react to the next task but better throughput. Agree, but that's mainly a client concern, the queue can propose multiple APIs. Most queues handle that with a "burst" mode, where you pop or push multiple messages at once instead of 1 by 1, to balance throughput vs latency.

jpc0 3y ago
> a spin lock with exponential backoff Also measure before you implement this yourself, to my knowledge this is infact what the linux mutex does anyway.