3 ms·
classic case of head of line blocking!
by xtoilette 1y ago
classic case of head of line blocking!
- yuliyp 1y agoI don't think this is head-of-line blocking. That is, it's not like a single slow request causes starvation of other requests. The IO thread for the connection is grabbing and dispatching data to workers as fast as it can. All the requests are uniform, so it's not like one request would be bigger/harder to handle for that thread.
- otterley 1y ago> First, we checked the number of TCP connections using lsof -i TCP:2137 and found that only a single TCP connection was used regardless of in-flight count. It's head-of-line blocking. When requests are serialized, the queue will grow as long as the time to service a request is longer than the interval between arriving requests. Queue growth is bad if sufficient capacity exists to service requests in parallel.
- voxic11 1y agoRequests are not serialized though. Http2/GRPC multiplexs multiple requests over one TCP connection.
- otterley 1y agoThe TCP connection itself is a single bidirectional queue. So if packets are lost or delayed, then multiplexing doesn't buy you anything.
- yuliyp 1y agoI guess I'd thought of head-of-line-blocking as the delay from a slow request stalling subsequent ones beyond the throughput limits of the system: i.e. a slow-to-parse request causes other cheap requests to wait.
- otterley 1y agoEverything in a queue is subject to head-of-line blocking by definition; it's really a question of whether the wait time meets your requirements.