3 ms·

Maybe I'm misunderstanding something but it being a blocking call does not make timeouts less important -- if anything they become more important!

• tptacek 11mo ago
  I don't understand how it is you're doing distributed systems design on a system you don't even have access to. Maybe the issue is timeouts, maybe the issue is some other technical change, maybe the issue is human/procedural. How could you possibly know? The owners of the system probably don't have a full answer within just a few hours of handling the incident!
  • kqr 11mo ago
    I would be worried if they had all the answers within a few hours! I was just caught off guard by the focus on technical control measures when there seems to have been fairly obvious problems with information channels. For example, "more global kill switches for features" is good, but would "only" have shaved 30 % off the time of recovery (if reading the timeline charitably). Being able to identify the broken component faster would have shaved 30–70 % off the time of recovery depending on how fast identification could happen – even with no improvements to the kill switch situation.
• vlovich123 11mo ago
  It’s literally in the same process as the router. Performance issues like the layer taking too long are handled differently than just timing out afaik. It’s possible it should time out but safe cancellation in Rust can be tricky depending on how this is architected. Timeouts are irrelevant to this outage and I wouldn’t classify it as a huge issue unless there’s a possibility that a layer has an indefinite blocking call as a misbehavior and forever hangs all requests. My hunch is that this is generally already handled - FL2 is not documented enough to speculate.