3 ms·

There should be a measure of consistency as well. Like p90, p95, p99. It's a different measure, but something similar to how we measure system availability in S

There should be a measure of consistency as well. Like p90, p95, p99. It's a different measure, but something similar to how we measure system availability in SWE.

prirun 3y ago
I had a flaky internet connection for a while, where it would have a few seconds of dropout maybe every hour. It was annoying as hell, especially for remote ssh connections. It turned out to be a bad cable from my house to the Spectrum box, which they did fix after 3 visits. Doing the math, this is 75/(24*3600) = .00086, or availability of 99.914%, which sounds great, but in reality it's garbage. Instead of p99, which wouldn't tell us anything really, I'd suggest something like average and total packet loss on a continuous 1-second ping by month for the last year, or even by day for non-zero days. Then if a service has dropout problems it will be easy to spot.
- dtaht 3y ago
  I am a big believer in passive monitoring (see libreqos). Measuring packet loss is a PITA because it is the absence of information. While you can infer it from protocols like TCP, QUIC and VOIP protocols make loss only visible to the end-points. A request response protocol like ping is a lousy metric for measuring voice quality. See irtt. Sending a VOIP-like flow in both directions, one small packet every 20ms, would be vastly better than ping, except there is an energy cost to it.
everforward 3y ago
I hope you'll forgive the nitpick, but those percentiles aren't a measure of consistency, they're a measure of the impact of outliers on the high end. E.g. [30,30,30,30,30,30,30,30,60,60] (8 30's and 2 60's) has the same p90/95/99 as [20,20,20,20,40,40,40,40,60,60] (4 20's, 4 40's, and 2 60's), but they're not equally consistent. They're probably fine here, because I don't think we actually care about consistency (I wouldn't complain if suddenly half my packets had lower latency) we care about the worst case latency and how often it happens. Standard deviation is what I'd use if I was measuring consistency.
- wtallis 3y ago
  This is an extremely important point, given how Bufferbloat arose in large part due to optimizing networks for the wrong performance metrics. There may be some instances where we truly care about consistency in the sense of standard deviation, but for the most part we only care about one side of the distribution.
dtaht 3y ago
yes!

There should be a measure of consistency as well. Like p90, p95, p99. It's a different measure, but something similar to how we measure system availability in SWE.

prirun 3y ago

I had a flaky internet connection for a while, where it would have a few seconds of dropout maybe every hour. It was annoying as hell, especially for remote ssh connections. It turned out to be a bad cable from my house to the Spectrum box, which they did fix after 3 visits. Doing the math, this is 75/(24*3600) = .00086, or availability of 99.914%, which sounds great, but in reality it's garbage. Instead of p99, which wouldn't tell us anything really, I'd suggest something like average and total packet loss on a continuous 1-second ping by month for the last year, or even by day for non-zero days. Then if a service has dropout problems it will be easy to spot.

dtaht 3y ago
I am a big believer in passive monitoring (see libreqos). Measuring packet loss is a PITA because it is the absence of information. While you can infer it from protocols like TCP, QUIC and VOIP protocols make loss only visible to the end-points. A request response protocol like ping is a lousy metric for measuring voice quality. See irtt. Sending a VOIP-like flow in both directions, one small packet every 20ms, would be vastly better than ping, except there is an energy cost to it.

everforward 3y ago

I hope you'll forgive the nitpick, but those percentiles aren't a measure of consistency, they're a measure of the impact of outliers on the high end. E.g. [30,30,30,30,30,30,30,30,60,60] (8 30's and 2 60's) has the same p90/95/99 as [20,20,20,20,40,40,40,40,60,60] (4 20's, 4 40's, and 2 60's), but they're not equally consistent. They're probably fine here, because I don't think we actually care about consistency (I wouldn't complain if suddenly half my packets had lower latency) we care about the worst case latency and how often it happens. Standard deviation is what I'd use if I was measuring consistency.

wtallis 3y ago
This is an extremely important point, given how Bufferbloat arose in large part due to optimizing networks for the wrong performance metrics. There may be some instances where we truly care about consistency in the sense of standard deviation, but for the most part we only care about one side of the distribution.

dtaht 3y ago

yes!