6 ms·
> that presumably shows up in the logs Ah, no. SNR problems in the shared medium is not something wathced or logged. I remember losing internet connection on h
by bayindirh 6y ago
> that presumably shows up in the logs
Ah, no. SNR problems in the shared medium is not something wathced or logged. I remember losing internet connection on heavy rains sometimes because an odd cable shorted at an underground gallery. It wasn't FTTX then, so it was not easy to find.
> You have a problem that's repeatable right on schedule.
Similar problems rooted to bandwidth scarcity happened all the time, so in a village I'm not sure there's a lot of people who can discern the difference between, ah it's slow again and Oh, I lost connection, look at that SNR value. I better write a technical report to ISP.
> It should be a no-brainer to send someone with test equipment out there to measure things in the field while the problem was happening.
Even the simplest problem, if it's happening 1 in 10.000 is not a no-brainer because, you won't think that'd happen. Especially because of an old TV.
- Nextgrid 6y ago> SNR problems in the shared medium is not something wathced or logged This is a problem then and should be addressed. > problems rooted to bandwidth scarcity happened all the time But the ISP should be able to tell whether their uplink is saturated (and actually upgrade it if it happens too often), so if they're consistently getting complaints around a certain timeframe and the uplink is not saturated during those timeframes then it must be something else. > Especially because of an old TV But the diagnostic procedure had nothing to do whether it's an old TV or something else. It was caused by EMI and is detectable with readily-available test equipment which indeed detected it immediately. Furthermore monitoring packet loss at the PPP level can be done (in fact, Andrews & Arnold - a small ISP in the UK - does it) so if an ISP doesn't have this in place to detect widespread faults then something is also wrong there - why are they operating with zero visibility on their network?
- bayindirh 6y ago> But the diagnostic procedure had nothing to do whether it's an old TV or something else. It was caused by EMI and is detectable with readily-available test equipment which indeed detected it immediately. They concluded that something is fishy because normal procedures didn't fix anything. I'm not sure that this is a problem they diagnose 5 times a week. > But the ISP should be able to tell whether their uplink is saturated (and actually upgrade it if it happens too often), so if they're consistently getting complaints around a certain timeframe and the uplink is not saturated during those timeframes then it must be something else. Good luck escalating this problem from a village to correct people in a big-shiny-ISP(TM). They're too busy(!) to look for 10 customers' problem. I accept that 18 months in unacceptable but, this is the reality we're in. Support and Ops teams are always overworked because management thinks that "the load just doesn't justify more hands in that department". > This is a problem then and should be addressed. The backbone operators and ISPs doesn't monitor SNR levels because they're transient and doesn't provide meaningful data. If you want to do it, it's millions of connections. Also, Not everyone use the standard equipment provided by the ISP so, they cannot get info from all devices (e.g.: from mine). So, endpoint ops are complex if you're working on a WAN with random equipment. Instead, using bandwidth graphs to find problems is more practical and reliable. Since the problem starts at 07:00, it shows like "They're not using their connections much, eh?" in the said graphs. Disclosure: My workplace also happens to be the national academic network operator. I'm two hops away from a country-wide core router.
- Nextgrid 6y ago> because normal procedures didn't fix anything But why did it take 18 months to realize that? You've got an issue that occurs consistently and is preventing customers from getting the service they're paying for (and are presumably in contracts so they can't even cancel without a major fight). > They're too busy(!) to look for 10 customers' problem So we're in agreement that this is unacceptable and resolving this problem has been grossly mismanaged and someone should be held accountable? > doesn't monitor SNR levels because they're transient and doesn't provide meaningful data This seems weird to me. You've got a nationwide network designed to run on old, poor quality wiring (the article doesn't specify whether it's DSL or coax, but it would apply to both). Wiring issues should be expected and automated processes in place to monitor it and quickly detect issues like this one. > Not everyone use the standard equipment provided by the ISP Can't this be collected at the DSLAM or CMTS level? Or in case of DSL, even at the PPP level? (Andrews & Arnold - a small DSL ISP in the UK - measures packet loss at that level and while it's not true SNR, it would be enough for these purposes as localized packet loss would pretty much always mean bad SNR or failing equipment)
- donalhunt 6y agoI can guarantee that the customer reports came in as "my broadband isn't working again this morning...". And that's for the customers who called in. Unless you had 50+% of the village calling up every morning for a week, it would not have been identified as a cluster event (and I suspect many ops orgs don't have the means to identify clusters anyway).
- brutusborn 6y agoWhat does the (!) after "busy" mean? P.s. Thankyou for sharing your experience, it's very insightful.
- nefitty 6y agoThey’re probably using it to express incredulity. “The mechanic told me it would cost $5000(!) to fix, so I decided to scrap the car instead.”
- deleted 6y ago[deleted]
- C1sc0cat 6y agoThis is openretch (openreach) not Virgin so not sure where you get shared medium from I do know people who worked in radio interference and they told me one time they traced a fault to a Budgie cage. The Budgie had a swing and when it was in use it caused a short which effected the subs (customer) line.
- bayindirh 6y agoIf you're running network over coax alongside cable TV, it's shared medium. I'm not in the UK or US though, so regardless of their name, every ISP is equal and is indeed a bog standard ISP from my vantage point.
- milankragujevic 6y agoBT openreach mainly does DSL and FTTH, AFAIK. edit: Just to clarify, FTTH is by definition a shared medium, but the huge capacity in practice means that there is little congestion usually. Also, since glass fiber is a dielectric, it is difficult to get noise in the signal, and it is immune to EMI. DSL is by definition a dedicated medium, having a twisted pair wire from the DSLAM at the central office or street cabinet to the user premises into the [splitter and] modem. One user per twisted pair only. However, the wires are not shielded and thus are susceptible to both crosstalk from other pairs and also EMI given that it's using electrical signals to transfer data. And of course, the speeds are low for most users due to the local loop length and in general VDSL2 35b maxes out at 300/100 Mbps at around ~80 meters, and VDSL2 17a maxes out at 100/50 Mbps at around 100 meters.
- LargoLasskhyfv 6y agoBy that you mean GPON, I guess? Not so here, where it is PTP. And an ISP with excellent peering.
- milankragujevic 6y agoEdit: disregard my comment, I mistook PTP to mean Ethernet while in reality it's Point to point Fiber. In any case my point about noise still stands. But PTP Fiber is really rare in the world, given how many people are stuck on coax. Yes, with FTTH (or FTTP) I meant GPON. Refarding PTP, do you mean a residential ethernet network? (Technically FTTB, copper / ethernet to the premises) Branding anything other than FTTH as "Fiber" is misleading. It's like saying DOCSIS 3.0 is Fiber because its a Hybrid Fiber-Coaxial network, while in reality all cable internet networks are HFC, the difference is the number of nodes serviced by a CMTS, which when reduced by a node split will result in higher speeds and more capacity. Or saying DSL is "Fiber" because uplink to the DSLAM is Fiber. All DSLAM uplinks are Fiber, difference is how many street cabinets are there because DSL speed is dictated by the local loop length, and with more street cabinets there is higher chance one gets built closer to you enabling you faster speeds.
- lima 6y ago> Ah, no. SNR problems in the shared medium is not something wathced or logged. A good ISP definitely does monitor SNR. It's a basic metric available from each individual customer device.
- bayindirh 6y ago> It's a basic metric available from each individual customer device. Not mine. Good luck accessing to the modem you provided which I reconfigured to be a dumb bridge and terminated the ADSL connection on my router. Oh, the router neither see the POTS line (it only gets Ethernet) nor have remove management capabilities. Where endpoints are heterogeneous, there's guaranteed, real fun (Even more so if you make assumptions).
- deleted 6y ago[deleted]
- lima 6y agoThey still have SNR metrics on their end of the connection. And this is why providers hate heterogeneous networks :-) Mine politely asks you to plug in the original modem before they'll help you debug a connectivity issue. But even then, there are sometimes standards like TR-069 that apply across vendors.
- JosephRedfern 6y ago> Ah, no. SNR problems in the shared medium is not something wathced or logged. I remember losing internet connection on heavy rains sometimes because an odd cable shorted at an underground gallery. It wasn't FTTX then, so it was not easy to find. Even if SNR itself isn't logged, dynamic line management would surely result in dropped speeds, which would be logged, and would stand out as anomalous?
- tonyb 6y agoThis isn’t accurate. The amount of data available on a DOCSIS network is incredible. Per modem SNR, power levels, preEq data, etc. If the data is used properly it cans find a loose screw on a tap or a damaged cable down the the foot. That being said impulse noise is one of the harder things to track down. If the modern tools don’t work most MSOs would end up doing the “unplug a leg and see if the SNR gets better” method until they found the offending house.