3 ms·
I’m pretty sure I can explain the decline in investment in 2015. The tl;dr version: Time Warner cooked the books by cutting spending on essential maintenance du
by CodeWriter23 9y ago
I’m pretty sure I can explain the decline in investment in 2015. The tl;dr version: Time Warner cooked the books by cutting spending on essential maintenance during the proposed acquisition by Comcast.
Here’s why I come to that conclusion. In early 2015, I had TWC Business Class installed at our Downtown LA warehouse. One day, the time between scanning a shipment to print the shipping documents and the acknowledgment beep from the system grew from sub-second to 15-30 seconds. The scanning would trigger access to an internet-hosted database to update the order, and would initiate a label request to our postage provider. The label requests would time out. It was huge headache for us.
I begin my investigation by pinging my local router. Nice 1-2ms responses. Pinged the modem, again 1-2ms responses. Tried pinging 8.8.8.8 and WOW! 1200-15000ms response times. FIFTEEN SECONDS for an ICMP request/reply! And a lot of packet loss to boot. I worked around the problem by using my phone as a hot spot for the computers in the warehouse.
I called TWC and since it was Business Class, they rolled a truck within 4 hours. The tech was from a third-party contractor and was very competent by all appearances. He was astonished by my observations. He put his scope on the line in my unit and saw a minor deviation in signal loss. He replaced the drop from the pole to my unit, and said the female receptacle on the pole looked a little worn so he moved the new drop to a new location on the terminal. And the ping times went back.
But the story doesn't end there; in fact this went on for nearly 2 months. During periods of the day, the lag and timeouts would occur. I improved the software to add some retries to the postage provider if the label request failed. I started pinging 8.8.8.8 on a constant basis so I could show the patterns of packet loss and high latency. About the third visit from the same tech, he told me that card in my local node was oversubscribed. He said a single card was intended to serve about 250 customers and that my card had about 600 customers on it.
I hammered on TWC for weeks. I coordinated with my neighbors who understood what "oversubscribed" meant and spoke decent English to do the same. After about 8 weeks, the same tech came knocking and told me the node had been upgraded and wanted to check in. Ping times were great. We had no problems after that.
Sounds like a one-off just bad luck, right? Not so fast. Fast forward about a year. The TWC/Comcast merger is off. I'm sitting at home one day, using my TWC consumer service. Something took to long to load, so I looked a little further, and what do you know? Long ass ping times to 8.8.8.8 and packet loss. I call. I have to wait days for the truck to arrive because I'm not Business Class at home. One tech comes out and observes a few db of signal loss and decides to replace the drop from the pole. I point out the drop was only about 2 years old and was replaced due to a different problem. It's now a two man job, and we wait for the second tech. Once he arrives, they cheerfully install a new drop, and then the problem didn't go away. I told them based on my experience I thought the local node was oversubscribed. They punted and said someone else would come the following day. They arrive with two techs and a lead or engineer. The lead calls various plays to rule out numerous things. After about an hour, they have ruled everything out. He confirms my local node has an excess of subscribers on it but they had not seen it causing this kind of problem before. He pulls up a network schematic on his ToughBook, and isolates the identifier of my local node. Then he switches to - THE SMOKING GUN - a list of about 25 or 30 node identifiers, each with a date in the second column. It's a maintenance schedule to upgrade the nodes that they let languish during the TWC/Comcast acquisition phase. He tells me my node will be upgraded in about 3 months and they leave. About 3 months later, I got a follow up call. Things worked fine, case closed.
I filed a story tip with Ars Technica about this. They never got back to me.
- CASHforGOLD 9y agoHad a similar problem in Seattle. Oversubscribed the node servicing the area. Near 75% packet loss 7am-9pm made running a business impossible. Refused to service, after over 40 phone calls and 6 service appointments. Dropped Comcast for alternate provider. Will never use Comcast again as long as I live for how awful dealing with their customer service was on a daily basis for almost 2 months.