4 ms·
> One of our network engineers made contact with DQE Communications quickly and after a little delay they were able to put us in contact with someone who could
by woliveirajr 7y ago
> One of our network engineers made contact with DQE Communications quickly and after a little delay they were able to put us in contact with someone who could fix the problem. DQE worked with us on the phone to stop advertising these “optimized” routes to Allegheny Technologies Inc. We're grateful for their help. Once this was done, the Internet stabilized, and things went back to normal.
It's funny how we have to still use phone to help fixing some internet routing problem, even if phone doesn't literally means the old black curly-wire equipment
- azernik 7y agoGotta have a channel that's out-of-band with the internet to fix problems with the internet.
- jedberg 7y agoNowadays with voip “the phone” isn’t as out of band as we’d like.
- heywire 7y agoIs it time to put an HF ham radio rig in each major provider’s office? I shudder thinking of a major outage where even phone communication can’t take place. I’m only half joking. Edit: and maybe we just give Verizon a toy walkie talkie
- kortex 7y agoYou joke, but cascade failure ain't no laughing matter, either. Emergency plans aren't for fair weather, they are for sh!tstorms. I don't want to say "we are too reliant on the internet" because it's cliche and connectivity is just part of growing the modern world. But we sure as heck need several layers of backup plans in case things go sideways. I, for one, hope there is a secret society of HAMs lurking as mild-mannered employees at every telco and ISP, ready to wire things back together when they short out.
- minou 7y agoThere is an IRC server (several servers, actually) + channel that a variety of network operators are on that has existed since the very early 2000s for these sorts of events. 414 users on it now, most peoples nicknames include their ASN to make it easier to find each other. Unfortunately, Verizon is one of those networks that isn't present there. But many other networks are represented there and it provides a direct path to those who have config/enable access on some of the largest networks out there. Cuts out the having to go via formal escalation paths and NOC groups that require a trouble ticket before you can engage them.
- carry_bit 7y agoYou can't use the amateur bands for commercial purposes though.
- BuildTheRobots 7y agoI'd love to know how out-of-band telephone networks (landline and mobile) actually are any more. A surprising amount goes via SIP on public IPs.