4 ms·

Basically, you start by getting giant piles of cash. You then start trying to build out new network assets. Then you run into the incumbents cheating on the p

by bcrl 4y ago

Basically, you start by getting giant piles of cash. You then start trying to build out new network assets. Then you run into the incumbents cheating on the permitting processes and utilizing market power to take or retain customers. I've been through this. And the municipalities don't even own up to their obligations under CRTC frameworks and delay MAAs for a year "just because". It's painful. Becoming the third player is an uphill battle in all regards. Fibre customers are very sticky, and getting people to change is nearly impossible. Bell is particularly awful as a competitor as they steeply discount the first 1-2 years: want gigabit fibre with all the TV channels and phone service for $60/month along with a $200 prepaid Visa gift card? Sure! Then month 13 comes along and the bill becomes $300 per month. As a competitor, there is absolutely no way I can offer a similar package or retain a customer when they take that package. Want that deal as an existing customer? No way! On top of all that, Canada is a relatively small market that is very spread out geographically. If you want to provide coverage similar to existing carriers, you have to build a lot of infrastructure. Nobody can afford to do that. Plus there are buckets of cash being rained down on other market players under a number of programs, so your competitor may be running on assets that cost 50-100% below your cost to build a similar infrastructure. The easiest thing for you to do is to get 2 internet connections from different carriers that do not share networks. In most markets the telco and cableco run different networks. My Telus and Bell services continued working fine yesterday. Heck, my 3 different Rogers wavelength services we not impacted at all (although L1 services have other failure modes that take longer to recover from since there is no automatic rerouting as with L2 networks). I'm pretty sure you really don't want to start down this path...

xist 4y ago
As someone in the industry thank you for saying this! The regulatory demands in Canada are much different than elsewhere in the world, and greatly outweigh the technical demands
kevin_nisbet 4y ago
> The easiest thing for you to do is to get 2 internet connections from different carriers that do not share networks. This is not as easy as it may seem, as it's common for the carriers to lease from each other in the backbones. Telus and Bell for cellular services share a radio network, each cell site connects to both the Bell and Telus core networks. The big Bell backbone outage in the Toronto area, 6 or 7 years ago, had impacts on all of Bell, Telus, and Rogers networks. As for cellular services we had glitches nationwide that had to be fixed during that backbone outage.
- icehawk 4y ago
  And even if they don't share networks they might share physical space. I had an incident where two different trans-Atlantic links from two different providers, shared a 20 mile stretch of conduit. So, of course, that's exactly where the fiber cut happened.
- bcrl 4y ago
  In the case of the Rogers outage, other carriers / ISPs were not offline. The Rogers problem appeared to be entirely in L2/L3 -- switches and routers were not configured correctly. The other telcos were not impacted as they run their own routing infrastructure. Basically, make sure your 2 ISPs have their own ASNs and transit providers.
  - kevin_nisbet 4y ago
    Sorry, I think you missed the point I was trying to share. The large providers all lease infrastructure from each other, so it's unsafe to assume that using two providers creates a total diversity in access. But you're correct, in that failures can happen at different layers, that may isolate outages to one provider or even a subset of customers. The outage I cited, only occurred in the Bell Wireline Core network, but caused service impacts in Bell, Telus, Rogers and other networks for Cellular, Internet, etc services.
    - bcrl 4y ago
      Again, it depends on what you're leasing from other carriers. Yes, if you're purchasing an L2 path from a carrier you're relying on their switches and routers to pass traffic correctly, but you bypass their routers and switches by purchasing an L1 wavelength circuit between 2 points. That's one of the main reasons that wavelength circuits are used (in addition to the guaranteed capacity). Third party carriers using fibre local loops from another carrier would not have gone down like TPIA users did on Rogers' network yesterday. I strongly believe that access to fibre local loops is needed in Canada. Leased copper local loops enabled plenty of competition between small local ISPs in the 1990s and early 2000s while they could still deliver speeds that were sufficient for internet users of the time. Having a tariff for local fibre circuits similar to the old LDDS tariff would bring back that vibrant competition we once had. Bonus: it's a lot harder for incumbents to put together bogus cost studies as compared to the failed disaggregated regime the CRTC has wasted nearly a decade on now.
icehawk 4y ago
Heck, my 3 different Rogers wavelength services we not impacted at all (although L1 services have other failure modes that take longer to recover from since there is no automatic rerouting as with L2 networks). Sort of, our wave circuits will automatically reroute but we had to shell out the $$ for protected waves and then explicitly mention they had to be configured to be auto-revertive as that was not the default behavior for that telecom provider. Then L2 and L3 don't notice when it shifts between the working and protect paths, it happens fast enough the only visible change then is the latency difference between the two.
- bcrl 4y ago
  In some regards it's better to get another wavelength circuit going to another city to pick up geographically diverse transit.
  - icehawk 4y ago
    True, our use case is a bit different as the waves go between PoPs/DCs/Offices so the only diversity can be in the path itself. Faster to have the path switch over and have L3 re-converge only when the working and protected path are unavailable.

by bcrl 4y ago

xist 4y ago

As someone in the industry thank you for saying this! The regulatory demands in Canada are much different than elsewhere in the world, and greatly outweigh the technical demands

kevin_nisbet 4y ago

> The easiest thing for you to do is to get 2 internet connections from different carriers that do not share networks. This is not as easy as it may seem, as it's common for the carriers to lease from each other in the backbones. Telus and Bell for cellular services share a radio network, each cell site connects to both the Bell and Telus core networks. The big Bell backbone outage in the Toronto area, 6 or 7 years ago, had impacts on all of Bell, Telus, and Rogers networks. As for cellular services we had glitches nationwide that had to be fixed during that backbone outage.

icehawk 4y ago
And even if they don't share networks they might share physical space. I had an incident where two different trans-Atlantic links from two different providers, shared a 20 mile stretch of conduit. So, of course, that's exactly where the fiber cut happened.
bcrl 4y ago
In the case of the Rogers outage, other carriers / ISPs were not offline. The Rogers problem appeared to be entirely in L2/L3 -- switches and routers were not configured correctly. The other telcos were not impacted as they run their own routing infrastructure. Basically, make sure your 2 ISPs have their own ASNs and transit providers.
- kevin_nisbet 4y ago
  Sorry, I think you missed the point I was trying to share. The large providers all lease infrastructure from each other, so it's unsafe to assume that using two providers creates a total diversity in access. But you're correct, in that failures can happen at different layers, that may isolate outages to one provider or even a subset of customers. The outage I cited, only occurred in the Bell Wireline Core network, but caused service impacts in Bell, Telus, Rogers and other networks for Cellular, Internet, etc services.
  - bcrl 4y ago
    Again, it depends on what you're leasing from other carriers. Yes, if you're purchasing an L2 path from a carrier you're relying on their switches and routers to pass traffic correctly, but you bypass their routers and switches by purchasing an L1 wavelength circuit between 2 points. That's one of the main reasons that wavelength circuits are used (in addition to the guaranteed capacity). Third party carriers using fibre local loops from another carrier would not have gone down like TPIA users did on Rogers' network yesterday. I strongly believe that access to fibre local loops is needed in Canada. Leased copper local loops enabled plenty of competition between small local ISPs in the 1990s and early 2000s while they could still deliver speeds that were sufficient for internet users of the time. Having a tariff for local fibre circuits similar to the old LDDS tariff would bring back that vibrant competition we once had. Bonus: it's a lot harder for incumbents to put together bogus cost studies as compared to the failed disaggregated regime the CRTC has wasted nearly a decade on now.

icehawk 4y ago

Heck, my 3 different Rogers wavelength services we not impacted at all (although L1 services have other failure modes that take longer to recover from since there is no automatic rerouting as with L2 networks). Sort of, our wave circuits will automatically reroute but we had to shell out the $$ for protected waves and then explicitly mention they had to be configured to be auto-revertive as that was not the default behavior for that telecom provider. Then L2 and L3 don't notice when it shifts between the working and protect paths, it happens fast enough the only visible change then is the latency difference between the two.

bcrl 4y ago
In some regards it's better to get another wavelength circuit going to another city to pick up geographically diverse transit.
- icehawk 4y ago
  True, our use case is a bit different as the waves go between PoPs/DCs/Offices so the only diversity can be in the path itself. Faster to have the path switch over and have L3 re-converge only when the working and protected path are unavailable.