3 ms·

Running a fully utilized fiber line is.. exactly the same amount of labor as running a 10% utilized one. The cost to track and bill people for caps and let the

Running a fully utilized fiber line is.. exactly the same amount of labor as running a 10% utilized one. The cost to track and bill people for caps and let them pay overages though..

toomuchtodo 3y ago
Chokepoints are peering exchanges, transit handoffs, and core network gear, which is sized for a percentile of utilization below constant 100%. Having implemented a billing system and utilization monitoring for managed hosting datacenter environment, it is trivial to collect metrics from equipment and bill accordingly. You don't want to nickle and dime or gouge customers, but you do want to ensure proportionality across the customer population. Some combination of caps, overage charges, traffic shaping, and edge CDN appliances is needed (Netflix, Akamai, etc). Off peak unlimited is also a potential strategy to drive low priority transfer to low utilization periods. Pick your poison. There is a reason most of Netflix global infra is monster CDN boxes closest to customers. https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/
- throitallaway 3y ago
  > There is a reason most of Netflix global infra is monster CDN boxes closest to customers. Because it would be ridiculously expensive to egress video streams directly from AWS. They've determined that it's cheaper to pay ISPs for rack space.
  - toomuchtodo 3y ago
    You misunderstand the model. They don't pay ISPs, they offer the appliance for free to the ISP for colocation, reducing the upstream networking costs. AWS egress being extortionate is a distinct issue. The cost to the ISP in the rack (cooling, power, space) is immaterial considering the transfer savings (not only opex, but reduced capex from needing to oversize network gear or specific ports for peak streaming consumption) over the life of the relationship. If you want to learn more, lurk on the NANOG (North American Network Operators' Group) mailing list. It is common to monitor AS traffic flows to understand whether you can offload with CDN appliance relationships or how to improve your network architecture (which networks to peer to directly or which internet exchanges [IXs] to meetup at). These agreements sometimes happen at a bar over beers during NANOG events. Call Verizon and ask them how much a 1Gbps circuit is. Consider why Comcast built their own national backbone. Also, some important history to remember. Fast.com measures your speeds to Netflix servers because some ISPs wanted to charge their customers and Netflix for that transfer (or to upgrade peering points) because of the revenue destruction from cable customers fleeing for streaming (net neutrality debate). https://archive.nanog.org/meetings/nanog45/presentations/Norton_Peering101_v0_N45.pdf https://archive.nanog.org/meetings/nanog45/presentations/Nor... (Peering 101, Bill Norton [Equinix cofounder], NANOG 45) https://drpeering.net/white-papers/Video-Internet-The-Next-Wave-Of-Massive-Disruption-To-The-U.S.-Peering-Ecosystem.html https://drpeering.net/white-papers/Video-Internet-The-Next-W... (Video Internet: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem (v1.7)) https://drpeering.net/HTML_IPP/ipptoc.html https://drpeering.net/HTML_IPP/ipptoc.html ("The Internet Peering Playbook") https://arstechnica.com/features/2008/09/peering-and-transit/ https://arstechnica.com/features/2008/09/peering-and-transit... ("How the ‘Net works: an introduction to peering and transit") https://arstechnica.com/information-technology/2014/07/how-comcast-became-a-powerful-and-controversial-part-of-the-internet-backbone/ https://arstechnica.com/information-technology/2014/07/how-c... ("How Comcast became a powerful—and controversial—part of the Internet backbone") https://openconnect.zendesk.com/hc/en-us/articles/360036163652-How-much-does-the-appliance-cost-my-organization- https://openconnect.zendesk.com/hc/en-us/articles/3600361636... > How much does the appliance cost my organization? > Appliances, including replacement appliances, are provided to qualifying ISPs at no charge when used within the terms of the license agreements.

Running a fully utilized fiber line is.. exactly the same amount of labor as running a 10% utilized one. The cost to track and bill people for caps and let them pay overages though..

toomuchtodo 3y ago

Chokepoints are peering exchanges, transit handoffs, and core network gear, which is sized for a percentile of utilization below constant 100%. Having implemented a billing system and utilization monitoring for managed hosting datacenter environment, it is trivial to collect metrics from equipment and bill accordingly. You don't want to nickle and dime or gouge customers, but you do want to ensure proportionality across the customer population. Some combination of caps, overage charges, traffic shaping, and edge CDN appliances is needed (Netflix, Akamai, etc). Off peak unlimited is also a potential strategy to drive low priority transfer to low utilization periods. Pick your poison. There is a reason most of Netflix global infra is monster CDN boxes closest to customers. https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/

throitallaway 3y ago
> There is a reason most of Netflix global infra is monster CDN boxes closest to customers. Because it would be ridiculously expensive to egress video streams directly from AWS. They've determined that it's cheaper to pay ISPs for rack space.
- toomuchtodo 3y ago
  You misunderstand the model. They don't pay ISPs, they offer the appliance for free to the ISP for colocation, reducing the upstream networking costs. AWS egress being extortionate is a distinct issue. The cost to the ISP in the rack (cooling, power, space) is immaterial considering the transfer savings (not only opex, but reduced capex from needing to oversize network gear or specific ports for peak streaming consumption) over the life of the relationship. If you want to learn more, lurk on the NANOG (North American Network Operators' Group) mailing list. It is common to monitor AS traffic flows to understand whether you can offload with CDN appliance relationships or how to improve your network architecture (which networks to peer to directly or which internet exchanges [IXs] to meetup at). These agreements sometimes happen at a bar over beers during NANOG events. Call Verizon and ask them how much a 1Gbps circuit is. Consider why Comcast built their own national backbone. Also, some important history to remember. Fast.com measures your speeds to Netflix servers because some ISPs wanted to charge their customers and Netflix for that transfer (or to upgrade peering points) because of the revenue destruction from cable customers fleeing for streaming (net neutrality debate). https://archive.nanog.org/meetings/nanog45/presentations/Norton_Peering101_v0_N45.pdf https://archive.nanog.org/meetings/nanog45/presentations/Nor... (Peering 101, Bill Norton [Equinix cofounder], NANOG 45) https://drpeering.net/white-papers/Video-Internet-The-Next-Wave-Of-Massive-Disruption-To-The-U.S.-Peering-Ecosystem.html https://drpeering.net/white-papers/Video-Internet-The-Next-W... (Video Internet: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem (v1.7)) https://drpeering.net/HTML_IPP/ipptoc.html https://drpeering.net/HTML_IPP/ipptoc.html ("The Internet Peering Playbook") https://arstechnica.com/features/2008/09/peering-and-transit/ https://arstechnica.com/features/2008/09/peering-and-transit... ("How the ‘Net works: an introduction to peering and transit") https://arstechnica.com/information-technology/2014/07/how-comcast-became-a-powerful-and-controversial-part-of-the-internet-backbone/ https://arstechnica.com/information-technology/2014/07/how-c... ("How Comcast became a powerful—and controversial—part of the Internet backbone") https://openconnect.zendesk.com/hc/en-us/articles/360036163652-How-much-does-the-appliance-cost-my-organization- https://openconnect.zendesk.com/hc/en-us/articles/3600361636... > How much does the appliance cost my organization? > Appliances, including replacement appliances, are provided to qualifying ISPs at no charge when used within the terms of the license agreements.