4 ms·

Does all this garbage traffic affect the performance of Cloudflare's servers? There must be some cost (performance and $$$) to filter this traffic. Was that a c

Does all this garbage traffic affect the performance of Cloudflare's servers? There must be some cost (performance and $$$) to filter this traffic. Was that a consideration when deciding whether to use 1.1.1.1 instead of some other IP address? :)

StreamBright 9y ago
Vendors like Cloudflare have usually really easy to use and cheap measures to drop traffic otherwise they would not be able to provide DDOS protection. Usually they have BGP or other means to propagate blacklisted IP ranges to peers meaning that not even their peers will route the garbage towards them. This is how you can survive DDOS that is bigger than your pipe while serving legit traffic (coming from different ranges than the ones you blacklisted).
eastdakota 9y ago
No. We have a lot of capacity. A lot.
- walrus01 9y ago
  For ordinary singlehomed users who don't get the "a lot". As an example cloudflare has 40Gbps of capacity to the SIX in Seattle. I would guess that they also have direct, at minimum, 10Gbps PNI peering sessions with other huge ISPs in the Pacific Northwest which never see the SIX fabric. So probably add another 20 individual 10GbE circuits at bare minimum to that 40 figure. All of which helps spread the traffic load out rather than shoving it all down a few pipes.
  - pyvpx 9y ago
    https://peeringdb.com/net/4224 https://peeringdb.com/net/4224 that's just the public stuff!
    - walrus01 9y ago
      For huge entities like this and other top 5 CDNs it makes me wonder how many full time staff positions are dedicated to buying rack space, power and crossconnect in major colo facilities worldwide. How many contract law experts, telecom real estate analysts, etc. That before you even get into things like experts in Japanese contract law. Just for rack, power and facilities at layer one of the osi model before any networking happens.
  - jsjohnst 9y ago
    I don’t know the exact number, but you’re off by about an order of magnitude roughly. Cloudflare peering is in the terabits/sec range globally.
    - walrus01 9y ago
      At many non profit IXes the interface size to the fabric is public data and published by both the IX and on peeringdb. Such as at the SIX. They have not yet upgraded to 1x100GbE. Another regional example, they have 20Gbps to the VANIX. What is opaque is the size and scale of their PNI peering, which parties generally don't share. For example in a mid sized city where Comcast is the cable monopoly they almost certainly use a 100GbE interface direct to Comcast for just that isp. Yes the scale is terabits globally. But it is highly decentralized.
      - jsjohnst 9y ago
        > But it is highly decentralized Isn’t that the entire point of a CDN, to have decentralized POPs scattered globally? Yes, they may max out at 100gb per public IX in most cases, but they still have lots of 100gb peers all over the globe.
        walrus01 9y ago
        Yes, it's exactly the point. What I was saying is I am not off by an order of magnitude, I know exactly how big they are. Two ASes I do engineering work for peer directly with cloudflare.
        jsjohnst 9y ago
        > I am not off by an order of magnitude, I know exactly how big they are. Your post implied a total of 200-400gb, the real number is 10x that (otherwise known as “order of magnitude”). I’m not disputing your knowledge or experience, but the post as written has issues.
        9y ago
walrus01 9y ago
Cloudflare is anycast announcing the space from something like 30 to 50 unique POPs worldwide, so the volume of shit traffic is significantly decentralized. It's not like 40 Gbps is hitting one location.
- dx034 9y ago
  They scaled up recently and now have ~150 POPs. They specialised on DDOS defense from the start so traffic volume isn't an issue for them. Everything below 100gbit/s globally will probably be well within normal fluctuations. But I guess they talk to ASNs that send a lot of garbage and ask them to investigate on their side.

StreamBright 9y ago

Vendors like Cloudflare have usually really easy to use and cheap measures to drop traffic otherwise they would not be able to provide DDOS protection. Usually they have BGP or other means to propagate blacklisted IP ranges to peers meaning that not even their peers will route the garbage towards them. This is how you can survive DDOS that is bigger than your pipe while serving legit traffic (coming from different ranges than the ones you blacklisted).

eastdakota 9y ago

No. We have a lot of capacity. A lot.

walrus01 9y ago
For ordinary singlehomed users who don't get the "a lot". As an example cloudflare has 40Gbps of capacity to the SIX in Seattle. I would guess that they also have direct, at minimum, 10Gbps PNI peering sessions with other huge ISPs in the Pacific Northwest which never see the SIX fabric. So probably add another 20 individual 10GbE circuits at bare minimum to that 40 figure. All of which helps spread the traffic load out rather than shoving it all down a few pipes.
- pyvpx 9y ago
  https://peeringdb.com/net/4224 https://peeringdb.com/net/4224 that's just the public stuff!
  - walrus01 9y ago
    For huge entities like this and other top 5 CDNs it makes me wonder how many full time staff positions are dedicated to buying rack space, power and crossconnect in major colo facilities worldwide. How many contract law experts, telecom real estate analysts, etc. That before you even get into things like experts in Japanese contract law. Just for rack, power and facilities at layer one of the osi model before any networking happens.
- jsjohnst 9y ago
  I don’t know the exact number, but you’re off by about an order of magnitude roughly. Cloudflare peering is in the terabits/sec range globally.
  - walrus01 9y ago
    At many non profit IXes the interface size to the fabric is public data and published by both the IX and on peeringdb. Such as at the SIX. They have not yet upgraded to 1x100GbE. Another regional example, they have 20Gbps to the VANIX. What is opaque is the size and scale of their PNI peering, which parties generally don't share. For example in a mid sized city where Comcast is the cable monopoly they almost certainly use a 100GbE interface direct to Comcast for just that isp. Yes the scale is terabits globally. But it is highly decentralized.
    - jsjohnst 9y ago
      > But it is highly decentralized Isn’t that the entire point of a CDN, to have decentralized POPs scattered globally? Yes, they may max out at 100gb per public IX in most cases, but they still have lots of 100gb peers all over the globe.
      - walrus01 9y ago
        Yes, it's exactly the point. What I was saying is I am not off by an order of magnitude, I know exactly how big they are. Two ASes I do engineering work for peer directly with cloudflare.
        jsjohnst 9y ago
        > I am not off by an order of magnitude, I know exactly how big they are. Your post implied a total of 200-400gb, the real number is 10x that (otherwise known as “order of magnitude”). I’m not disputing your knowledge or experience, but the post as written has issues.
        9y ago

walrus01 9y ago

Cloudflare is anycast announcing the space from something like 30 to 50 unique POPs worldwide, so the volume of shit traffic is significantly decentralized. It's not like 40 Gbps is hitting one location.

dx034 9y ago
They scaled up recently and now have ~150 POPs. They specialised on DDOS defense from the start so traffic volume isn't an issue for them. Everything below 100gbit/s globally will probably be well within normal fluctuations. But I guess they talk to ASNs that send a lot of garbage and ask them to investigate on their side.