4 ms·

Edge Network Performance: Akamai, Cloudflare, CloudFront, Fastly and Google

15 points by eastdakota 5y ago

dejangp 5y ago
So the "All networks" TTLB metric for example says CloudFlare is #1 in 5000+ networks, other CDNs are faster in almost 15.000 other networks, meaning CloudFlare is only the fastest in roughly 30% of the networks. Then the blog post ends by saying CloudFlare is not the fastest in 10% of cases which seems simply false. The metric selection as well seems to be selected very much in a way where it's skewing the actual results into making them be much better than they are. A much better result would be to actually show the actual global median response times, which would likely be extremely close to each other between all major providers. Meanwhile this post attempts to make CloudFlare look multiple X better than Akamai for example. Looking at the post in general it seems that it's very much aimed at being as misleading as possible. Maybe it's just me.
- jgrahamc 5y ago
  Then the blog post ends by saying CloudFlare is not the fastest in 10% of cases which seems simply false. It says "we are aiming to improve our performance in 10% of the networks where we are not #1 today". That's not the same as "we're not the fastest in 10% of networks". The network team is working right now with this data to improve in 10% of the networks and we're going to keep going.
  - Zerpiez 5y ago
    Maybe they can also fix the cases where traffic is routed from Amsterdam to Madrid.
  - dejangp 5y ago
    I see, my bad. I guess I was a bit hasty since the metrics kind of bothered me. Still, regarding the other point. Any thoughts on that? We've been monitoring performance of various CDNs ourselves, mainly to improve our own performance through our own RUM metrics as well as platforms such Citrix and PerfOps, and while CloudFlare is definitely one of the fastest, it's definitely not the fastest as per your results. For example CacheFly and Limelight seem to be excelling in the US way ahead of others. Limelight being a multi-hundred million dollar revenue CDN was somehow excluded entirely.
nanankcornering 5y ago
Would be really cool if we could see those metrics per ASNs. Awesome report.
- Zerpiez 5y ago
  Would be nice indeed, but that would reveal their data is flawed. In fact, I'd like to see the data per ASN and per subnet. Because you can have an ASN that spans over a huge area (e.g. all of US), where particular dense populations will show everything is fine, but more rural areas will "suffer". In fact I used to work with a huge adult site, where there were performance metrics down to a /24 subnet level - so in cases like that, performance problems could be investigated quite easily.
  - nanankcornering 5y ago
    That's true. Other than ASN & subnet, would like to see metrics per Cloudflare plans. Enterprise plan have more servers & locations, while the free plan goes to the "cheapest locations"
awoywood 5y ago
Did they measure from the browsers of end users? They said they used the browsers API, but how did they get access to a end user browser in every ISP?
deleted 5y ago
[deleted]
awoywood 5y ago
Also, is the data for every ISP or country available?
1a1a11a 5y ago
there are potentially ethical issues if they did a real measurement from end-user browser. Similar measurement from Academia https://conferences.sigcomm.org/sigcomm/2015/pdf/papers/p653.pdf https://conferences.sigcomm.org/sigcomm/2015/pdf/papers/p653...
- awoywood 5y ago
  Thanks, interesting paper! But their technique only allows to measure if a resource loads or not, how does CF measure time to load on the end user browser?

15 points by eastdakota 5y ago

So the "All networks" TTLB metric for example says CloudFlare is #1 in 5000+ networks, other CDNs are faster in almost 15.000 other networks, meaning CloudFlare is only the fastest in roughly 30% of the networks. Then the blog post ends by saying CloudFlare is not the fastest in 10% of cases which seems simply false. The metric selection as well seems to be selected very much in a way where it's skewing the actual results into making them be much better than they are. A much better result would be to actually show the actual global median response times, which would likely be extremely close to each other between all major providers. Meanwhile this post attempts to make CloudFlare look multiple X better than Akamai for example. Looking at the post in general it seems that it's very much aimed at being as misleading as possible. Maybe it's just me.

jgrahamc 5y ago
Then the blog post ends by saying CloudFlare is not the fastest in 10% of cases which seems simply false. It says "we are aiming to improve our performance in 10% of the networks where we are not #1 today". That's not the same as "we're not the fastest in 10% of networks". The network team is working right now with this data to improve in 10% of the networks and we're going to keep going.
- Zerpiez 5y ago
  Maybe they can also fix the cases where traffic is routed from Amsterdam to Madrid.
- dejangp 5y ago
  I see, my bad. I guess I was a bit hasty since the metrics kind of bothered me. Still, regarding the other point. Any thoughts on that? We've been monitoring performance of various CDNs ourselves, mainly to improve our own performance through our own RUM metrics as well as platforms such Citrix and PerfOps, and while CloudFlare is definitely one of the fastest, it's definitely not the fastest as per your results. For example CacheFly and Limelight seem to be excelling in the US way ahead of others. Limelight being a multi-hundred million dollar revenue CDN was somehow excluded entirely.

nanankcornering 5y ago

Would be really cool if we could see those metrics per ASNs. Awesome report.

Zerpiez 5y ago
Would be nice indeed, but that would reveal their data is flawed. In fact, I'd like to see the data per ASN and per subnet. Because you can have an ASN that spans over a huge area (e.g. all of US), where particular dense populations will show everything is fine, but more rural areas will "suffer". In fact I used to work with a huge adult site, where there were performance metrics down to a /24 subnet level - so in cases like that, performance problems could be investigated quite easily.
- nanankcornering 5y ago
  That's true. Other than ASN & subnet, would like to see metrics per Cloudflare plans. Enterprise plan have more servers & locations, while the free plan goes to the "cheapest locations"

awoywood 5y ago

Did they measure from the browsers of end users? They said they used the browsers API, but how did they get access to a end user browser in every ISP?

deleted 5y ago

[deleted]

awoywood 5y ago

Also, is the data for every ISP or country available?

1a1a11a 5y ago

there are potentially ethical issues if they did a real measurement from end-user browser. Similar measurement from Academia https://conferences.sigcomm.org/sigcomm/2015/pdf/papers/p653.pdf https://conferences.sigcomm.org/sigcomm/2015/pdf/papers/p653...

awoywood 5y ago
Thanks, interesting paper! But their technique only allows to measure if a resource loads or not, how does CF measure time to load on the end user browser?