5 ms·

I highly doubt your claims about Cloudflare being an order of magnitude slower, they are consistently one of the fastest CDNs: https://www.cdnperf.com/ https:/

I highly doubt your claims about Cloudflare being an order of magnitude slower, they are consistently one of the fastest CDNs: https://www.cdnperf.com/ https://www.cdnperf.com/ Top 15: 1 Google Cloud CDN 36.70 ms 2 jsDelivr CDN 36.80 ms 3 Akamai CDN 38.00 ms 4 Verizon (Edgecast) CDN 38.30 ms 5 Azure CDN 38.50 ms 6 Fastly CDN 41.38 ms 7 Cloudflare CDN 41.80 ms 8 AWS CloudFront CDN 43.00 ms 9 CacheFly 43.90 ms 10 BunnyCDN 46.00 ms 11 StackPath CDN 46.53 ms 12 KeyCDN 47.00 ms 13 CDN.NET 48.46 ms 14 G-Core Labs CDN 48.74 ms 15 Quantil CDN 50.20 ms

onlyrealcuzzo 7y ago
Everything about what the top comment said seems like Fastly marketing propaganda. 7% increase in sales? With what sample size? With what confidence? How did you isolate variables? For reference, I work at a company with $110M in annual sales. We were planning to start using Fastly. For obvious reasons, we wanted to know how much of a net positive that would be for the business. So we wanted to A/B test it. For us, at least, it's not as straight forward as it would seem.
- tylermenezes 7y ago
  You can Google my name if you think I'm working for Fastly. The only connection I have is that they are our CDN. They donate it, as we're a nonprofit, but I have zero obligation to say anything about them either way. 7% increase represents an increase in the number of students/parents purchasing tickets to our events. (It's not like they're spending more individually, but more people have been willing to give us money.) That's based on several years with no substantial change in the way we market or the design of our site.
  - icedchai 7y ago
    How do you know the 7% increase was due to Fastly and not some other small unrelated change to your web site? A few 10 ms makes no difference.
    - wongarsu 7y ago
      > A few 10 ms makes no difference. About a 1-2% difference per 100 ms seems well supported (e.g. [1]), at least if your page load time is already low enough. 7% is very high, but some effect is expected. 1: https://developers.google.com/web/fundamentals/performance/why-performance-matters/ https://developers.google.com/web/fundamentals/performance/w...
      - jjeaff 7y ago
        I don't buy it. So what you are saying is 2% for every 100ms. So that means if we decrease page load time by 1ms, then we should expect to to see .02% increase in sales? So if we do $10m annually, 1ms decrease in sales should boost sales by $2,000! We would all probably agree that 1ms will make no statistical difference. The problem with these studies is that most are dealing with much longer load times. Like 3 seconds vs 19 seconds! Obviously, that will make a HUGE difference. You can't then extrapolate that down to the millisecond. The other problem is that many of these studies are basing their numbers on average load times. So they are comparing two groups and averaging load time. Group A averages 100ms faster than group B. And group B increased sales by 2%. But what really happened is group A had 800ms load times across the board. And group be had 800ms load times for 98% of their page loads and 20,000ms on the remaining 2%. So working with averages can be largely misleading. I can't see the details of the 1 study that claims 100ms increments, but I'm very skeptical.
        wongarsu 7y ago
        Yes, the 1ms will likely be statistically insignificant. But so are $2000 extra income out of your $10m. If anything, you have a better hope measuring the 1ms than the $2000. I agree with your larger point that average latency is not a good measure. Even for the perception of a single user consistency is more important than a good average. For large groups the average is even less useful. But intuitively I see lots of places where 100ms makes a world of difference. Just like there's a big, very perceptible difference between a 200ms animation and a 300ms animation, a time to full render of 200ms can change your experience compared to 300ms. The slower the page load, the more deliberate your movement. The closer you get to 16ms (1 frame) for a page load the smaller the investment for clicking a link, and the higher the willingness to experiment and explore. Some of that inevitably leads to conversions and sales.
  - dna_polymerase 7y ago
    > 7% increase represents an increase in the number of students/parents purchasing tickets to our events. (It's not like they're spending more individually, but more people have been willing to give us money.) We got that part, but correlation doesn't always imply causation. Maybe that change was seasonal or just plain old growth not accounted for in your 7%?
foobarbazetc 7y ago
CDNPerf is an okay way to get an overview, but not a great way to actually compare CDNs. That said, it’s not 10x or whatever. Maybe like 5-10% depending on location.
- bufferoverflow 7y ago
  Do you know of a better way than CDNPerf? They do 300 million requests per day from random users from around the world.
  - briffle 7y ago
    Cedexis is also a good tool to measure, while most of it is a paid product, they have a few reports you can see. they also list cloud providers, and have other reports such as throughput, etc. https://www.cedexis.com/get-the-data/country-report/ https://www.cedexis.com/get-the-data/country-report/
  - acdha 7y ago
    Is there any documentation on how they measure it? If they're running tests from data centers / peering points or testing things which are highly-likely to be cached the results are going to be harder to generalize to normal usage.
- tylermenezes 7y ago
  There are a number of features Fastly has which others (excluding eg Akmai) don't which make head-to-head comparisons more difficult. I have no doubt that Cloudflare is very fast in perfect conditions, but there were a much higher number of cache misses because there's less flexible configuration which did literally make the difference between 20ms vs 200ms in production.
  - anonymoushn 7y ago
    Can you elaborate on this? Enterprise customers are generally able to get their SE to set whatever cache key they like, and if that is insufficient they can use the worker cache API: https://developers.cloudflare.com/workers/reference/cache-api/ https://developers.cloudflare.com/workers/reference/cache-ap... As I understand it a charity may get the former thing for free through https://www.cloudflare.com/galileo/ https://www.cloudflare.com/galileo/ , but the latter probably still has usage-based billing. Disclosure: I work for Cloudflare.

onlyrealcuzzo 7y ago

Everything about what the top comment said seems like Fastly marketing propaganda. 7% increase in sales? With what sample size? With what confidence? How did you isolate variables? For reference, I work at a company with $110M in annual sales. We were planning to start using Fastly. For obvious reasons, we wanted to know how much of a net positive that would be for the business. So we wanted to A/B test it. For us, at least, it's not as straight forward as it would seem.

tylermenezes 7y ago
You can Google my name if you think I'm working for Fastly. The only connection I have is that they are our CDN. They donate it, as we're a nonprofit, but I have zero obligation to say anything about them either way. 7% increase represents an increase in the number of students/parents purchasing tickets to our events. (It's not like they're spending more individually, but more people have been willing to give us money.) That's based on several years with no substantial change in the way we market or the design of our site.
- icedchai 7y ago
  How do you know the 7% increase was due to Fastly and not some other small unrelated change to your web site? A few 10 ms makes no difference.
  - wongarsu 7y ago
    > A few 10 ms makes no difference. About a 1-2% difference per 100 ms seems well supported (e.g. [1]), at least if your page load time is already low enough. 7% is very high, but some effect is expected. 1: https://developers.google.com/web/fundamentals/performance/why-performance-matters/ https://developers.google.com/web/fundamentals/performance/w...
    - jjeaff 7y ago
      I don't buy it. So what you are saying is 2% for every 100ms. So that means if we decrease page load time by 1ms, then we should expect to to see .02% increase in sales? So if we do $10m annually, 1ms decrease in sales should boost sales by $2,000! We would all probably agree that 1ms will make no statistical difference. The problem with these studies is that most are dealing with much longer load times. Like 3 seconds vs 19 seconds! Obviously, that will make a HUGE difference. You can't then extrapolate that down to the millisecond. The other problem is that many of these studies are basing their numbers on average load times. So they are comparing two groups and averaging load time. Group A averages 100ms faster than group B. And group B increased sales by 2%. But what really happened is group A had 800ms load times across the board. And group be had 800ms load times for 98% of their page loads and 20,000ms on the remaining 2%. So working with averages can be largely misleading. I can't see the details of the 1 study that claims 100ms increments, but I'm very skeptical.
      - wongarsu 7y ago
        Yes, the 1ms will likely be statistically insignificant. But so are $2000 extra income out of your $10m. If anything, you have a better hope measuring the 1ms than the $2000. I agree with your larger point that average latency is not a good measure. Even for the perception of a single user consistency is more important than a good average. For large groups the average is even less useful. But intuitively I see lots of places where 100ms makes a world of difference. Just like there's a big, very perceptible difference between a 200ms animation and a 300ms animation, a time to full render of 200ms can change your experience compared to 300ms. The slower the page load, the more deliberate your movement. The closer you get to 16ms (1 frame) for a page load the smaller the investment for clicking a link, and the higher the willingness to experiment and explore. Some of that inevitably leads to conversions and sales.
- dna_polymerase 7y ago
  > 7% increase represents an increase in the number of students/parents purchasing tickets to our events. (It's not like they're spending more individually, but more people have been willing to give us money.) We got that part, but correlation doesn't always imply causation. Maybe that change was seasonal or just plain old growth not accounted for in your 7%?

foobarbazetc 7y ago

CDNPerf is an okay way to get an overview, but not a great way to actually compare CDNs. That said, it’s not 10x or whatever. Maybe like 5-10% depending on location.

bufferoverflow 7y ago
Do you know of a better way than CDNPerf? They do 300 million requests per day from random users from around the world.
- briffle 7y ago
  Cedexis is also a good tool to measure, while most of it is a paid product, they have a few reports you can see. they also list cloud providers, and have other reports such as throughput, etc. https://www.cedexis.com/get-the-data/country-report/ https://www.cedexis.com/get-the-data/country-report/
- acdha 7y ago
  Is there any documentation on how they measure it? If they're running tests from data centers / peering points or testing things which are highly-likely to be cached the results are going to be harder to generalize to normal usage.
tylermenezes 7y ago
There are a number of features Fastly has which others (excluding eg Akmai) don't which make head-to-head comparisons more difficult. I have no doubt that Cloudflare is very fast in perfect conditions, but there were a much higher number of cache misses because there's less flexible configuration which did literally make the difference between 20ms vs 200ms in production.
- anonymoushn 7y ago
  Can you elaborate on this? Enterprise customers are generally able to get their SE to set whatever cache key they like, and if that is insufficient they can use the worker cache API: https://developers.cloudflare.com/workers/reference/cache-api/ https://developers.cloudflare.com/workers/reference/cache-ap... As I understand it a charity may get the former thing for free through https://www.cloudflare.com/galileo/ https://www.cloudflare.com/galileo/ , but the latter probably still has usage-based billing. Disclosure: I work for Cloudflare.