3 ms·
In contrast to the other comments, this is a valid concern regarding whether or not they asked first. I assume legal cleared this blog post, which would likely
by judge2020 5y ago
In contrast to the other comments, this is a valid concern regarding whether or not they asked first. I assume legal cleared this blog post, which would likely infer that legal either (A) got permission to publish benchmarks, or (B) doesn't think that part of the TOS is enforceable/doesn't expect CF to actually do anything about it.
The TOS clause, for reference: https://www.cloudflare.com/terms/#:~:text=(f)%20perform%20or%20publish%20any%20benchmark%20tests%20or%20analyses%20relating%20to%20the%20Services%20without%20Cloudflare%E2%80%99s%20written%20consent%3B https://www.cloudflare.com/terms/#:~:text=(f)%20perform%20or... (ctrl-f benchmark )
- InGoodFaith 5y agoThey stated in the blogpost that they only performed the benchmarks against their own services > So instead, we ran the same tests (echoing headers and measuring TTFB via Catchpoint) against our own platform
- kailanb 5y agoTo clarify, Fastly did not perform any benchmarks on Cloudflare's network. We ran our own benchmark, comparable to the one that Cloudflare ran on our platform, and compared those results to the ones that Cloudflare published as part of the original blog post. The Cloudflare terms prohibit benchmarking without their explicit permission.
- dsizzle 5y ago> To clarify, Fastly did not perform any benchmarks on Cloudflare's network. We ran our own benchmark… Am I the only having cognitive dissonance here? I was confused how they complained about not being able to perform benchmarks but then… ran a benchmark? There must be some distinction here that I’m missing
- kailanb 5y agoSorry, I was trying to clarify in my comment but failed to put it through clearly. When I say "We ran our own benchmark" I mean Fastly ran a benchmark on a Fastly Compute@Edge service.
- kyralis 5y agoFastly is unable to reproduce the benchmark against Cloudflare. They reproduced a modified (and theoretically improved) benchmark against their own service, but then had to compare it to the original values provided by Cloudflare.
- IanAWP 5y agoCloudFlare ran a benchmark on both services and published their results. The contention is that the benchmark did not represent fastly performance fairly, so fastly reran the benchmark in what they consider to be a fair way, on their service only. They did not retest CloudFlare in their benchmark. The results in the post compare CloudFlare's results for CloudFlare and fastly's results for fastly
- dsizzle 5y agoYeah, ok that must be the distinction. However, I see that the TOS says you may not "perform or publish" benchmarks. Seems this still violates the "or publish" part, even if Fastly avoided doing the performing. What would be the point of forbidding publishing if it is already forbidden to perform said benchmarks in the first place? The TOS seems to also forbid publishing on the benchmark even when someone else performs the Fastly part of the benchmark. Edit: It must be the TOS doesn't apply if you don't use the service.
- LamaOfRuin 5y agoFortunately, no one needed to agree to the CloudFlare ToS to read their blog post.
- dsizzle 5y agoHaha. Actually, now that I think about it, I guess the TOS doesn't apply if you don't use the service!
- ethbr0 5y agoNotably, the self-service and enterprise TOS differ a bit here. From "Cloudflare Self-Serve Subscription Agreement" https://www.cloudflare.com/terms/ https://www.cloudflare.com/terms/ , "2.2 Restrictions. Unless otherwise expressly permitted in writing by Cloudflare, you will not and you have no right to: [...] (f) perform or publish any benchmark tests or analyses relating to the Services without Cloudflare’s written consent;" From "ENTERPRISE SUBSCRIPTION TERMS OF SERVICE" https://www.cloudflare.com/enterpriseterms/ https://www.cloudflare.com/enterpriseterms/ , "2.3. Restrictions and Acceptable Use. Customer must not: [...] (i) perform or publish any performance or benchmark tests or analyses relating to the Service, other than solely for Customer’s internal use;" Generally, it sounds like the kind of thing that would be obvious to a lawyer (we want to prohibit random people badmouth our product, right?) but stupid to engineers. The best outcome would be if Fastly and Cloudflare both dropped any language limiting benchmarking and sharing of results. If they want to protect their brands, the most common sense requirement would be "We require you to tell us, within X hours after publishing, of benchmarks you're publishing and send us a copy of the article in which you published them (or link if publicly accessible)."