5 ms·

Cloudflare not allowing benchmarks in their TOS is very sketchy, that puts them in the same tier as Oracle. Cloudflare have pulled enough shady stuff now that

Cloudflare not allowing benchmarks in their TOS is very sketchy, that puts them in the same tier as Oracle. Cloudflare have pulled enough shady stuff now that they've fallen out of my favor. Their generous free product bought them a lot of community goodwill but their real face has been showing the past few years.

pm90 5y ago
I would give them a bit more wiggle room before giving up. We’re currently living under a monopoly of AWS (GCP is the most serious “competitor”, still far far behind in market share) and I would love to see more competition in this space. That said: this is definitely not the kind of behavior I expect from them and I’m hoping there’s a better reason than them not willing to submit their products to an impartial evaluation.
- varsketiz 5y ago
  You are forgetting Azure, which is much bigger than GCP.
  - pm90 5y ago
    I’m not forgetting Azure, it’s a deliberate omission because it is not a Tier 1 cloud product. It might be better than OCI or IBM Cloud but it simply does not have the maturity of either AWS or GCP.
    - robertlagrant 5y ago
      Interesting. As someone who's only really used Azure (other than a little GCP) - what's missing?
- latch 5y ago
  AWS is a monopoly if you're living in a HN bubble. The vast majority of "hosting" (internet or otherwise) doesn't use AWS. Large telco's (Telefonica, AT&T, China Unicom, ...) have numerous data centers. Conglomerates (Tata, ...) have numerous data centers. You have wholesalers (Equinix, Digital Realty, ...). You have dedicated providers (OVH, Hetzner, PhoenixNap, ...). VPS' (DO, Linode, Vultr,...) and shared hosting (GoDaddy, ...) Then you have a lot of private data centers, e.g. I worked at a major bank that had its own datacenters (multiple around the world). Admittedly, these are super small compared to an AWS DC, but there's literally thousands of these around the world (I know for banking and government, but I assume other industries do this too).
- goodpoint 5y ago
  Wrong. Cloudflare introduce another type of monopoly on top of the existing cloud services. A very harmful one - they are trying to centralize the web. And kill Tor, while at it.
tzs 5y ago
> Cloudflare not allowing benchmarks in their TOS is very sketchy, that puts them in the same tier as Oracle. There's a non-sketchy way for a company to do that kind of thing which avoids putting them in Oracle territory. Simply require that anyone publishing benchmarks (1) publish complete configuration information sufficient to allow the company and other interested third parties to reproduce the benchmarks, and (2) allows benchmarking of their products that they are benchmarking against yours under similar terms. It is nicer of course to not have any benchmarking restrictions, but if you do that you are putting yourself at a disadvantage to companies like Oracle. They can benchmark against you put you can't do the same against them. Once one important player in a market goes down the Oracle route others tend to follow. But if they aren't asses they will follow with the kind of restriction with reciprocity I outlined above instead of an Oracle-like restriction.
- throwaway2037 5y ago
  I like your ideas, but they seem difficult to enforce. It assumes good faith on all sides. One of the biggest complaints about AI/ML research results: It is frequently hard/impossible to replicate the results. One idea: The edge competitors can create a public (SourceHut?) project that runs various daily tests against themselves. This would similar to JSON library benchmarks. [1] Then allow each competitors to continuously tweak there settings to accomplish the task in the shortest amount of time. Also: It would be nice to see a cost analysis. For years, IBM's DB2 was insanely fast if you could afford to pay outrageous hardware, software license, and consulting costs. I'm not in the edge business, but I guess there are some operators where you can just pay a lot more and get better performance -- if you really need it. [1] https://github.com/miloyip/nativejson-benchmark https://github.com/miloyip/nativejson-benchmark
  - mhitza 5y ago
    > One of the biggest complaints about *CS* research results: FTFY.

pm90 5y ago

I would give them a bit more wiggle room before giving up. We’re currently living under a monopoly of AWS (GCP is the most serious “competitor”, still far far behind in market share) and I would love to see more competition in this space. That said: this is definitely not the kind of behavior I expect from them and I’m hoping there’s a better reason than them not willing to submit their products to an impartial evaluation.

varsketiz 5y ago
You are forgetting Azure, which is much bigger than GCP.
- pm90 5y ago
  I’m not forgetting Azure, it’s a deliberate omission because it is not a Tier 1 cloud product. It might be better than OCI or IBM Cloud but it simply does not have the maturity of either AWS or GCP.
  - robertlagrant 5y ago
    Interesting. As someone who's only really used Azure (other than a little GCP) - what's missing?
latch 5y ago
AWS is a monopoly if you're living in a HN bubble. The vast majority of "hosting" (internet or otherwise) doesn't use AWS. Large telco's (Telefonica, AT&T, China Unicom, ...) have numerous data centers. Conglomerates (Tata, ...) have numerous data centers. You have wholesalers (Equinix, Digital Realty, ...). You have dedicated providers (OVH, Hetzner, PhoenixNap, ...). VPS' (DO, Linode, Vultr,...) and shared hosting (GoDaddy, ...) Then you have a lot of private data centers, e.g. I worked at a major bank that had its own datacenters (multiple around the world). Admittedly, these are super small compared to an AWS DC, but there's literally thousands of these around the world (I know for banking and government, but I assume other industries do this too).
goodpoint 5y ago
Wrong. Cloudflare introduce another type of monopoly on top of the existing cloud services. A very harmful one - they are trying to centralize the web. And kill Tor, while at it.

tzs 5y ago

> Cloudflare not allowing benchmarks in their TOS is very sketchy, that puts them in the same tier as Oracle. There's a non-sketchy way for a company to do that kind of thing which avoids putting them in Oracle territory. Simply require that anyone publishing benchmarks (1) publish complete configuration information sufficient to allow the company and other interested third parties to reproduce the benchmarks, and (2) allows benchmarking of their products that they are benchmarking against yours under similar terms. It is nicer of course to not have any benchmarking restrictions, but if you do that you are putting yourself at a disadvantage to companies like Oracle. They can benchmark against you put you can't do the same against them. Once one important player in a market goes down the Oracle route others tend to follow. But if they aren't asses they will follow with the kind of restriction with reciprocity I outlined above instead of an Oracle-like restriction.

throwaway2037 5y ago
I like your ideas, but they seem difficult to enforce. It assumes good faith on all sides. One of the biggest complaints about AI/ML research results: It is frequently hard/impossible to replicate the results. One idea: The edge competitors can create a public (SourceHut?) project that runs various daily tests against themselves. This would similar to JSON library benchmarks. [1] Then allow each competitors to continuously tweak there settings to accomplish the task in the shortest amount of time. Also: It would be nice to see a cost analysis. For years, IBM's DB2 was insanely fast if you could afford to pay outrageous hardware, software license, and consulting costs. I'm not in the edge business, but I guess there are some operators where you can just pay a lot more and get better performance -- if you really need it. [1] https://github.com/miloyip/nativejson-benchmark https://github.com/miloyip/nativejson-benchmark
- mhitza 5y ago
  > One of the biggest complaints about *CS* research results: FTFY.