2 ms·
> Cloudflare not allowing benchmarks in their TOS is very sketchy, that puts them in the same tier as Oracle. There's a non-sketchy way for a company to do tha
by tzs 5y ago
> Cloudflare not allowing benchmarks in their TOS is very sketchy, that puts them in the same tier as Oracle.
There's a non-sketchy way for a company to do that kind of thing which avoids putting them in Oracle territory. Simply require that anyone publishing benchmarks (1) publish complete configuration information sufficient to allow the company and other interested third parties to reproduce the benchmarks, and (2) allows benchmarking of their products that they are benchmarking against yours under similar terms.
It is nicer of course to not have any benchmarking restrictions, but if you do that you are putting yourself at a disadvantage to companies like Oracle. They can benchmark against you put you can't do the same against them.
Once one important player in a market goes down the Oracle route others tend to follow. But if they aren't asses they will follow with the kind of restriction with reciprocity I outlined above instead of an Oracle-like restriction.
- throwaway2037 5y agoI like your ideas, but they seem difficult to enforce. It assumes good faith on all sides. One of the biggest complaints about AI/ML research results: It is frequently hard/impossible to replicate the results. One idea: The edge competitors can create a public (SourceHut?) project that runs various daily tests against themselves. This would similar to JSON library benchmarks. [1] Then allow each competitors to continuously tweak there settings to accomplish the task in the shortest amount of time. Also: It would be nice to see a cost analysis. For years, IBM's DB2 was insanely fast if you could afford to pay outrageous hardware, software license, and consulting costs. I'm not in the edge business, but I guess there are some operators where you can just pay a lot more and get better performance -- if you really need it. [1] https://github.com/miloyip/nativejson-benchmark https://github.com/miloyip/nativejson-benchmark
- mhitza 5y ago> One of the biggest complaints about *CS* research results: FTFY.