3 ms·
Show HN: The most fair and transparent database benchmarks
Hi HN
https://db-benchmarks.com is a platform and a framework for making the most fair, transparent and open source database and search engines benchmarks. No more benchmarketing, because:
* anyone can easily reproduce all the results on their hardware
* the results you can see on the website are under version control on GitHub and anyone can run the same website locally in just a minute
* every byte of the project is 100% open source including test results and detailed test log
* the results are highly accurate with low coefficient of variation
The first release has Clickhouse, Elasticsearch, Manticore Search and MySQL covered on small / medium and big data realistic data collections. In plans: Percona Server, MongoDB, CocroachDB, FerretDB
Links:
https://db-benchmarks.com
https://github.com/db-benchmarks/db-benchmarks
https://github.com/db-benchmarks/ui
Disclaimer: I'm a member of Manticore Search team and the framework was initially made to compare Manticoresearch with Elasticsearch, but I did my best to not give any unfair advantage to Manticore since it's important for the team to understand the real situation. However, if you see something is missing or wrong feel free to make a pull request or an issue on Github. Or make your own fork with even more fair results!
Ideas/thoughts/comments?
- zX41ZdbW 4y agoThe calculation method and the overall page design is very similar to ClickHouse benchmarks: https://clickhouse.com/benchmark/dbms/ https://clickhouse.com/benchmark/dbms/ https://presentations.clickhouse.com/original_website/benchmark.html https://presentations.clickhouse.com/original_website/benchm... (2013) The table and diagram layout, the checkboxes and controls, the "geometric mean of ratios", etc. But it does not have proper attribution and it looks unfair. You can reference directly to me (Alexey Milovidov) if needed.
- snikolaev 4y agoNice to hear from you, Alexey. Thank you for pointing out this glaring omission. It's fixed now - https://github.com/db-benchmarks/ui/blob/main/readme.md#notes https://github.com/db-benchmarks/ui/blob/main/readme.md#note... , https://github.com/db-benchmarks/db-benchmarks/blob/main/README.md#notes https://github.com/db-benchmarks/db-benchmarks/blob/main/REA... ,