12 ms·
Interesting discussion evolving! Disclaimer: I am co-author of that paper as well. That said: you may call it "suspicious" if the authors come to that conclusi
by pebau 6y ago
Interesting discussion evolving! Disclaimer: I am co-author of that paper as well.
That said: you may call it "suspicious" if the authors come to that conclusion, but on the same grounds it is likewise suspicious if the writer of a tool that has not excelled doubts the results :)
Let us rather look at facts - the benchmark is published and open, and actually similar figures have been reported by other, completely independent benchmarks. The paper has undergone tough scrutiny by 5 independent experts in the field before publication.
Doubting about the value of databases reminds me of the old times of COBOL vs SQL: "we don't need SQL data management". Incidentally, IT world since then has embraced databases of all kinds...and exactly arrays should be the big exception? That does not make sense. Tools will need to accept that there are other tools as well, and ideally discussion is merit-based.
"everyone who works with big array datasets already knows Python"...well, if the only tool you know is a hammer then...ya know. There are so many more worlds than just your comfort zone, python! Just think of R, for example.
PS: I am not questioning xarray - in projects we have used xarray as a frontend to rasdaman, and this combination works like a charm: python wrapper around scalability and federation, connected through the open OGC standards.
- pebau 6y agoanother PS: the article is not only about benchmarking, but also about a detailed, deep functionality comparison of 19 array tools.
- shoyer 6y agoHi Peter, good to run into you again :) I agree, formulating open benchmarks is great work, and there's nothing suspicious about performing well on benchmarks you write. It's just worth keeping in mind: https://matthewrocklin.com/blog/work/2017/03/09/biased-benchmarks https://matthewrocklin.com/blog/work/2017/03/09/biased-bench... My initial remarks were a little careless and overly provocative! I don't doubt that there are cases where a true "array database" provides value. I do think the use-cases are less clear for arrays than they are for tabular data, because the users of arrays tend to be more sophisticated.
- shoyer 6y agoBy the way, is there another reference with a full description of how the benchmarks are setup? I'd be curious how Xarray (end user API) + Dask (distributed compute) + Zarr (distributed storage) compares. Xarray definitely takes a different philosophical approach based on its roots in the Python data science ecosystem, compared to "all in one" solutions like a full array databases.