3 ms·

Well, it was me who did that test and frankly I only included it to give some context for people who've never heard of Parquet. If you want evidenced benchmark

Well, it was me who did that test and frankly I only included it to give some context for people who've never heard of Parquet. If you want evidenced benchmarks I'm sure you can google for those yourself or indeed just run your own - the benefit differs as the data does (financial data obviously is mainly numerical). I'm not, in this post, actually talking about Parquet being faster than csv but instead about how it makes data interchange easier (mostly that is about having an embedded schema).

crabbone 4y ago
So why are you posting this? If I have to do the work of benchmarking? More importantly, why should I trust you, if you didn't bother making your article trustworthy? Working many years in storage, I can engineer a benchmark that will show that CSV is faster than Parquet. So what? > having an embedded schema CSV also has embedded schema, that's what the header row is. Your arguments are just poor / show insufficient research / insufficient ability to interpret results. If you wanted to make that argument stronger / believable, you should've at least claimed that Parquet has better schema than CSV. But this is also a problematic claim, because your idea of "better" comes from a perspective of making something more universal. But, in particular cases, I could, for example, come up with a CSV header that would be able to communicate more type information than what's available in Parquet, and subsequently, beat Parquet on that metric. Anyways. My bigger point: maybe you are right, but you don't make a compelling case.