3 ms·
Here is the blog describing the Capacitor storage engine (successor to ColumnIO, which was the inspiration for Parquet): https://cloud.google.com/blog/big-data
by vgt 10y ago
Here is the blog describing the Capacitor storage engine (successor to ColumnIO, which was the inspiration for Parquet):
https://cloud.google.com/blog/big-data/2016/04/inside-capacitor-bigquerys-next-generation-columnar-storage-format https://cloud.google.com/blog/big-data/2016/04/inside-capaci...
Here's the blog post describing Google's in-memory execution (some of what the new Dremel version is doing, including pipelined execution, dynamic in-memory shuffling, dynamic partition sizing, etc):
https://cloud.google.com/blog/big-data/2016/08/in-memory-query-execution-in-google-bigquery https://cloud.google.com/blog/big-data/2016/08/in-memory-que...
These are some of the more significant improvements since the Dremel paper:
http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36632.pdf http://static.googleusercontent.com/media/research.google.co...