4 ms·
Very BigQuery like system. Has anything been sacrificed for this type of scale-out? BQ and other Dremel based systems are weak at joins for star based schemas
by filereaper 10y ago
Very BigQuery like system.
Has anything been sacrificed for this type of scale-out?
BQ and other Dremel based systems are weak at joins for star based schemas, and frequently advise denormalization of data.
- scribu 10y ago> Very BigQuery like system. What makes you say that? The main innovation in BigQuery was the ability to store and query nested data. Amazon Redshift doesn't support querying nested data. It only has some convenience functions for loading flat data from nested JSON files hosted on S3. And what I assume Spectrum does is just perform that loading step behind the scenes.
- deleted 10y ago[deleted]
- puzzle 10y agoAt a first read, it sounded like you didn't need to run a cluster ("Spectrum scales to thousands of instances"), which is BigQuery's big advantage. Later, it says that you still need a cluster, but it's only used in the final processing stages. So it looks like it's 3/4 of the way to BQ.