3 ms·
The author had a terrible brain cramp in the sentence "Redshift uses columnar compression which allows it to bypass a field which is not used in the query." Th
by CurtMonash 13y ago
The author had a terrible brain cramp in the sentence "Redshift uses columnar compression which allows it to bypass a field which is not used in the query."
That totally confuses columnar compression with columnar I/O, an error I've been railing against for several years, e.g. in http://www.dbms2.com/2011/02/06/columnar-compression-database-storage/ http://www.dbms2.com/2011/02/06/columnar-compression-databas... (I.e., ever since Oracle tried to popularize the confusion.) But this is a particularly bad instance.
- keithgabryelski 13y agoI also don't see a schema that is tuned for redshift. There is no description of a sortkey or a distribution key (let alone which compression encoding was used) And at least for the first query you could use UNLOAD instead of select (depending on how you are managing the data coming out of redshift it might be a reasonable solution and doesn't force all your data through the leader node constrained by whatever client driver you are using to read the results). Instead of trying to select out of redshift this much data -- select the data into another table or (again) use unload.