In memory representation also tends to have some data structure overhead so 5GB compressed -> 50GB uncompressed -> 100-250GB as a data structure (seems 2-5x is

In memory representation also tends to have some data structure overhead so 5GB compressed -> 50GB uncompressed -> 100-250GB as a data structure (seems 2-5x is pretty normal). It starts out seeming pretty innocuous but quickly explodes. In addition, some things like Drill can do some automatic indexing/metadata recording which can reduce the amount of data it needs to access