This is why you'd use something like Impala; SQL is your cut, sort, uniq etc. That's why they don't exist; put the Parquet file into HDFS, turn on shortcut acce

This is why you'd use something like Impala; SQL is your cut, sort, uniq etc. That's why they don't exist; put the Parquet file into HDFS, turn on shortcut access, and use Impala to query it. I see throughput of about 250M records/sec on a 12-CPU box [1]. If your data doesn't fit in memory (FS cache), I/O will probably be your bottleneck. [1] Technically, 6 2-core guests on a single hypervisor; using guests to test deployment and scaling more easily.