4 ms·
The utility of grep depends on the amount of data and how it's stored. Grepping very large files of the size where you might consider parquet, that may take ma
by marginalia_nu 4y ago
The utility of grep depends on the amount of data and how it's stored. Grepping very large files of the size where you might consider parquet, that may take many hours. It's almost certainly faster to sit down and build a program that extracts data from a parquet.
On the other hand if grepping is fast, parquet doesn't really make sense in that scenario. You're better off with a structured format like JSON or XML if grep finishes in less than a few minutes.