For that volume, I would dump the data to disk somewhere (even if "somewhere" is S3). Load it up to extract the actually relevant data, and index that into what

For that volume, I would dump the data to disk somewhere (even if "somewhere" is S3). Load it up to extract the actually relevant data, and index that into whatever data store is most efficient for the types of queries you'll be doing. Relational is most effective for aggregates; MongoDB is good for stable document storage; Elasticsearch is less stable but good for fast search, especially full-text, stemming, and "fuzzy" or weighted searches. It may also be sensible to index your data into multiple databases so you can query it efficiently in different ways for different purposes.