4 ms·
> But somehow when talking about Athena I always end up thinking about using big CSVs on S3 as data source, so what I really meant was Athena+S3 I guess. CSV i
by gopalv 9y ago
> But somehow when talking about Athena I always end up thinking about using big CSVs on S3 as data source, so what I really meant was Athena+S3 I guess.
CSV is a very unwieldy storage format for Athena + S3, since Athena charges you by data scan sizes.
ORC is pretty much optimized for S3-like use-cases (of being read over HTTP), so you'll find that a columnar structure like that would be a much nicer way to store and query often.
There's a pretty good run-down of the options and formats here - http://tech.marksblogg.com/billion-nyc-taxi-rides-aws-athena.html http://tech.marksblogg.com/billion-nyc-taxi-rides-aws-athena...