5 ms·
Athena is great for if you have _a lot_ of data. It isn't great for real time queries. I use a SQLite database stored in S3 + SQLite's FTS5 for a really cheap
by psanford 5y ago
Athena is great for if you have _a lot_ of data. It isn't great for real time queries.
I use a SQLite database stored in S3 + SQLite's FTS5 for a really cheap full text search implementation.
- hk1337 5y agoYeah, Athena is great for doing queries across multiple S3 objects. I have also used sqlite recently too, usually either with pandas reading from S3 then dumping it to a sqlite database table. I don't really like pandas but it's useful for creating the table structure for me.
- gsempe 5y agoI'm really interested to read articles or how to around this solution. Do you have anything public that could help me?
- psanford 5y agoFor accessing the database I use sqlite3vfshttp[0] with an s3 presigned url for authentication. [0]: https://github.com/psanford/sqlite3vfshttp https://github.com/psanford/sqlite3vfshttp