4 ms·
Your Athena data set are immutable S3 blobs of data however, while Postgres reads data that could be modified while it's reading them, concurrently. Postgres h
by Erwin 8y ago
Your Athena data set are immutable S3 blobs of data however, while Postgres reads data that could be modified while it's reading them, concurrently.
Postgres has to deal with a transaction modifying 100,000 rows in a table, needing that transaction to then see its changes but then aborting, leaving the data exactly as it was before. Athena/Presto don't have to worry about any transactions, table files bloated by changes that need vacuuming, etc. etc.
So I don't think it's fair to claim PG does not scale, when you are comparing a transactional and analytical database.
- lukev 8y agoThat's rather the point though. The people saying "YAGNI, just use Postgres" aren't, a priori, any more correct than people saying "You definitely need Hadoop if you're gonna scale!" You need to analyze our own use case critically, understand the tradeoffs of different tools and choose pragmatically.