5 ms·
Exactly same case here - we're using Athena (packaged presto from AWS) and have a Postgres instance w/ identical data in a different data model. Presto is mindb
by DanFeldman 8y ago
Exactly same case here - we're using Athena (packaged presto from AWS) and have a Postgres instance w/ identical data in a different data model. Presto is mindblowingly good at the low latency queries and scales well to anything nontrivial. It seems opening files for reading is the biggest source of delay in presto! Postgres is not handling the scale well, but it was a nice experiment.
- Erwin 8y agoYour Athena data set are immutable S3 blobs of data however, while Postgres reads data that could be modified while it's reading them, concurrently. Postgres has to deal with a transaction modifying 100,000 rows in a table, needing that transaction to then see its changes but then aborting, leaving the data exactly as it was before. Athena/Presto don't have to worry about any transactions, table files bloated by changes that need vacuuming, etc. etc. So I don't think it's fair to claim PG does not scale, when you are comparing a transactional and analytical database.
- lukev 8y agoThat's rather the point though. The people saying "YAGNI, just use Postgres" aren't, a priori, any more correct than people saying "You definitely need Hadoop if you're gonna scale!" You need to analyze our own use case critically, understand the tradeoffs of different tools and choose pragmatically.