3 ms·
you can comfortably fit 8TB of data on a single box running postgres, more dependent on your hardware :)
by fortytw2 9y ago
you can comfortably fit 8TB of data on a single box running postgres, more dependent on your hardware :)
- kstrauser 9y agoThat's about 1/250th of our working set size. :) Of course, that's in a Spark cluster running massively parallel queries and not on a single (however large) node.
- ibotty 9y agoYou did not ask that question. I assume you knew before that PostgreSQL is not the tool for your workload.
- kstrauser 9y agoI didn’t ask a question. I love PostgreSQL for small relational stuff (where “small” is up to a few TB) and use it any time it’s appropriate. I wouldn’t classify it as a Big Data store, though. That’s no criticism at all. I wouldn’t use Spark for a transactional DB because it’s not the best tool for the job. Same for PostgreSQL when the dataset grows far beyond what a single instance can reasonably process.
- riku_iki 9y agoCurious, how long it will take to create index on single node for a such large table?