4 ms·
Okay, I kind of want to question the "just throw it in Postgres," attitude I've seen throughout this thread. I've dealt with similar sizes (maybe 750GB, number
by Boxxed 7y ago
Okay, I kind of want to question the "just throw it in Postgres," attitude I've seen throughout this thread. I've dealt with similar sizes (maybe 750GB, number of rows in the small billions) and as much as I adore Postgres it just wasn't a great solution.
-- Loading would take hours, even with dropping the indexes beforehand
-- Creating the indexes afterwards would take even longer
-- Query performance was very hard to reason about. Even simple queries required altering some spillage limits.
-- Analytical queries were basically out of the question, with something like "select sum(thing) from table" taking many minutes.
Even if I could live with the above as just an initial load, there would have been a lot of churn as new data arrived that probably would have been even worse.