3 ms·
For this amount of data, I would use good old Postgres, partition the data by ingestion time, then just detach the old partitions when you need to archive it.
by dikei 7y ago
For this amount of data, I would use good old Postgres, partition the data by ingestion time, then just detach the old partitions when you need to archive it.
For joining data from multiple database, if the data is large, I would use something like Presto(https://prestosql.io/ https://prestosql.io/) to join and process the data. But that's partly because we have already had Presto clusters running.