3 ms·
Try ClickHouse. You can set it up on a laptop, in a VM, or get a trial on one of the cloud services. You didn't say which RDS database you are using but ClickHo
by hodgesrm 5y ago
Try ClickHouse. You can set it up on a laptop, in a VM, or get a trial on one of the cloud services. You didn't say which RDS database you are using but ClickHouse can read data directly from MySQL and PostgreSQL. Pull it into tables on ClickHouse and try out your analytic queries to see if it seems reasonable.
100s of millions of rows is relatively small for ClickHouse, so you don't need to worry about advanced optimizations like compression codecs or materialized views. Standard datatypes and the default LZ4 compression should be just fine. Y
Here's quickstart video that will teach you most of what you need to get started: https://www.youtube.com/watch?v=phTu24qCIw0 https://www.youtube.com/watch?v=phTu24qCIw0
I work on ClickHouse at Altinity and created the education video. I'm kind of biased. ClickHouse is not the only OSS data warehouse. There is also Druid and Pinot. I think, however, for this particular case where you are pulling from RDS that ClickHouse will turn out to be a good solution.
Good luck whatever approach you choose!