3 ms·
How big are y'all that you need InfluxDB? For one database that receives 100M 600 byte JSON records/day, a single node AWS PostgreSQL RDS instance is handling
by asah 3y ago
How big are y'all that you need InfluxDB?
For one database that receives 100M 600 byte JSON records/day, a single node AWS PostgreSQL RDS instance is handling it effortlessly, and the DBA work is very part-time. We keep year+ of summaries and 48 hours of detail, unloading the rest to S3 as parquet files, queryable by Athena if we need. AFAICT, we're spending <$4K/mon all in, including backups.
p.s. a buddy at a top-3 TV streaming service is also doing this for logging all viewing activity, but with Aurora.
- asah 3y agoupdate: it's actually more like $1500/mon and that includes an RDS proxy, failover instance and online backups with one-click restore.
- lbruder 3y agowe're storing 120M records a day in influx 1.8, offloading cold data to S3, all on a single m5.xlarge instance that runs other backend services on the side. Less than 400 bucks per month overall aws cost, with lots of other VMs in there. We could use RDS too, but why change it to something more expensive if it already works...
- asah 3y agothanks! killer example. how are you dealing with failover? how are you dealing with interactive reporting? how large are your records? is the s3 data in cold storage? how much is stored in S3 right now? is it warm-enough to query via (e.g.) athena?