3 ms·
Been searching for years to a good alternative to postgres for storing gobs of weather timeseries data. So far we have been running postgres system for many yea
by brootstrap 8y ago
Been searching for years to a good alternative to postgres for storing gobs of weather timeseries data. So far we have been running postgres system for many years in production and have hired multiple contractors to implement a 'real timeseries solution'. All of which have been utter shit and complete failures. The AWS services are expensive as all hell. With a little bit of imagination we created a unique schema for timeseries data that doesnt require terabytes of space, and processes billions of data points a day, and has blazing fast queries into said data.
- DelaneyM 8y agoI once handled gobs of web tracking data in Cassandra, using Hadoop over Cassandra to build queryable rollups in MySQL and Pig for on the fly analysis Neither tech works on its own, but together (substituting Cassandra columns for hdfs) was magic for the specific data configuration & use case.
- jws 8y agoI moved a decade's worth of weather time series data from well indexed Sqlite to InfluxDB and was nothing but pleased. It ended up taking an order of magnitude less storage and so much faster to query that I didn't even bother to benchmark it. You can probably write a simple query to your Postgres database to cough out the text file to load InfluxDB to see how it works for you. Then it comes down to how easy it is to replace your query and insert functions… So it's all easy except for the hard part.
- porker 8y agoDid you try https://www.timescale.com/ https://www.timescale.com/?