3 ms·
After 1.0 is released our two highest priorities are to handle high cardinality data sets (on disk and in memory inverted index) and continuous queries that are
by pauldix 10y ago
After 1.0 is released our two highest priorities are to handle high cardinality data sets (on disk and in memory inverted index) and continuous queries that are more automatic (downsample everything and give users the option for query results that scale automatically like Graphite)
- mattbillenstein 10y agoRight on -- for the record, this data is ending up in mongodb at the moment, so I need to write logic for binning by time and so forth myself. Our use case is maybe odd in this space -- sampling a handful of values per item a couple times a day, but for millions of items. I compared a dataset to postgres and the new mongo storage engine is very good -- dunno if wiredTiger is something you guys are looking at, but it's very compact and fast to query with the proper indexes. Dumped some stats into a gist as well: https://gist.github.com/mattbillenstein/89969980025414e2bca8325cf503342f https://gist.github.com/mattbillenstein/89969980025414e2bca8...
- oky 10y agoi had a similar issue with mongo and time series data. for special purpose event data like that gist, i think you should consider using a column store.
- mattbillenstein 10y agoI may eventually -- we're using mongo for the rest of the metadata for now, so this is good enough to not need something else at the moment.