3 ms·
I am using a time-series db where each metric measurement requires some static context information to be included, e.g. Event 1 happened in Device A at Node B,
by crudbug 10y ago
I am using a time-series db where each metric measurement requires some static context information to be included, e.g. Event 1 happened in Device A at Node B, Location C. These static entities (A, B, C) are perfect for graph vertices. But, when I am persisting the graph on disk - I store all the data statically, No links just columns - the static entities become dimensional data.
Is there a better way to this ?
Ideally, we should persist the Object Graph directly without any data conversion. I see a difference in - How we compute and How we store data.
- rodionos 10y agoNot specific to GraphEngine, but you could store all meta-data about the device separate from the time-series itself. The composite metric name (key?) would then include the measurement name and the entity name (device name in this case).
- crudbug 10y agoThat is one possible solution. But, for queries we will need some kind of JOIN at runtime. I do store the entity data separately in a transactional row store. For analytical queries, we use a column store. For this, I have to include the entity data (context) for each fired event. The whole Compute & Storage semantics are different. If only, we had a persistent data-aware compute & storage graph engine that supported transactional & analytical workloads.
- rodionos 10y agoStoring time-series in a columnar format (blobs o time-values) and entity data in row format sounds good to me. If you look at data historians (PI et al), that's how they lay out the data: metrics storage is separate from the asset store, or Asset Framework as it's called.