4 ms·

I think the scada vendors have already solved the same set of problems that timescale solves. Edit: pretty sure the scada vendors partition the data in to tabl

I think the scada vendors have already solved the same set of problems that timescale solves. Edit: pretty sure the scada vendors partition the data in to tables so each table is a day or week of data, they do rollups on the data so the detail is available if you do a query with a 1 minute timespan but the data is already aggregated in to representative values for a 1 minute interval if the query timespan is 1 day. Sounds like timescales clustering story might be better, but with OSI or wonderware your clustering story is ms SQL or oracle.

wenc 9y ago
Except that most process historians store data in a non-relational way which makes it difficult to integrate with other types of data. Process historians, like most NoSQL time-series offerings, are traditionally optimized for fast data retrieval (for HMIs, real-time monitoring dashboards, etc.) but are really weak when it comes to analytics. There are new products like Seeq (a kind of a Tableau for process historians) that attempt to address this, but somehow miss the fact that for analytics, you really need your data to exist in a dataframe-like shape. Today, if you want to do any kind of non-trivial analytics, you typically have to first ETL data from the historian to a relational database via REST or OLE DB API. TimescaleDB on the other hand already stores data in a native relational format and lets you run full SQL on it.
wenc 9y ago
> pretty sure the scada vendors partition the data in to tables so each table is a day or week of data, they do rollups on the data so the detail is available if you do a query with a 1 minute timespan but the data is already aggregated in to representative values for a 1 minute interval if the query timespan is 1 day. I have not seen this, but if this is the case, and if these aggregates are stored in a proper relational database, that solves a big chunk of the problems people have with doing analytics on historian data. As far as I know GE Proficy does not do this. Which is a pain.

wenc 9y ago

Except that most process historians store data in a non-relational way which makes it difficult to integrate with other types of data. Process historians, like most NoSQL time-series offerings, are traditionally optimized for fast data retrieval (for HMIs, real-time monitoring dashboards, etc.) but are really weak when it comes to analytics. There are new products like Seeq (a kind of a Tableau for process historians) that attempt to address this, but somehow miss the fact that for analytics, you really need your data to exist in a dataframe-like shape. Today, if you want to do any kind of non-trivial analytics, you typically have to first ETL data from the historian to a relational database via REST or OLE DB API. TimescaleDB on the other hand already stores data in a native relational format and lets you run full SQL on it.

wenc 9y ago

> pretty sure the scada vendors partition the data in to tables so each table is a day or week of data, they do rollups on the data so the detail is available if you do a query with a 1 minute timespan but the data is already aggregated in to representative values for a 1 minute interval if the query timespan is 1 day. I have not seen this, but if this is the case, and if these aggregates are stored in a proper relational database, that solves a big chunk of the problems people have with doing analytics on historian data. As far as I know GE Proficy does not do this. Which is a pain.