4 ms·
Time-series data handling/storage seems a mostly solved problem in the mining, oil, and manufacturing industries. Deployed in the field since the 80's, https:/
by marshf 8y ago
Time-series data handling/storage seems a mostly solved problem in the mining, oil, and manufacturing industries. Deployed in the field since the 80's,
https://www.osisoft.com/about-osisoft/#more-about-pi-system https://www.osisoft.com/about-osisoft/#more-about-pi-system
Discourse: Industry user now OSIsoft employee
- bigger_cheese 8y agoYes I can confirm data historians like PI are used heavily in my industry (manufacturing).
- cheriot 8y agoA solved problem at what scale? A kid with pencil and paper can solve the problem when it's small enough.
- seanhunter 8y agoI don't have personal knowledge, but my understanding is that timeseries data in the oil industry is insane scale. Think 10s or 100s of wells in a field, sensors at regular distances in every well each taking multiple readings (temperature, flow etc) on a sub-second cadence. And a big oil company will of course have many fields. It's in the same league as timeseries data in financial markets, which I have more direct experience of. Financial timeseries data probably has more metadata (trade/order flags etc) but in terms of raw data it's similar.
- cheriot 8y agoWhat the article is talking about is an order of magnitude larger. Think millions of data points per minute coming out of a medium sized data center. It gets harder when you look at the cardinality of the data (explained better by the article), which corresponds to the sized of the indexes.