5 ms·

What are some use cases for a time-series database?

mjdesa 7y ago
The big use cases are devops/monitoring data and IoT data. But it also applies to pretty much any use case where you want to answer questions about data in relation to time.
lolinder 7y ago
They're commonly used for collecting metrics, such as performance or traffic data.
ska 7y ago
Pretty much anything that is naturally generating time series data, and when your queries are going to be about things-over-time in relationship to each other. Think automotive monitoring (external or internal), algorithmic trading, retail monitoring, aviation, etc.
mcallica 7y ago
A good article for some background: https://dzone.com/articles/what-the-heck-is-time-series-data-and-why-do-i-nee https://dzone.com/articles/what-the-heck-is-time-series-data...
cevian 7y ago
[Timescaledb engineer here] We like to say that time-series data is any data that is insert-mostly with data associated with the most recent time period. That's a pretty broad definition, intentionally so. We see usage in telecoms, heavy industry, science, health, IoT, etc. It's really about recording the history of your data as it evolves, instead of just the current state.
- _-david-_ 7y ago
  How does it compare with using temporal tables in a relational database?
  - cevian 7y ago
    Well as far as I understand, temporal tables usually have either a valid time or system time. TimescaleDB is geared towards something you'd call measurement time, and most modifications are inserts to recent measurement time. In contrast, temporal tables are often still update-heavy and there is often no correlation with recent time, especially for valid time fields.
  - manigandham 7y ago
    Temporal table = state of the data in the table at a particular time. Very useful for auditing or seeing how data has changed between different periods. Timeseries = data with a primary key that includes time, potentially with other time properties. For example, metrics are commonly associated with a value at some time.
- srazzaque 7y ago
  I've been eyeing TimescaleDB at a distance for some time now. I'm curious if you have seen it used in finance as an alternative to KDB+ installations anywhere? Have you thought to release any benchmarks against KDB+?
  - mfreed 7y ago
    To my understanding, KDB+'s license explicitly forbids benchmarking: "1.3 Kdb+ On Demand Software Performance. End User shall not distribute or otherwise make available to any third party any report regarding the performance of the Kdb+ On Demand Software, Kdb+ On Demand Software benchmarks or any information from such a report unless End User receives the express, prior written consent of Kx to disseminate such report or information."
    - srazzaque 7y ago
      That's a shame. I'm not sure what the rationale is for such clauses - especially where performance of your tech is known to be pretty good (as is the case for KDB+). There's definitely a huge opportunity to displace KDB+ as the mainstay for timeseries in capital markets. It is a premium product, but it obviously comes with a cost - both for the product and its operators. Assuming all use cases can be catered for, if one needs an extra N machines if using TimescaleDB to cater for the same workloads, it might nullify any savings. If only there were a way to understand that without breaking their EULA...
    - beagle3 7y ago
      Does this also apply to a locally installed version? It didn't in the past. "On Demand" is IIRC the cloud kdb+, which is thus much less predictable and easy to misrepresent.
      - mfreed 7y ago
        Not my specialty, but website shows two versions available for download (at least for free non-commercial use): 64-bit "on-demand" or 32-bit. Both have a similar no benchmarking clause. https://kx.com/connect-with-us/download/ https://kx.com/connect-with-us/download/ 32-bit version: "(c) 32 Bit Kdb+ Software Evaluations. User shall not distribute or otherwise make available to any third party any report regarding the performance of the 32 Bit Kdb+ Software, 32 Bit Kdb+ Software benchmarks or any information from such a report unless User receives the express prior written consent of Kx to disseminate such report or information." But more broadly, feedback from the finance/capital markets suggest that the choice of kdb's proprietary Q query language vs. standard SQL is top-of-mind, expanding access and insights to time-series data from a small set of highly-specialized engineers to any of their developers / analysts / tools.
happyweasel 7y ago
the operational historian parts of SCADA systems

The big use cases are devops/monitoring data and IoT data. But it also applies to pretty much any use case where you want to answer questions about data in relation to time.

lolinder 7y ago

They're commonly used for collecting metrics, such as performance or traffic data.

ska 7y ago

Pretty much anything that is naturally generating time series data, and when your queries are going to be about things-over-time in relationship to each other. Think automotive monitoring (external or internal), algorithmic trading, retail monitoring, aviation, etc.

mcallica 7y ago

A good article for some background: https://dzone.com/articles/what-the-heck-is-time-series-data-and-why-do-i-nee https://dzone.com/articles/what-the-heck-is-time-series-data...

cevian 7y ago

[Timescaledb engineer here] We like to say that time-series data is any data that is insert-mostly with data associated with the most recent time period. That's a pretty broad definition, intentionally so. We see usage in telecoms, heavy industry, science, health, IoT, etc. It's really about recording the history of your data as it evolves, instead of just the current state.

_-david-_ 7y ago
How does it compare with using temporal tables in a relational database?
- cevian 7y ago
  Well as far as I understand, temporal tables usually have either a valid time or system time. TimescaleDB is geared towards something you'd call measurement time, and most modifications are inserts to recent measurement time. In contrast, temporal tables are often still update-heavy and there is often no correlation with recent time, especially for valid time fields.
- manigandham 7y ago
  Temporal table = state of the data in the table at a particular time. Very useful for auditing or seeing how data has changed between different periods. Timeseries = data with a primary key that includes time, potentially with other time properties. For example, metrics are commonly associated with a value at some time.
srazzaque 7y ago
I've been eyeing TimescaleDB at a distance for some time now. I'm curious if you have seen it used in finance as an alternative to KDB+ installations anywhere? Have you thought to release any benchmarks against KDB+?
- mfreed 7y ago
  To my understanding, KDB+'s license explicitly forbids benchmarking: "1.3 Kdb+ On Demand Software Performance. End User shall not distribute or otherwise make available to any third party any report regarding the performance of the Kdb+ On Demand Software, Kdb+ On Demand Software benchmarks or any information from such a report unless End User receives the express, prior written consent of Kx to disseminate such report or information."
  - srazzaque 7y ago
    That's a shame. I'm not sure what the rationale is for such clauses - especially where performance of your tech is known to be pretty good (as is the case for KDB+). There's definitely a huge opportunity to displace KDB+ as the mainstay for timeseries in capital markets. It is a premium product, but it obviously comes with a cost - both for the product and its operators. Assuming all use cases can be catered for, if one needs an extra N machines if using TimescaleDB to cater for the same workloads, it might nullify any savings. If only there were a way to understand that without breaking their EULA...
  - beagle3 7y ago
    Does this also apply to a locally installed version? It didn't in the past. "On Demand" is IIRC the cloud kdb+, which is thus much less predictable and easy to misrepresent.
    - mfreed 7y ago
      Not my specialty, but website shows two versions available for download (at least for free non-commercial use): 64-bit "on-demand" or 32-bit. Both have a similar no benchmarking clause. https://kx.com/connect-with-us/download/ https://kx.com/connect-with-us/download/ 32-bit version: "(c) 32 Bit Kdb+ Software Evaluations. User shall not distribute or otherwise make available to any third party any report regarding the performance of the 32 Bit Kdb+ Software, 32 Bit Kdb+ Software benchmarks or any information from such a report unless User receives the express prior written consent of Kx to disseminate such report or information." But more broadly, feedback from the finance/capital markets suggest that the choice of kdb's proprietary Q query language vs. standard SQL is top-of-mind, expanding access and insights to time-series data from a small set of highly-specialized engineers to any of their developers / analysts / tools.

happyweasel 7y ago

the operational historian parts of SCADA systems