4 ms·

I have used TimescaleDB for several purposes. As it is built on top of Postgres, all the existing tools, libraries and processes work out of the box. This is a

I have used TimescaleDB for several purposes. As it is built on top of Postgres, all the existing tools, libraries and processes work out of the box. This is a huge advantage if you are operating Postgres anyway: Your existing backup tools will work, as does your user Managment. Scaling out is a more complex story: Read Scaleout works seamless using read replicas (another existing Postgres mechanism). Replicas can also help you for high availability, Write scaleout is difficult with TimescaleDB (same for Postgres): Sharding can be an option, or buying bigger machines.

imglorp 7y ago
Will it work on top of RDS?
- mfrye0 7y ago
  No. https://forums.aws.amazon.com/thread.jspa?threadID=261623 https://forums.aws.amazon.com/thread.jspa?threadID=261623 The theory is they are not supporting it in favor of promoting their new time series DB.
  - akulkarni 7y ago
    (Co-founder Timescale) This is good feedback. Do you need something on RDS specifically or just managed on AWS?
    - mfrye0 7y ago
      Being a small team, we try to have managed hosting as much as possible. One less thing to think/worry about. Then our stack is primarily in AWS. So when it comes to DBs, I'd prefer to keep it all within our VPC for security / latency reasons - for sensitive data at least.
      - akulkarni 7y ago
        Great info. Thanks for sharing.
shin_lao 7y ago
From what you're writing it seems TimescaleDB doesn't really solve large timeseries problems. So it looks like more a good transitory solution than a definitive solution. Is that correct?
- thisone 7y ago
  In my opinion you're correct, it doesn't solve the problem of querying large sets of timeseries data over a long time span. It doesn't pre-aggregate your data so if you're trying to get aggregates for a large data set, it's not fast.
  - akulkarni 7y ago
    (Co-founder TimescaleDB) Pre-aggregation via continuous aggregations is in our next release, which I believe is slated for Monday. We also have more in the works for large datasets: eg scaling out storage across multiple machines as well as data tiering for lower storage costs. Feel free to email me if you'd like more info on either: ajay (at) timescale (dot com)
  - djk447 7y ago
    Hi! Engineer at Timescale who’s just been working on this feature, it’s coming in our next release and going to continue to be developed over the next few releases! Would love to have you try it out and get your feedback :)
    - scrollaway 7y ago
      God I love you folks. I can't wait for timescale to be available on rds, the more I read about it the more I love it and its authors.
  - shin_lao 7y ago
    Thanks for the feedback. How is TimescaleDB doing with compression by the way? Meaning if you compare your raw data and the disk usage, what's the ratio?
akulkarni 7y ago
(Co-founder TimescaleDB) Write scale-out is something we have been working on for nearly a year now. Here are some preliminary benchmarks we shared at Postgres Conf earlier this year (benchmarks have continued to improve since then): https://twitter.com/acoustik/status/1108813512321708032 https://twitter.com/acoustik/status/1108813512321708032 We'll have more to share regarding write scale-out later this month :)
NewsAware 7y ago
We ingest a lot of time series (IoT) data and use Postgres for other data so Timescale works quite well for us. One thing Timescale treats as second class citizen though is updates to existing data points which are 1-2 areas of magnitude slower than inserts. Granted this is also the case for all other TSDB solutions out there which are for obvious reasons optimized for inserts and reads aggregated along the time dimension. Still would be amazing if you could add to the already existing differentiation of allowing fast updates for cases like ours where we dont store events relating to a singular point in time but rather time-spans so new incoming data points might be "merged" into existing time-spans.
- eternalban 7y ago
  Same domain. I've come to the conclusion that these systems are dualistic in nature and the architecture needs to reflect that. I think the more general expression of this would be a time-series -> document-store pipeline. The concern then, if the time-series processing step requires domain data, is serving up fresh views of the evolving state in toto. That epsilon of temporal lag of 'current state' in the premier stage is a product issue. For some use cases it can be a very tight constraint, but I think quite a lot of others will be perfectly fine with snapshot isolation with generous requirement on freshness.

imglorp 7y ago

Will it work on top of RDS?

mfrye0 7y ago
No. https://forums.aws.amazon.com/thread.jspa?threadID=261623 https://forums.aws.amazon.com/thread.jspa?threadID=261623 The theory is they are not supporting it in favor of promoting their new time series DB.
- akulkarni 7y ago
  (Co-founder Timescale) This is good feedback. Do you need something on RDS specifically or just managed on AWS?
  - mfrye0 7y ago
    Being a small team, we try to have managed hosting as much as possible. One less thing to think/worry about. Then our stack is primarily in AWS. So when it comes to DBs, I'd prefer to keep it all within our VPC for security / latency reasons - for sensitive data at least.
    - akulkarni 7y ago
      Great info. Thanks for sharing.

shin_lao 7y ago

From what you're writing it seems TimescaleDB doesn't really solve large timeseries problems. So it looks like more a good transitory solution than a definitive solution. Is that correct?

thisone 7y ago
In my opinion you're correct, it doesn't solve the problem of querying large sets of timeseries data over a long time span. It doesn't pre-aggregate your data so if you're trying to get aggregates for a large data set, it's not fast.
- akulkarni 7y ago
  (Co-founder TimescaleDB) Pre-aggregation via continuous aggregations is in our next release, which I believe is slated for Monday. We also have more in the works for large datasets: eg scaling out storage across multiple machines as well as data tiering for lower storage costs. Feel free to email me if you'd like more info on either: ajay (at) timescale (dot com)
- djk447 7y ago
  Hi! Engineer at Timescale who’s just been working on this feature, it’s coming in our next release and going to continue to be developed over the next few releases! Would love to have you try it out and get your feedback :)
  - scrollaway 7y ago
    God I love you folks. I can't wait for timescale to be available on rds, the more I read about it the more I love it and its authors.
- shin_lao 7y ago
  Thanks for the feedback. How is TimescaleDB doing with compression by the way? Meaning if you compare your raw data and the disk usage, what's the ratio?

akulkarni 7y ago

(Co-founder TimescaleDB) Write scale-out is something we have been working on for nearly a year now. Here are some preliminary benchmarks we shared at Postgres Conf earlier this year (benchmarks have continued to improve since then): https://twitter.com/acoustik/status/1108813512321708032 https://twitter.com/acoustik/status/1108813512321708032 We'll have more to share regarding write scale-out later this month :)

NewsAware 7y ago

We ingest a lot of time series (IoT) data and use Postgres for other data so Timescale works quite well for us. One thing Timescale treats as second class citizen though is updates to existing data points which are 1-2 areas of magnitude slower than inserts. Granted this is also the case for all other TSDB solutions out there which are for obvious reasons optimized for inserts and reads aggregated along the time dimension. Still would be amazing if you could add to the already existing differentiation of allowing fast updates for cases like ours where we dont store events relating to a singular point in time but rather time-spans so new incoming data points might be "merged" into existing time-spans.

eternalban 7y ago
Same domain. I've come to the conclusion that these systems are dualistic in nature and the architecture needs to reflect that. I think the more general expression of this would be a time-series -> document-store pipeline. The concern then, if the time-series processing step requires domain data, is serving up fresh views of the evolving state in toto. That epsilon of temporal lag of 'current state' in the premier stage is a product issue. For some use cases it can be a very tight constraint, but I think quite a lot of others will be perfectly fine with snapshot isolation with generous requirement on freshness.