14 ms·

Announcing Timescale Cloud

89 points by bsg75 7y ago

mcallica 7y ago
Just announced today, here's the blog post: https://blog.timescale.com/timescale-cloud-first-fully-managed-time-series-database-service-runs-on-aws-gcp-azure/ https://blog.timescale.com/timescale-cloud-first-fully-manag...
- dang 7y ago
  That gives more background info so we've switched to it from https://www.timescale.com/cloud https://www.timescale.com/cloud. Thanks!
mharroun 7y ago
It seems disingenuous to call it the first. AWS has it's own time series db. In terms of open source Apache druid had a managed cloud variant that imply.io runs.
- akulkarni 7y ago
  It’s the first multi-cloud. AWS obviously just AWS. Based on Imply Cloud website it looks like it too is just AWS. Timescale Cloud gives you freedom to choose your own cloud provider, and lets you migrate between clouds with a few clicks. I’m fairly certain we are the first to offer that, but I’m open to any counter examples.
  - fitzoh 7y ago
    Also, Amazon Timestream isn't usable yet, it's still in private preview.
  - SteveBash 7y ago
    Q: Regarding multi-cloud. Say if AWS has an outage will Timescale cloud fallback to use GCP or Azure? Can something like this be provided? Not sure if the network latency between different cloud providers would allow doing a multi-master replication scheme.
    - ddorian43 7y ago
      Can be done with an async-replication db like ScyllaDB. Or even CP db (just harder/slower).
      - dianasaur323 7y ago
        Yeah! CockroachDB is also a really cool multi-cloud DB. That being said, they are really more for transactional workloads, and less purpose built for time-series. I guess there are always trade-offs in the software world.
    - dianasaur323 7y ago
      You select the public cloud vendor you want your machine spun up on. So no, if AWS has a full outage, it won't fall back to a different cloud. Failover is done at an availability zone level. Since TimescaleDB is also open-source, if you want that kind of replication scheme, you can always install on VMs across clouds. However, as you rightly pointed out, network latency is a definite concern and impacts the feasibility of RPO and RTO.
      - mfreed 7y ago
        One thing to add: The Timescale Cloud does allow you to do is create asynchronous read replicas across different clouds and regions (with a couple clicks). You can then "fork" a read replica (at any point in time) and make it a primary to start serving out of that cloud (again, with just a couple clicks). That's not quite the same as auto-replication/failover between clouds, but getting pretty far there.
contrahax 7y ago
"Powered by Aiven" + nearly identical interface, so this is kind of a reselling arrangement? Aiven supports timescale on postgres, so what additional features does Timescale Cloud provide?
- akulkarni 7y ago
  Yes this is a partnership with Aiven. Aiven Postgres supports the open-source version of TimescaleDB. But Timescale Cloud is the only hosted service where you can get the full TimescaleDB experience, which includes our community and enterprise features (eg interpolation, data retention policies, continuous aggregates, data reordering, etc). [1] It also includes special machines types and plans more suited for time-series data that we co-developed with Aiven. It is also the only hosted service directly staffed by the TimescaleDB development team. [1] https://www.timescale.com/products https://www.timescale.com/products
- derefr 7y ago
  One main difference in these sorts of arrangements is vertically-integrated support. If your Aiven database is having Timescale architecture problems, your support contact is someone working for Aiven who would need to turn around and reach out to Timescale about the bug (or suggest that you do so.) If your Timescale Cloud database is having Timescale architecture problems, your support contact is someone working at Timescale who can just call over the guy who wrote the code with the bug in it. (On the other hand, if your problem is with the Aiven backing cluster, it'll presumbly take slightly longer for Timescale Cloud to resolve, given that they'd have to bounce that forward.)
  - akulkarni 7y ago
    In this case we (Timescale) are your main line of support. Our job is for you to have a 100% positive experience on Timescale Cloud. The buck, as it were, stops with us.
- deleted 7y ago
  [deleted]
solidasparagus 7y ago
Is there any autoscaling or pay-for-what-you-use pricing? It's not 100% clear, but it looks you essentially choose the instance type you want when using Timescale Cloud. I understand why you might not want to call that out specifically in these promotional materials, but it's an important consideration when choosing which managed DB to use and when evaluating cost. What specifically does this mean in practice - "Grow, shrink and migrate your workloads between configurations and plans with ease."
- dianasaur323 7y ago
  It's more like pay-for-what-you use. You can check out the pricing calculator for more detail: https://www.timescale.com/cloud-pricing https://www.timescale.com/cloud-pricing Growing, shrinking, and migrating involve moving to a different instance type, so you have to select a different instance type. That being said, there is very very little downtime (on the order of 3-5 seconds while the DNS resolves)
  - akulkarni 7y ago
    Plus growing, shrinking, migrating only requires a few clicks
  - solidasparagus 7y ago
    Thanks for the clarifications, that's helpful. I wouldn't call it pay-for-what-you-use unless the pricing varies with your actual usage instead of changing when you change plans.
    - dianasaur323 7y ago
      Interesting point of view - it's certainly always a bit hard to find the right verbage that everyone can understand, but hopefully this discussion clarified things!
      - dkersten 7y ago
        Last time I used a traditional hosting provider, I could get a new bare metal server setup in under half an hour. I would hardly call them "pay what you use" even though I could start and stop servers and change the plan I'm on and still be only two to three times slower than doing the same on AWS.
savrajsingh 7y ago
This is cool, is there a guide to migrate from influxDB? Mostly interested in the cost savings.
- cevian 7y ago
  We have a tool to migrate from Influx. https://docs.timescale.com/v1.3/tutorials/outflux https://docs.timescale.com/v1.3/tutorials/outflux
- dianasaur323 7y ago
  We haven't done a formal price comparison, since it's actually a bit hard to compare apples to apples since the two databases are architected differently. Definitely something we should consider doing! Thanks for the idea. Migration wise, I would use Outflux for batch migration and Telegraf to support a live migration (https://docs.timescale.com/v1.3/tutorials/outflux https://docs.timescale.com/v1.3/tutorials/outflux) and (https://docs.timescale.com/v1.3/tutorials/telegraf-output-plugin https://docs.timescale.com/v1.3/tutorials/telegraf-output-pl...).
nodesocket 7y ago
I've been ingesting stock market trading data into MongoDB and using Metabase[1] to visualize it. It is essentially date, price, volume, ticker symbol, exchange. Around 400M documents so far. Queries out of Metabase take upwards of 2-5 minutes to run even simple questions like: Plot the average price of Apple for the last 5 days grouping by minute. Would Timescale Cloud be a better replacement in terms of performance? Is there a nice GUI visualization platform like Metabase for it? [1] https://metabase.com/ https://metabase.com/
- RobAtticus 7y ago
  We did benchmarks a while back comparing TimescaleDB to Mongo[1], and TimescaleDB was quite a bit better. So I think you'd definitely see much improved query time. Also it appears that Metabase supports a PostgreSQL connection[2], so you could probably continue to use it. [1] https://blog.timescale.com/how-to-store-time-series-data-mongodb-vs-timescaledb-postgresql-a73939734016/ https://blog.timescale.com/how-to-store-time-series-data-mon... [2] https://metabase.com/docs/latest/operations-guide/start.html#installing-and-running-metabase https://metabase.com/docs/latest/operations-guide/start.html...
- dianasaur323 7y ago
  Yeah - TimescaleDB comes with a time_bucket function that allows you to group things by minute, and specify a where clause that queries for just the last 5 days. You can build indexes that include the ticker, and also reorder data on disk to optimize how much data you scan on disk. So, TLDR - you should definitely try it! I did some quick googling, and it looks like Metabase supports PostgreSQL, so it should work with Timescale. We would love to hear how it goes!
- DenseComet 7y ago
  Hey I was wondering how you ingest stock market data. What source do you use, and do you use a connector, or did you write one yourself?
  - nodesocket 7y ago
    I use Alpaca[1], specifically the Node.js `websocket.onStockTrades` method and a bit of custom JavaScript code. [1] https://github.com/alpacahq/alpaca-trade-api-js https://github.com/alpacahq/alpaca-trade-api-js
fnord77 7y ago
how does this compare to druid/imply.io ?
- dianasaur323 7y ago
  I think the quickest comparison is SQL vs NoSQL. We haven't done performance benchmarks against Druid yet, but do know of several users who have switched because they want to use PostgreSQL instead.
QuickToBan 7y ago
Why would I pay to use this open source software? It doesn't make sense to me.

89 points by bsg75 7y ago

Just announced today, here's the blog post: https://blog.timescale.com/timescale-cloud-first-fully-managed-time-series-database-service-runs-on-aws-gcp-azure/ https://blog.timescale.com/timescale-cloud-first-fully-manag...

dang 7y ago
That gives more background info so we've switched to it from https://www.timescale.com/cloud https://www.timescale.com/cloud. Thanks!

mharroun 7y ago

It seems disingenuous to call it the first. AWS has it's own time series db. In terms of open source Apache druid had a managed cloud variant that imply.io runs.

akulkarni 7y ago
It’s the first multi-cloud. AWS obviously just AWS. Based on Imply Cloud website it looks like it too is just AWS. Timescale Cloud gives you freedom to choose your own cloud provider, and lets you migrate between clouds with a few clicks. I’m fairly certain we are the first to offer that, but I’m open to any counter examples.
- fitzoh 7y ago
  Also, Amazon Timestream isn't usable yet, it's still in private preview.
- SteveBash 7y ago
  Q: Regarding multi-cloud. Say if AWS has an outage will Timescale cloud fallback to use GCP or Azure? Can something like this be provided? Not sure if the network latency between different cloud providers would allow doing a multi-master replication scheme.
  - ddorian43 7y ago
    Can be done with an async-replication db like ScyllaDB. Or even CP db (just harder/slower).
    - dianasaur323 7y ago
      Yeah! CockroachDB is also a really cool multi-cloud DB. That being said, they are really more for transactional workloads, and less purpose built for time-series. I guess there are always trade-offs in the software world.
  - dianasaur323 7y ago
    You select the public cloud vendor you want your machine spun up on. So no, if AWS has a full outage, it won't fall back to a different cloud. Failover is done at an availability zone level. Since TimescaleDB is also open-source, if you want that kind of replication scheme, you can always install on VMs across clouds. However, as you rightly pointed out, network latency is a definite concern and impacts the feasibility of RPO and RTO.
    - mfreed 7y ago
      One thing to add: The Timescale Cloud does allow you to do is create asynchronous read replicas across different clouds and regions (with a couple clicks). You can then "fork" a read replica (at any point in time) and make it a primary to start serving out of that cloud (again, with just a couple clicks). That's not quite the same as auto-replication/failover between clouds, but getting pretty far there.

contrahax 7y ago

"Powered by Aiven" + nearly identical interface, so this is kind of a reselling arrangement? Aiven supports timescale on postgres, so what additional features does Timescale Cloud provide?

akulkarni 7y ago
Yes this is a partnership with Aiven. Aiven Postgres supports the open-source version of TimescaleDB. But Timescale Cloud is the only hosted service where you can get the full TimescaleDB experience, which includes our community and enterprise features (eg interpolation, data retention policies, continuous aggregates, data reordering, etc). [1] It also includes special machines types and plans more suited for time-series data that we co-developed with Aiven. It is also the only hosted service directly staffed by the TimescaleDB development team. [1] https://www.timescale.com/products https://www.timescale.com/products
derefr 7y ago
One main difference in these sorts of arrangements is vertically-integrated support. If your Aiven database is having Timescale architecture problems, your support contact is someone working for Aiven who would need to turn around and reach out to Timescale about the bug (or suggest that you do so.) If your Timescale Cloud database is having Timescale architecture problems, your support contact is someone working at Timescale who can just call over the guy who wrote the code with the bug in it. (On the other hand, if your problem is with the Aiven backing cluster, it'll presumbly take slightly longer for Timescale Cloud to resolve, given that they'd have to bounce that forward.)
- akulkarni 7y ago
  In this case we (Timescale) are your main line of support. Our job is for you to have a 100% positive experience on Timescale Cloud. The buck, as it were, stops with us.
deleted 7y ago
[deleted]

solidasparagus 7y ago

Is there any autoscaling or pay-for-what-you-use pricing? It's not 100% clear, but it looks you essentially choose the instance type you want when using Timescale Cloud. I understand why you might not want to call that out specifically in these promotional materials, but it's an important consideration when choosing which managed DB to use and when evaluating cost. What specifically does this mean in practice - "Grow, shrink and migrate your workloads between configurations and plans with ease."

dianasaur323 7y ago
It's more like pay-for-what-you use. You can check out the pricing calculator for more detail: https://www.timescale.com/cloud-pricing https://www.timescale.com/cloud-pricing Growing, shrinking, and migrating involve moving to a different instance type, so you have to select a different instance type. That being said, there is very very little downtime (on the order of 3-5 seconds while the DNS resolves)
- akulkarni 7y ago
  Plus growing, shrinking, migrating only requires a few clicks
- solidasparagus 7y ago
  Thanks for the clarifications, that's helpful. I wouldn't call it pay-for-what-you-use unless the pricing varies with your actual usage instead of changing when you change plans.
  - dianasaur323 7y ago
    Interesting point of view - it's certainly always a bit hard to find the right verbage that everyone can understand, but hopefully this discussion clarified things!
    - dkersten 7y ago
      Last time I used a traditional hosting provider, I could get a new bare metal server setup in under half an hour. I would hardly call them "pay what you use" even though I could start and stop servers and change the plan I'm on and still be only two to three times slower than doing the same on AWS.

savrajsingh 7y ago

This is cool, is there a guide to migrate from influxDB? Mostly interested in the cost savings.

cevian 7y ago
We have a tool to migrate from Influx. https://docs.timescale.com/v1.3/tutorials/outflux https://docs.timescale.com/v1.3/tutorials/outflux
dianasaur323 7y ago
We haven't done a formal price comparison, since it's actually a bit hard to compare apples to apples since the two databases are architected differently. Definitely something we should consider doing! Thanks for the idea. Migration wise, I would use Outflux for batch migration and Telegraf to support a live migration (https://docs.timescale.com/v1.3/tutorials/outflux https://docs.timescale.com/v1.3/tutorials/outflux) and (https://docs.timescale.com/v1.3/tutorials/telegraf-output-plugin https://docs.timescale.com/v1.3/tutorials/telegraf-output-pl...).

nodesocket 7y ago

I've been ingesting stock market trading data into MongoDB and using Metabase[1] to visualize it. It is essentially date, price, volume, ticker symbol, exchange. Around 400M documents so far. Queries out of Metabase take upwards of 2-5 minutes to run even simple questions like: Plot the average price of Apple for the last 5 days grouping by minute. Would Timescale Cloud be a better replacement in terms of performance? Is there a nice GUI visualization platform like Metabase for it? [1] https://metabase.com/ https://metabase.com/

RobAtticus 7y ago
We did benchmarks a while back comparing TimescaleDB to Mongo[1], and TimescaleDB was quite a bit better. So I think you'd definitely see much improved query time. Also it appears that Metabase supports a PostgreSQL connection[2], so you could probably continue to use it. [1] https://blog.timescale.com/how-to-store-time-series-data-mongodb-vs-timescaledb-postgresql-a73939734016/ https://blog.timescale.com/how-to-store-time-series-data-mon... [2] https://metabase.com/docs/latest/operations-guide/start.html#installing-and-running-metabase https://metabase.com/docs/latest/operations-guide/start.html...
dianasaur323 7y ago
Yeah - TimescaleDB comes with a time_bucket function that allows you to group things by minute, and specify a where clause that queries for just the last 5 days. You can build indexes that include the ticker, and also reorder data on disk to optimize how much data you scan on disk. So, TLDR - you should definitely try it! I did some quick googling, and it looks like Metabase supports PostgreSQL, so it should work with Timescale. We would love to hear how it goes!
DenseComet 7y ago
Hey I was wondering how you ingest stock market data. What source do you use, and do you use a connector, or did you write one yourself?
- nodesocket 7y ago
  I use Alpaca[1], specifically the Node.js `websocket.onStockTrades` method and a bit of custom JavaScript code. [1] https://github.com/alpacahq/alpaca-trade-api-js https://github.com/alpacahq/alpaca-trade-api-js

fnord77 7y ago

how does this compare to druid/imply.io ?

dianasaur323 7y ago
I think the quickest comparison is SQL vs NoSQL. We haven't done performance benchmarks against Druid yet, but do know of several users who have switched because they want to use PostgreSQL instead.

QuickToBan 7y ago

Why would I pay to use this open source software? It doesn't make sense to me.