15 ms·

Timescale Cloud: Multi-cloud, fully-managed time-series in AWS, Azure, and GCP

200 points by bithavoc 6y ago

k_bx 6y ago
I wonder, could GNU/Linux change its license (in a substantial way) that would allow it to get revenue from Amazon for deployments like AWS, similar to how it was done by Timescale? Would seem fare to get a good chunk of revenue to Linux development. p.s.: could Postgres change its license to make Timescale share their revenue? :D
- xiamx 6y ago
  Why would they do that. These guys are generating revenue by offering managed service, not by selling opensource software.
  - stingraycharles 6y ago
    For the same reasons MongoDB and Redis changed their licenses to avoid AWS et al offering it as a managed service?
    - xiamx 6y ago
      That's a different story. Aws was competiting for revenue with the developers of MongoDB and Redis' for profit company. Postgres' publisher does not use managed service as a source of income.
      - stingraycharles 6y ago
        Of course, but I was just offering a reason for why they would want to do that. It’s not as if it’s unreasonable if they did that.
- dekhn 6y ago
  No
- DetroitThrow 6y ago
  With AGPL, if you block AWS, you also block other users unless you have additional licenses for your software, like Mongo and Timescale have to do, and I'm not sure people would want to deal with that when free alternatives like MySQL exist. With Amazon, they were able to make their own Linux flavor so they could pay Red Hat less. I think if Linux itself sold enterprise licenses, we'd end up with a much more fragmented or corporate batch of OSes taking its place, and I think that's very negative for the ecosystem in general. No one wants to deal with that :/ I'm surprised people want to deal with MongoDB at this point.
brightball 6y ago
I would love to see something like this with pricing that also competed on the lower end with various RDS and Cloud SQL offerings. Just using Timescale instead of RDS or Cloud SQL PostgreSQL would be a virtual assumption, but with the starting price points so much higher it makes the decision to START with Timescale a lot harder.
- allset_ 6y ago
  Agreed. Having your lowest tier offering priced at over $500/mo is cost prohibitive in a lot of situations. Even their "dev" tier is $130/mo for a single instance. It would be much nicer if they just let you choose how much storage, CPU, and memory you need rather than only letting you select from a limited set of offerings.
  - mfreed 6y ago
    The main difference on Timescale Cloud between dev (starting $115) and production is just the resource availability and slightly longer PITR recovery period for non-dev options. Both offer same full TimescaleDB experience, console, ability to upgrade/downgrade, fork your database, etc. If you do want a lower priced option, we actually recently also released a second managed cloud offering in public preview, Timescale Forge: https://timescale.com/forge https://timescale.com/forge. Timescale Cloud provides great flexibility with multi-cloud and 75+ regions, security conscious options like VPC peering and SOC2/ISO/HIPAA compliance, and a more traditional DBaaS Experience. Timescale Forge is a innovative platform with capabilities like decoupled compute and storage (so you can select yourself the exact configuration you want, starting at $49/month), instant pause/resume, native support for our Prometheus integration, and more coming...
    - dkersten 6y ago
      Eek, Forge price starts at $49 but the next tier up is $166, after which it incrementally increases. That’s quite a jump. If my budget is in that price range, then such a jump is likely a huge deal for me.
eloff 6y ago
TimescaleDB is great, not just for timeseries data, but also if you're trying to scale your Postgres database (with some cavaets, obviously.) One often has a small number of tables, often a single table, with a high write workload. Changing it to a timescale hypertable, provided you can work with the additional restrictions that has, can let you instantly 10X your write throughput (or more with multi-node timescale.) That's a lot cheaper than trying to scale your database write IO by 10x. I like multi-cloud services like this because it reduces the vendor lock-in that happens with these public clouds.
- mfreed 6y ago
  (Timescale person) This is also true for Timescale Cloud -- especially once people are using it for TimescaleDB (which, as you probably know, is implemented as a low-level extension to Postgres), we see some users start to migrate over their more traditional Postgres workloads to Timescale Cloud as well. One database platform for all their data, one (great) support team to speak with, etc. And thanks for the compliment!
- ants_a 6y ago
  Why wouldn't you use built in partitioning for this?
  - mulmen 6y ago
    I think its a time/money tradeoff. I recall a comment along those lines (paraphrased) the last time I asked about TimeseriesDB. It's the fundamental calculation of SaaS/Cloud.
    - ants_a 6y ago
      I don't feel there is a significant time investment with using the builtin way. Timescale has some neat features for handling time series data, but for just sharding out data for scalability I can't see it offering anything substantial. And in that vein, I feel the marketing is somewhat disingenuous in the performance claims. When comparing to vanilla PostgreSQL it is comparing against a very naive usage pattern. People have been getting the same kinds of performance increases from partitioning out time series data for a loooong time.
crb 6y ago
I see Timescale Helm charts for Kubernetes: https://github.com/timescale/timescaledb-kubernetes https://github.com/timescale/timescaledb-kubernetes. Are you using Kubernetes to host the managed service?
- mfreed 6y ago
  Yep, we like to eat our dog food =) Timescale Forge, which I mentioned elsewhere, is based all around Kubernetes (including StatefulSets), which enables its decoupled compute/storage and instant pause/resume. For that we employ helm charts, but also have additionally built k8s operators + more internally to handle the dynamic operation and reconfiguration you want from an "full lifecycle"-managed database service, while the helm charts are slightly more static.
- jjjensen90 6y ago
  I brought Timescale to my current job, and I have to plug their Helm charts. They have made deployment, backup, scaling, and config so easy on k8s for Timescale. Probably the best postgres chart that exists, period. Great work TSDB team.
aeyes 6y ago
> You can now incrementally scale up or down in the region of your choice, with CPU options ranging from 2 to 72 CPUs and storage ranging from 20GB to 10TB What does that mean? The pricing calculator on https://www.timescale.com/cloud https://www.timescale.com/cloud still has the same predefined storage configurations. Also, you might want to remove this customer opinion from the post, standard Postgres won't break a sweat at 100M rows: > We have over 100 million rows of data in development and production, and we needed a database that could handle this volume
- mfreed 6y ago
  In the Timescale Cloud portal, once you create a database, it's one-click "upgrade" to move between any of these configurations, and it happens with ~zero downtime. That's also true for multi-cloud migration, you just select that you want to migrate from a 100GB instance on AWS to a 200GB instance on GCP, and the migration happens behind the scenes. (Technically, it does so by spinning up a replica of the new configuration / location, and continuously migrates the database, including any online changes happening concurrently. Then, at the last minute, it "swaps" over control from the primary to replica, and then retires the old primary. This is all hidden behind DNS too.)
- fourseventy 6y ago
  Depends what you are doing with those 100M rows. We have a database with about that many rows and we need to do sums of a few hundred thousand rows, as well as bucket those rows into daily buckets and graph them in real time. With just regular Postgres these operations are pretty slow (> ~5 seconds), but with Timescale continuous aggregates and hypertables they run in milliseconds.
- claytonjy 6y ago
  That's my quote :) In hindsight I should have said "hundreds of millions"; we're at about a third of a million _sensor packets_ in production, some of which are dozens of rows across several tables. Yeah, you could absolutely stick this in a vanilla postgres instance, but as a sibling post said, you wouldn't get the same performance on the queries we care most about.
GordonS 6y ago
Does this come pre-configured for high availability (e.g. replication to a standby node)? Given the pricing I would have assumed "yes", but I don't see any mention of it in the blog post?
- GordonS 6y ago
  I meant to ask about backup with point-in-time restore too - is that provided out of the box?
  - mfreed 6y ago
    Yes, there are a number of great options here provided out-of-box: 1. All plans (including the lowest-cost dev tier) includes PITR recovery out of the box. You can also "fork" any current database to a second instance (e.g., for dev & test), also at any recent point-in-time. 2. Once you create any instance, you can later create a "read replica" with one click, which creates an asynchronous replica of your primary (e.g., for ad-hoc queries to not interfere with the primary, or to just load-balance reads). 3. On the HA side, our "pro" plans includes automatic synchronous replication between primary/replica, so that any primary crashes failover without downtime to the replica. You also get DNS-named endpoints for replicas, again for load balancing reads.
    - GordonS 6y ago
      On the Basic tier, I presume you have to pay for those read-only replicas? Can they be used for automatic failover, or only for scaling reads? I don't know what VM SKUs you use on Azure, but at a glance it looks like your solution would cost 300-400% the cost of the VM, just for your Basic tier. As a fan of TimescaleDB, I really want to like this, but I'm just not seeing enough value-add for the price. Maybe I'm missing something, please sell me on the benefits over managing VMs myself?
- temp667 6y ago
  I'm curious when folks say they expect a dev plan offering at a pretty low price point to come out of the box with HA - are they comparing this to an Oracle High Availability solution? Because $75 or $150 for HA on oracle is not going to happen - period. It's just interesting how the price points are so wildly divergent.
mfreed 6y ago
And...if anybody is interested in helping us build the best cloud service for time-series data, we're hiring cloud engineers! https://www.timescale.com/careers/4128985002-cloud-engineer https://www.timescale.com/careers/4128985002-cloud-engineer Timescale is a remote-first company with amazing folks all over the world. We're also actively looking for technical support engineers with strong Postgres backgrounds: https://www.timescale.com/careers/4615287002-technical-support-engineer https://www.timescale.com/careers/4615287002-technical-suppo...
vladaionescu 6y ago
We've used Timescale at ShiftLeft since before 1.0. Recommend 100%
- akulkarni 6y ago
  Thank you Vlad for the kind words :-)
mfreed 6y ago
Would love to hear from the HN community: what are some of your favorite features in managed database or infra platforms, or what do you wish more platforms offered that are still hard-to-find?
nodesocket 6y ago
I am currently using MongoDB to store Ethereum tick price data. I'd like to switch to Timescale, but wondering if there is a supported Node.js API client? EDIT: Looks like the documentation recommends using Sequelize ORM.
- mfreed 6y ago
  ORMs that work for Postgres should "just work" for TimescaleDB. So while we mention Sequelize in some tutorials, if you have a favorite Node.js API client, you should just go for it. As aside, you mention enjoy this developer Q&A that a community member published a few days ago that talk about his experience using TimescaleDB to build a crypto trading bot. https://blog.timescale.com/blog/how-i-power-a-successful-crypto-trading-bot-with-timescaledb/ https://blog.timescale.com/blog/how-i-power-a-successful-cry... Also, an earlier tutorial: https://blog.timescale.com/blog/analyzing-bitcoin-ethereum-and-4100-other-cryptocurrencies-using-postgresql-and-timescaledb/ https://blog.timescale.com/blog/analyzing-bitcoin-ethereum-a...
- mfreed 6y ago
  Also, we have a very active slack community if you want any other opinions: https://slack.timescale.com/ https://slack.timescale.com/ I'm mike there - come say hi.
simonebrunozzi 6y ago
You have one of the smartest and nicest head of Marketing there. I worked with Prashant at AWS (more than 10 years ago!) and have the fondest memories. Besides this: congrats on the launch. I tried your product a while ago and was impressed. Big cloud providers are the obvious perfect channel for you. Just get ready for some nasty competition from the same guys :)
- akulkarni 6y ago
  Aw, I agree! Will let Prashant know. Re: Cloud providers - I also agree, it's a complicated world ;)
spo81rty 6y ago
I was interested in Timescaledb recently but found the cost to be extremely expensive and you can only get all the good features in their cloud version as PaaS. I would also suggest looking at Citus for scaling Postgresql. With the Azure reserved discounts, Citus is like less than half the price of Timescaledb. Citus also helps tremendously with multi-tenant scenarios. Of course, depends on what you are trying to do.
- drpebcak 6y ago
  I’m curious what you mean by this - the community version has most of the features in it.. and if you are willing to pay for something, you’d an run the enterprise version on-prem.. my experience with that is that it’s quite cheap, and you get support as well. Timescale also has a multi node version now that distributes your data across a cluster sort of like Citus.
  - spo81rty 6y ago
    I want a hosted PaaS solution. I don't want to self host it. That is the pricing I am comparing.
    - akulkarni 6y ago
      Citus and Timescale are apples and oranges. Citus is just scale-out Postgres. TimescaleDB is Postgres for time-series. There are a long list of capabilities that we've built for time-series that Citus does not have: native 15x compression, data retention policies, continuous aggregates (real-time materialized views refreshed automatically), interpolation, etc. Nothing against Citus (we are friends with that team) - but they are designed to solve a different problem than we are.

200 points by bithavoc 6y ago

k_bx 6y ago

I wonder, could GNU/Linux change its license (in a substantial way) that would allow it to get revenue from Amazon for deployments like AWS, similar to how it was done by Timescale? Would seem fare to get a good chunk of revenue to Linux development. p.s.: could Postgres change its license to make Timescale share their revenue? :D

xiamx 6y ago
Why would they do that. These guys are generating revenue by offering managed service, not by selling opensource software.
- stingraycharles 6y ago
  For the same reasons MongoDB and Redis changed their licenses to avoid AWS et al offering it as a managed service?
  - xiamx 6y ago
    That's a different story. Aws was competiting for revenue with the developers of MongoDB and Redis' for profit company. Postgres' publisher does not use managed service as a source of income.
    - stingraycharles 6y ago
      Of course, but I was just offering a reason for why they would want to do that. It’s not as if it’s unreasonable if they did that.
dekhn 6y ago
No
DetroitThrow 6y ago
With AGPL, if you block AWS, you also block other users unless you have additional licenses for your software, like Mongo and Timescale have to do, and I'm not sure people would want to deal with that when free alternatives like MySQL exist. With Amazon, they were able to make their own Linux flavor so they could pay Red Hat less. I think if Linux itself sold enterprise licenses, we'd end up with a much more fragmented or corporate batch of OSes taking its place, and I think that's very negative for the ecosystem in general. No one wants to deal with that :/ I'm surprised people want to deal with MongoDB at this point.

brightball 6y ago

I would love to see something like this with pricing that also competed on the lower end with various RDS and Cloud SQL offerings. Just using Timescale instead of RDS or Cloud SQL PostgreSQL would be a virtual assumption, but with the starting price points so much higher it makes the decision to START with Timescale a lot harder.

allset_ 6y ago
Agreed. Having your lowest tier offering priced at over $500/mo is cost prohibitive in a lot of situations. Even their "dev" tier is $130/mo for a single instance. It would be much nicer if they just let you choose how much storage, CPU, and memory you need rather than only letting you select from a limited set of offerings.
- mfreed 6y ago
  The main difference on Timescale Cloud between dev (starting $115) and production is just the resource availability and slightly longer PITR recovery period for non-dev options. Both offer same full TimescaleDB experience, console, ability to upgrade/downgrade, fork your database, etc. If you do want a lower priced option, we actually recently also released a second managed cloud offering in public preview, Timescale Forge: https://timescale.com/forge https://timescale.com/forge. Timescale Cloud provides great flexibility with multi-cloud and 75+ regions, security conscious options like VPC peering and SOC2/ISO/HIPAA compliance, and a more traditional DBaaS Experience. Timescale Forge is a innovative platform with capabilities like decoupled compute and storage (so you can select yourself the exact configuration you want, starting at $49/month), instant pause/resume, native support for our Prometheus integration, and more coming...
  - dkersten 6y ago
    Eek, Forge price starts at $49 but the next tier up is $166, after which it incrementally increases. That’s quite a jump. If my budget is in that price range, then such a jump is likely a huge deal for me.

eloff 6y ago

TimescaleDB is great, not just for timeseries data, but also if you're trying to scale your Postgres database (with some cavaets, obviously.) One often has a small number of tables, often a single table, with a high write workload. Changing it to a timescale hypertable, provided you can work with the additional restrictions that has, can let you instantly 10X your write throughput (or more with multi-node timescale.) That's a lot cheaper than trying to scale your database write IO by 10x. I like multi-cloud services like this because it reduces the vendor lock-in that happens with these public clouds.

mfreed 6y ago
(Timescale person) This is also true for Timescale Cloud -- especially once people are using it for TimescaleDB (which, as you probably know, is implemented as a low-level extension to Postgres), we see some users start to migrate over their more traditional Postgres workloads to Timescale Cloud as well. One database platform for all their data, one (great) support team to speak with, etc. And thanks for the compliment!
ants_a 6y ago
Why wouldn't you use built in partitioning for this?
- mulmen 6y ago
  I think its a time/money tradeoff. I recall a comment along those lines (paraphrased) the last time I asked about TimeseriesDB. It's the fundamental calculation of SaaS/Cloud.
  - ants_a 6y ago
    I don't feel there is a significant time investment with using the builtin way. Timescale has some neat features for handling time series data, but for just sharding out data for scalability I can't see it offering anything substantial. And in that vein, I feel the marketing is somewhat disingenuous in the performance claims. When comparing to vanilla PostgreSQL it is comparing against a very naive usage pattern. People have been getting the same kinds of performance increases from partitioning out time series data for a loooong time.

crb 6y ago

I see Timescale Helm charts for Kubernetes: https://github.com/timescale/timescaledb-kubernetes https://github.com/timescale/timescaledb-kubernetes. Are you using Kubernetes to host the managed service?

mfreed 6y ago
Yep, we like to eat our dog food =) Timescale Forge, which I mentioned elsewhere, is based all around Kubernetes (including StatefulSets), which enables its decoupled compute/storage and instant pause/resume. For that we employ helm charts, but also have additionally built k8s operators + more internally to handle the dynamic operation and reconfiguration you want from an "full lifecycle"-managed database service, while the helm charts are slightly more static.
jjjensen90 6y ago
I brought Timescale to my current job, and I have to plug their Helm charts. They have made deployment, backup, scaling, and config so easy on k8s for Timescale. Probably the best postgres chart that exists, period. Great work TSDB team.

aeyes 6y ago

> You can now incrementally scale up or down in the region of your choice, with CPU options ranging from 2 to 72 CPUs and storage ranging from 20GB to 10TB What does that mean? The pricing calculator on https://www.timescale.com/cloud https://www.timescale.com/cloud still has the same predefined storage configurations. Also, you might want to remove this customer opinion from the post, standard Postgres won't break a sweat at 100M rows: > We have over 100 million rows of data in development and production, and we needed a database that could handle this volume

mfreed 6y ago
In the Timescale Cloud portal, once you create a database, it's one-click "upgrade" to move between any of these configurations, and it happens with ~zero downtime. That's also true for multi-cloud migration, you just select that you want to migrate from a 100GB instance on AWS to a 200GB instance on GCP, and the migration happens behind the scenes. (Technically, it does so by spinning up a replica of the new configuration / location, and continuously migrates the database, including any online changes happening concurrently. Then, at the last minute, it "swaps" over control from the primary to replica, and then retires the old primary. This is all hidden behind DNS too.)
fourseventy 6y ago
Depends what you are doing with those 100M rows. We have a database with about that many rows and we need to do sums of a few hundred thousand rows, as well as bucket those rows into daily buckets and graph them in real time. With just regular Postgres these operations are pretty slow (> ~5 seconds), but with Timescale continuous aggregates and hypertables they run in milliseconds.
claytonjy 6y ago
That's my quote :) In hindsight I should have said "hundreds of millions"; we're at about a third of a million _sensor packets_ in production, some of which are dozens of rows across several tables. Yeah, you could absolutely stick this in a vanilla postgres instance, but as a sibling post said, you wouldn't get the same performance on the queries we care most about.

GordonS 6y ago

Does this come pre-configured for high availability (e.g. replication to a standby node)? Given the pricing I would have assumed "yes", but I don't see any mention of it in the blog post?

GordonS 6y ago
I meant to ask about backup with point-in-time restore too - is that provided out of the box?
- mfreed 6y ago
  Yes, there are a number of great options here provided out-of-box: 1. All plans (including the lowest-cost dev tier) includes PITR recovery out of the box. You can also "fork" any current database to a second instance (e.g., for dev & test), also at any recent point-in-time. 2. Once you create any instance, you can later create a "read replica" with one click, which creates an asynchronous replica of your primary (e.g., for ad-hoc queries to not interfere with the primary, or to just load-balance reads). 3. On the HA side, our "pro" plans includes automatic synchronous replication between primary/replica, so that any primary crashes failover without downtime to the replica. You also get DNS-named endpoints for replicas, again for load balancing reads.
  - GordonS 6y ago
    On the Basic tier, I presume you have to pay for those read-only replicas? Can they be used for automatic failover, or only for scaling reads? I don't know what VM SKUs you use on Azure, but at a glance it looks like your solution would cost 300-400% the cost of the VM, just for your Basic tier. As a fan of TimescaleDB, I really want to like this, but I'm just not seeing enough value-add for the price. Maybe I'm missing something, please sell me on the benefits over managing VMs myself?
temp667 6y ago
I'm curious when folks say they expect a dev plan offering at a pretty low price point to come out of the box with HA - are they comparing this to an Oracle High Availability solution? Because $75 or $150 for HA on oracle is not going to happen - period. It's just interesting how the price points are so wildly divergent.

mfreed 6y ago

And...if anybody is interested in helping us build the best cloud service for time-series data, we're hiring cloud engineers! https://www.timescale.com/careers/4128985002-cloud-engineer https://www.timescale.com/careers/4128985002-cloud-engineer Timescale is a remote-first company with amazing folks all over the world. We're also actively looking for technical support engineers with strong Postgres backgrounds: https://www.timescale.com/careers/4615287002-technical-support-engineer https://www.timescale.com/careers/4615287002-technical-suppo...

vladaionescu 6y ago

We've used Timescale at ShiftLeft since before 1.0. Recommend 100%

akulkarni 6y ago
Thank you Vlad for the kind words :-)

mfreed 6y ago

Would love to hear from the HN community: what are some of your favorite features in managed database or infra platforms, or what do you wish more platforms offered that are still hard-to-find?

nodesocket 6y ago

I am currently using MongoDB to store Ethereum tick price data. I'd like to switch to Timescale, but wondering if there is a supported Node.js API client? EDIT: Looks like the documentation recommends using Sequelize ORM.

mfreed 6y ago
ORMs that work for Postgres should "just work" for TimescaleDB. So while we mention Sequelize in some tutorials, if you have a favorite Node.js API client, you should just go for it. As aside, you mention enjoy this developer Q&A that a community member published a few days ago that talk about his experience using TimescaleDB to build a crypto trading bot. https://blog.timescale.com/blog/how-i-power-a-successful-crypto-trading-bot-with-timescaledb/ https://blog.timescale.com/blog/how-i-power-a-successful-cry... Also, an earlier tutorial: https://blog.timescale.com/blog/analyzing-bitcoin-ethereum-and-4100-other-cryptocurrencies-using-postgresql-and-timescaledb/ https://blog.timescale.com/blog/analyzing-bitcoin-ethereum-a...
mfreed 6y ago
Also, we have a very active slack community if you want any other opinions: https://slack.timescale.com/ https://slack.timescale.com/ I'm mike there - come say hi.

simonebrunozzi 6y ago

You have one of the smartest and nicest head of Marketing there. I worked with Prashant at AWS (more than 10 years ago!) and have the fondest memories. Besides this: congrats on the launch. I tried your product a while ago and was impressed. Big cloud providers are the obvious perfect channel for you. Just get ready for some nasty competition from the same guys :)

akulkarni 6y ago
Aw, I agree! Will let Prashant know. Re: Cloud providers - I also agree, it's a complicated world ;)

spo81rty 6y ago

I was interested in Timescaledb recently but found the cost to be extremely expensive and you can only get all the good features in their cloud version as PaaS. I would also suggest looking at Citus for scaling Postgresql. With the Azure reserved discounts, Citus is like less than half the price of Timescaledb. Citus also helps tremendously with multi-tenant scenarios. Of course, depends on what you are trying to do.

drpebcak 6y ago
I’m curious what you mean by this - the community version has most of the features in it.. and if you are willing to pay for something, you’d an run the enterprise version on-prem.. my experience with that is that it’s quite cheap, and you get support as well. Timescale also has a multi node version now that distributes your data across a cluster sort of like Citus.
- spo81rty 6y ago
  I want a hosted PaaS solution. I don't want to self host it. That is the pricing I am comparing.
  - akulkarni 6y ago
    Citus and Timescale are apples and oranges. Citus is just scale-out Postgres. TimescaleDB is Postgres for time-series. There are a long list of capabilities that we've built for time-series that Citus does not have: native 15x compression, data retention policies, continuous aggregates (real-time materialized views refreshed automatically), interpolation, etc. Nothing against Citus (we are friends with that team) - but they are designed to solve a different problem than we are.