4 ms·

On topic: While the comparison seems terrible for Timestream, as a customer who does not want to manage my databases I would love a similar GCP option, if the

by nwsm 6y ago

On topic: While the comparison seems terrible for Timestream, as a customer who does not want to manage my databases I would love a similar GCP option, if the product had better tradeoffs. It's also interesting that Timescale attributes this AWS product to their own licensing. They had some much discussed [0] developments on that front, and if it did in fact force AWS to build their own implementation, that seems like a win for opensource, but not so much for serverless users. Slightly off topic: I have a side project that generates ~10 daily metrics for ~350 entities. Around 150k readings currently (the number of metrics gathered per entity per day has increased since I started). I'm pretty interested in these timeseries databases, but I think my use case is still on the side of using MySQL/Postgres out of the box. Not to mention that I can get by with archiving data > 1 year old. Does anyone have any quick checklist or metrics to make decisions like this? When does it make sense to evolve from a vanilla RDB to a timeseries one? [0] https://news.ycombinator.com/item?id=24579905 https://news.ycombinator.com/item?id=24579905

mfreed 6y ago
Hi @nwsm: Timescale offers a fully managed version of TimescaleDB on AWS, GCP, and Azure, starting at $49/month. https://www.timescale.com/timescale-signup https://www.timescale.com/timescale-signup
- nwsm 6y ago
  Thank you. At that price it's probably overkill for me, but I wish you had more info on your site that wasn't gated by a signup. How is the product managed in my GCP environment?
  - mfreed 6y ago
    Sorry harder to found, but pricing calculator on product page (not gated): https://www.timescale.com/products#pricing https://www.timescale.com/products#pricing
ryanbooz 6y ago
I'd just add that if you have any belief that your application will grow, and it's time-series in nature, there's usually only upside to starting with a time-series specific database now. The hoops you'll jump through as the application grows will often bring you more headache than you can imagine now. Before coming to work at Timescale a few months ago, I spent 18 years managing products at two companies, both of which were time-based applications (utility billing/energy data & IIoT). In both cases, the app started small and everything seemed fine. We could usually get around performance issues with other hacks. But in both cases there was a tipping point because the original database (one relational, no NoSQL) just wasn't designed for the challenge as the app scaled. So whether you manage it yourself in a smaller environment for now or try something like Timescale Cloud, I can (almost) guarantee that you'll thank yourself in a year or two. ;-)
aforwardslash 6y ago
> Does anyone have any quick checklist or metrics to make decisions like this? I'm not a database expert in any way, but I've had to evaluate database options for both existing and new applications, and my "methodology" is as follows: - Understand the workload Is the database workload read-intensive, write-intensive, or both? Does the application require strong consistency? Does it require replication? What kind of replication? Can writes be batched, or are processed ad-hoc? What kind of read operations will you do, and what is the desired response times for those operations (ex. is 2s tolerable? 500ms? 20ms?) - Scouting Select a set of available options to benchmark using whatever criteria you may see fit, but following the previous step constraints. - Benchmark Create a somewhat simple model that fits your workload and models your real case, using the language you'll be using to build your application; Create benchmark code for the most critical/complex operations; Run it at least 5 times on similar hardware of what you expect production to be (this is important; eg. storage (both memory and disk) latency varies wildly between "my laptop" and a cloud provider) and record the results; Be sure to keep an eye on memory consumption, cpu and disk usage when running the benchmarks; - Decide :) Usual criteria: * Price; * Features (of course); * Easiness of deploy or cloud provider availablity; * Documentation quality; * Driver quality for your application language (it may happen you already excluded a candidate on the previous step due to this); * Query language familiarity; * Performance; * Adoption (translated to: can I expect a new developer to have the basic skills on this tech?);
valyala 6y ago
> Does anyone have any quick checklist or metrics to make decisions like this? When does it make sense to evolve from a vanilla RDB to a timeseries one? 10 daily metrics for 350 entities result in `10 * 350=3.5K` reading per day. This translates to `3.5K * 365=1.3M` readings per year. Such workload can be easily handled by any DBMS out there. There is no need to search for specialized time series database for this workload. The need in specialized TSDB solutions arises when ingestion rate reaches a million of readings per second and the number of readings stored in the database exceeds hundreds of billions and trillions. Such a workload cannot be handled by general-purpose database, but it is easily handled by specialized databases such as VictoriaMetrics. Fun fact that it easily handles 10 trillions (e.g. 10K billions) of readings in a single-node setup - see https://victoriametrics.github.io/CaseStudies.html#wixcom https://victoriametrics.github.io/CaseStudies.html#wixcom
- randomacc385937 6y ago
  Seriously stop spamming your product in every single one of your messages. Especially when you fail to mention that you have a vested interest

by nwsm 6y ago

mfreed 6y ago

Hi @nwsm: Timescale offers a fully managed version of TimescaleDB on AWS, GCP, and Azure, starting at $49/month. https://www.timescale.com/timescale-signup https://www.timescale.com/timescale-signup

nwsm 6y ago
Thank you. At that price it's probably overkill for me, but I wish you had more info on your site that wasn't gated by a signup. How is the product managed in my GCP environment?
- mfreed 6y ago
  Sorry harder to found, but pricing calculator on product page (not gated): https://www.timescale.com/products#pricing https://www.timescale.com/products#pricing

ryanbooz 6y ago

I'd just add that if you have any belief that your application will grow, and it's time-series in nature, there's usually only upside to starting with a time-series specific database now. The hoops you'll jump through as the application grows will often bring you more headache than you can imagine now. Before coming to work at Timescale a few months ago, I spent 18 years managing products at two companies, both of which were time-based applications (utility billing/energy data & IIoT). In both cases, the app started small and everything seemed fine. We could usually get around performance issues with other hacks. But in both cases there was a tipping point because the original database (one relational, no NoSQL) just wasn't designed for the challenge as the app scaled. So whether you manage it yourself in a smaller environment for now or try something like Timescale Cloud, I can (almost) guarantee that you'll thank yourself in a year or two. ;-)

aforwardslash 6y ago

> Does anyone have any quick checklist or metrics to make decisions like this? I'm not a database expert in any way, but I've had to evaluate database options for both existing and new applications, and my "methodology" is as follows: - Understand the workload Is the database workload read-intensive, write-intensive, or both? Does the application require strong consistency? Does it require replication? What kind of replication? Can writes be batched, or are processed ad-hoc? What kind of read operations will you do, and what is the desired response times for those operations (ex. is 2s tolerable? 500ms? 20ms?) - Scouting Select a set of available options to benchmark using whatever criteria you may see fit, but following the previous step constraints. - Benchmark Create a somewhat simple model that fits your workload and models your real case, using the language you'll be using to build your application; Create benchmark code for the most critical/complex operations; Run it at least 5 times on similar hardware of what you expect production to be (this is important; eg. storage (both memory and disk) latency varies wildly between "my laptop" and a cloud provider) and record the results; Be sure to keep an eye on memory consumption, cpu and disk usage when running the benchmarks; - Decide :) Usual criteria: * Price; * Features (of course); * Easiness of deploy or cloud provider availablity; * Documentation quality; * Driver quality for your application language (it may happen you already excluded a candidate on the previous step due to this); * Query language familiarity; * Performance; * Adoption (translated to: can I expect a new developer to have the basic skills on this tech?);

valyala 6y ago

> Does anyone have any quick checklist or metrics to make decisions like this? When does it make sense to evolve from a vanilla RDB to a timeseries one? 10 daily metrics for 350 entities result in `10 * 350=3.5K` reading per day. This translates to `3.5K * 365=1.3M` readings per year. Such workload can be easily handled by any DBMS out there. There is no need to search for specialized time series database for this workload. The need in specialized TSDB solutions arises when ingestion rate reaches a million of readings per second and the number of readings stored in the database exceeds hundreds of billions and trillions. Such a workload cannot be handled by general-purpose database, but it is easily handled by specialized databases such as VictoriaMetrics. Fun fact that it easily handles 10 trillions (e.g. 10K billions) of readings in a single-node setup - see https://victoriametrics.github.io/CaseStudies.html#wixcom https://victoriametrics.github.io/CaseStudies.html#wixcom

randomacc385937 6y ago
Seriously stop spamming your product in every single one of your messages. Especially when you fail to mention that you have a vested interest