3 ms·

I use TimescaleDB for mass storage and query of security events (up to 100s of millions) - the speed of queries and aggregate queries even on a single node is v

I use TimescaleDB for mass storage and query of security events (up to 100s of millions) - the speed of queries and aggregate queries even on a single node is very impressive. I haven't done anything with regards to anomaly/trend detection yet, but it's planned. Not really sure where you see a database (TimescaleDB) fitting into that though?

jcims 6y ago
We're in that scale domain where everything is a pain in the ass but not obviously outside the scope of commercial solutions. I just checked and we're averaging ~500k events per second in the five areas I'm interested in. I feel that we could probably use a time-series database to reflect our streams as 'last observed state' type collections as well as do the aggregations that we need to feed back into anomaly detection. I'd like to also use something like that to create a 'heat map service' where you can feed a property/window/range and get back scalar for color coding and possibly a slice of values for sparkline type UI. Without getting hands on, though, it's hard to say for sure.
- akulkarni 6y ago
  @jcims I'm really interested to see if we can help. If you're open to discussing, please feel free to email me: ajay (at) timescale.com
  - jcims 6y ago
    It wouldn't be me reaching out but I'll put a bug in the right person's ear. This has been something I've been thinking about for a bit, the HN post is just a bit serendipitous.
    - akulkarni 6y ago
      Sounds good, thanks!
akulkarni 6y ago
Hi GordonS, would love to hear about your use case if you don't mind sharing! ajay (at) timescale.com