4 ms·

That is a pretty good and detailed blog post. I wonder, however, if I can use any aggregate call to build out a continuous aggregate. I'm specifically think abo

That is a pretty good and detailed blog post. I wonder, however, if I can use any aggregate call to build out a continuous aggregate. I'm specifically think about things like sketches and hyperloglogs.

djk447 5y ago
(NB: Post author here) Yes! You can use any of the hyperfunction aggregates to build a continuous agg. Including sketches like the percentile approximation stuff. With hyperloglog, it's still experimental, so you have to jump through some hoops to get it into a continuous agg as it could be dropped on next release, but once that's been stabilized it'll be very easy to use in continuous aggs. Feel free to test now, just know that it may get dropped and you'd have to re-create the continuous agg after the next release when it gets stabilized.
- sgtfrankieboy 5y ago
  Interesting article! I was wondering if there plans for supporting Continuous Aggregates on Distributed hypertables? Its currently listed as not supported under the limitations docs page. I think the two would fit together perfectly as it will allow users to aggregate very large tables.
  - djk447 5y ago
    Yes! They do fit together very well. There are plans to do that, I'm not sure the exact timeline, I think it's in the design phase now. We do already support partitionwise aggregation, and this meshes very well with that, so that you're sending much, much less data back over the network between the data node and access node when you're trying to do things like percentiles. But I agree that continuous aggregate support will be an even bigger win!
  - mfreed 5y ago
    We plan to release support for continuous aggregates on distributed hypertables in the next month or two. Initially version will support case where the raw data is a distributed hypertable, while the cagg table is a hypertable on the access node. Then follow-up with support where the cagg table can be distributed as well. We are simultaneously working on support for compressed continuous aggregates as well, which we are targeting the same timeframe.

djk447 5y ago

(NB: Post author here) Yes! You can use any of the hyperfunction aggregates to build a continuous agg. Including sketches like the percentile approximation stuff. With hyperloglog, it's still experimental, so you have to jump through some hoops to get it into a continuous agg as it could be dropped on next release, but once that's been stabilized it'll be very easy to use in continuous aggs. Feel free to test now, just know that it may get dropped and you'd have to re-create the continuous agg after the next release when it gets stabilized.

sgtfrankieboy 5y ago
Interesting article! I was wondering if there plans for supporting Continuous Aggregates on Distributed hypertables? Its currently listed as not supported under the limitations docs page. I think the two would fit together perfectly as it will allow users to aggregate very large tables.
- djk447 5y ago
  Yes! They do fit together very well. There are plans to do that, I'm not sure the exact timeline, I think it's in the design phase now. We do already support partitionwise aggregation, and this meshes very well with that, so that you're sending much, much less data back over the network between the data node and access node when you're trying to do things like percentiles. But I agree that continuous aggregate support will be an even bigger win!
- mfreed 5y ago
  We plan to release support for continuous aggregates on distributed hypertables in the next month or two. Initially version will support case where the raw data is a distributed hypertable, while the cagg table is a hypertable on the access node. Then follow-up with support where the cagg table can be distributed as well. We are simultaneously working on support for compressed continuous aggregates as well, which we are targeting the same timeframe.