djk447

12 ms·

31.
by djk447 5y ago
Thank you! Huge shout out to Shane, Jacob and others on our team who helped with the graphics / design elements!
32.
by djk447 5y ago
I think this would be a tumbling window rather than a true "rolling" tdigest. I suppose you could decrement the buckets, but it gets a little weird as splits can't really be unsplit. The tumbling window one would probably wor
33.
by djk447 5y ago
NB: Post author here. This is interesting and I totally get at least some of the problems you're facing. I wonder if you could take some of the strategies from t-digest and modify a bit to accomplish...I'd be interested in seeing
34.
by djk447 5y ago
NB: Post author here. Yep! Briefly noted that in the post, but deserves re-stating! it's definitely a more complex analysis to figure out the percentage of users affected (though often more important) could be majority could also be on
35.
by djk447 5y ago
Fixed!
36.
by djk447 5y ago
NB: Post author here. Oops, yep, that should probably be order from smallest to largest. Thanks for the correction!
37.
by djk447 5y ago
NB: Post author here. Love this example. Might have to use that in a future post. Feel like a lot of us are running into a similar thing with remote work and video calls these days...
38.
by djk447 5y ago
Totally true...thoughts on how I could rephrase? I guess it's more the "weight" of points greater than and less than the median should be the same, so symmetric distributions definitely have it, asymmetric may or may not. Def
39.
by djk447 5y ago
NB: Post author here. We've been meaning to add IQR as an accessor function for these, may have to go back and do it...the frequency trails [1] stuff from Brendan Gregg also goes into some of this and it's really cool as a visuali
40.
by djk447 5y ago
NB: Post author here. This is great! So fun...will have to use in the future...
41.
by djk447 5y ago
NB: Post author here. Yeah, that was one of the reasons we chose it as one of the ones to implement, seemed like that was a really interesting tradeoff, we also used uddsketch[1] which provides relative error guarantees, which is pretty nif
42.
by djk447 5y ago
NB: Post author here. Totally can be true. In our case, we use these approximation methods that allow you to get multiple percentiles "for free" definitely need to choose the right ones for the job. (We talk a bit more about the w
43.
by djk447 5y ago
NB: Post author here. I found it surprisingly difficult to explain well. Took a lot of passes and a lot more words than I was expecting. It seems like such a simple concept. I thought the post was gonna be the shortest of my recent ones, an
44.
by djk447 5y ago
NB: Post author here. Thanks for sharing! Hadn't heard of that algorithm, have seen a number of other ones out there, we chose a couple that we knew about / were requested by users. (And we are open to more user requests if folks
45.
by djk447 5y ago
NB: Post author here. That's really nifty, wish I'd heard about it earlier. Might go back and add a link to it in the post at some point too! Very useful. Definitely know I wasn't breaking new ground or anything, but fun to s
46.
by djk447 5y ago
NB: Post author here. Std deviation definitely helps a lot, still often not as good as percentiles, was actually thinking about adding some of that in the post but it was already getting so long. It's funny how things you think are sim
47.
by djk447 5y ago
totally. that blog was also one of the sources I mentioned in the post! Good stuff NB: Post author here.
48.
by djk447 5y ago
Under the hood we use the uddsketch percentile approximation algorithm [1], which is able to both be combined and also provides relative error guarantees. I'll also be writing a post about our percentile approximation stuff in the futu
49.
by djk447 5y ago
Hadn't heard of this, but really interesting, thanks for posting! Will have to see if we can use that somehow... (NB: Post author here.)
50.
by djk447 5y ago
Absolutely! We're actually developing a lot of that: https://github.com/timescale/timescaledb-toolkit/tree/main/d... A number of the things you're looking for we've done experimentally and
51.
by djk447 5y ago
(NB: Post author here) Glad you liked the GIFs! Will have to take a look at this. Thanks for sharing!
52.
by djk447 5y ago
Yes! They do fit together very well. There are plans to do that, I'm not sure the exact timeline, I think it's in the design phase now. We do already support partitionwise aggregation, and this meshes very well with that, so that
53.
by djk447 5y ago
(NB: Post author here) Yes! You can use any of the hyperfunction aggregates to build a continuous agg. Including sketches like the percentile approximation stuff. With hyperloglog, it's still experimental, so you have to jump through s
54.
by djk447 5y ago
Same here. I feel like I built several years of my career on reading that documentation and knowing where to find things when other folks didn't, so they'd come and ask me and I'd just send them to the right place.
55.
by djk447 5y ago
NB: Article author here. Interesting idea with Simpson's rule! Perhaps we'll add another method... The other approach is very similar to the LOCF (last observation carried forward) approach, except that it uses the distance to the
56.
by djk447 5y ago
NB: I'm the author of the post :) That's a nifty way of doing things that I didn't know as much about (definitely left out the historians bit as I didn't want to get too into the weeds) and have sometimes encountered the
57.
by djk447 5y ago
Technically, the hyper in hypertable was originally connected to the fact that they hypercubes as part of the partitioning logic: https://github.com/timescale/timescaledb/blob/master/src/hyp... Ove
58.
by djk447 5y ago
NB: Author of post here. Yeah. In general, we didn't think this was horribly ground-breaking work in terms of it being a novel analysis type, the post was meant to explain it in an accessible way for folks who are new to the concept.
59.
by djk447 5y ago
NB: I'm the author of the post, and I work at Timescale. Yep! Though I will say a number of the hyperfunctions are optimized to work on time-series data whether or not there's a hypertable involved, but the name is meant to echo t
60.
by djk447 5y ago
(Disclaimer: I'm an engineer at Timescale, I didn't work directly on this feature but have some knowledge of what it does). I'm not 100% sure I understand exactly what you're asking, but, perhaps some more info will be u

More ›