3 ms·
I was at a wedding when the question was posted, hence taking forever to reply (apologies!). I am of course biased, I want to say that up front. For Uber we wo
by roskilli 8y ago
I was at a wedding when the question was posted, hence taking forever to reply (apologies!).
I am of course biased, I want to say that up front. For Uber we wouldn’t really want to use Thanos for a few reasons but one primary one is that Thanos is upfront that they don’t really optimize for latency with metrics that don’t currently reside on disk, which can take significantly more time coming from S3 rather being mmapd on a local disk. We have historical metrics needed for anomaly detection (5 weeks of data) queried very frequently and the model of downloading the S3 data for each request or caching it locally (which would fill up the disk of query nodes quite quickly, since we have petabytes of metrics data) doesn’t really scale for us. Also we’re conscious of having to pay all the AWS bandwidth costs considering the dataset is in the petabytes and we run things both on premise and cloud.
Anyway I could definitely talk at length about this, perhaps we should write up something and put it on our wiki.
- wvl0 8y agoThat's a great reply there already. We haven't yet reached a situation where we'd like to continuously query 'old' metrics. Would love to see a nice writeup with any further considerations. Thanks for the reply!