3 ms·
We haven't reached that stage yet, fortunately; though at some point someone will want to do a big multi-year aggregation report across all indexes and still ex
by lobster_johnson 10y ago
We haven't reached that stage yet, fortunately; though at some point someone will want to do a big multi-year aggregation report across all indexes and still expect it to take not more than a few seconds.
My ideal solution would be one that rotated the dataset into historical rollups on a daily basis, so that we only stored the raw data for today, and gradually merged earlier entries at lower granularities. However, I haven't thought much about how to do that with Elasticsearch. I can see a way of doing it by embedding the value in the field label, and using the field value as a count, but Elasticsearch really doesn't like lots of unique fields; you shouldn't be using more than a few hundred at most in a single installation (across all indexes).