3 ms·
Appreciate the response. I wasn't trying to say this is grep, I fully understand that this is an inverted index which is way more interesting to build on top of
by heipei 5y ago
Appreciate the response. I wasn't trying to say this is grep, I fully understand that this is an inverted index which is way more interesting to build on top of S3.
I merely wanted to say that by using S3 within AWS you always have the fallback option of brute-force "grep" across your semi-structured "data lake" or whatever it's called thanks to the aggregate bandwidth and Athena.
- fulmicoton 5y agoAh my bad! Yes, Humio (and Loki) are opting for this approach. This does decouple compute and storage in a trivial manner. There is indeed a realm in which this brute force approach is the best approach. We could probably make a 4D chart with QPS, data size, latency, and retention period and define regions where the elastic/SOLR approach, Humio, and quickwit are the most relevant.