42 ms·
Did anyone yet use Redis streams to store actual logs? Like server logs, application logs, etc. I understand that Elasticsearch is a common place to put logs,
by skrebbel 8y ago
Did anyone yet use Redis streams to store actual logs? Like server logs, application logs, etc.
I understand that Elasticsearch is a common place to put logs, also because I assume that searching through logs is a common use case, but I wonder whether Redis has particular benefits for this use case. The data structure seems particularly tailored to it (but not so much to searching I guess).
- atombender 8y agoMy guess is that this would work fine until the working set size exceeds available memory. Redis (unless something new has happened the last couple of years since I used it) requires that data fit in RAM. So could work well for low-frequency logging like alerts. Not as a general purpose log system.
- antoncohen 8y agoLog volume can easily exceed reasonable memory sizes. Even a small company can generate TBs of logs each month. Having a single box with TBs of memory wouldn't be desirable. For logs without full indexing, Loki (https://github.com/grafana/loki https://github.com/grafana/loki) is a recent entry into the space, and it probably a good option to look at. It indexes metadata (labels), so it allows searching by labels but not full text. It is also supposed to be horizontally-scalable, which is probably something you want in a log storage solution.