29 ms·

Whatever solution you use to store your logs, I would suggest to generate them as events. This will help you to reconcile two important aspects that have been s

Whatever solution you use to store your logs, I would suggest to generate them as events. This will help you to reconcile two important aspects that have been separated for too long, with no real reason : logging and analytics. It may require a little bit more effort but I believe it's worth it. I've expanded on this idea here [1] [1] - https://github.com/acionescu/event-bus#why https://github.com/acionescu/event-bus#why

shubhamjain 10y ago
Interesting! this is something that I have thought over often. My own experience of log aggregation is limited to ELK stack and Loggly (for a brief time) where the setup worked fine but the workflow didn't. We just stopped browsing logs after a while. Although, a giant centralised system for logs sounds incredible convenient, making sense of them starts to become a huge problem and then, it's just easier to ignore log system. I am sure the solutions discussed here have features to overcome this (filters / alerts) but IMHO, we'd be better of collecting less things — limited app events that have fixed formatting and are easier to make use of in debugging and monitoring.
- zer0gravity 10y ago
  If you come to think at it, logs are app events. You actually want to collect as many events as you can, with respect to performance, and analyze the hell out of them. You would also get all sorts of benefits, because then you can corelate the app events with user events, and it may be easier to track bugs and unintended behaviour.
balamaci 10y ago
I've found https://github.com/logstash/logstash-logback-encoder https://github.com/logstash/logstash-logback-encoder pretty sufficient for this as it helps generate json logs with extra custom fields based on SFL4J log statements. You can use it's layout to send the json formatted logs to other endpoints not just logback. It's actually being used like that to send to kafka in https://github.com/danielwegener/logback-kafka-appender https://github.com/danielwegener/logback-kafka-appender

shubhamjain 10y ago

Interesting! this is something that I have thought over often. My own experience of log aggregation is limited to ELK stack and Loggly (for a brief time) where the setup worked fine but the workflow didn't. We just stopped browsing logs after a while. Although, a giant centralised system for logs sounds incredible convenient, making sense of them starts to become a huge problem and then, it's just easier to ignore log system. I am sure the solutions discussed here have features to overcome this (filters / alerts) but IMHO, we'd be better of collecting less things — limited app events that have fixed formatting and are easier to make use of in debugging and monitoring.

zer0gravity 10y ago
If you come to think at it, logs are app events. You actually want to collect as many events as you can, with respect to performance, and analyze the hell out of them. You would also get all sorts of benefits, because then you can corelate the app events with user events, and it may be easier to track bugs and unintended behaviour.

balamaci 10y ago

I've found https://github.com/logstash/logstash-logback-encoder https://github.com/logstash/logstash-logback-encoder pretty sufficient for this as it helps generate json logs with extra custom fields based on SFL4J log statements. You can use it's layout to send the json formatted logs to other endpoints not just logback. It's actually being used like that to send to kafka in https://github.com/danielwegener/logback-kafka-appender https://github.com/danielwegener/logback-kafka-appender