2 ms·
It'd be great to read something about how people are handling logging in production with K8S+ELK for example.
by nikon 9y ago
It'd be great to read something about how people are handling logging in production with K8S+ELK for example.
- mdaniel 9y agoI don't know how much it qualifies as "reading," but we've experienced great success using a DaemonSet of https://github.com/rtoma/logspout-redis-logstash#readme https://github.com/rtoma/logspout-redis-logstash#readme Because Kubernetes is great about applying docker labels, we get the k8s container name, Pod name, Pod namespace, UID, and then the normal docker metadata provided by logspout-redis-logstash. Then use the normal, and essential IMHO, multi-line codec on the logstash side of things: https://www.elastic.co/guide/en/logstash/5.4/plugins-codecs-multiline.html https://www.elastic.co/guide/en/logstash/5.4/plugins-codecs-... We have a few ``if [docker][image] =~ "foo"`` statements to snowflake the types of multiline split patterns, but all in all it just works. The next level up the hierarchy of needs is to also grab the systemd journal content from the Node itself and send that along, too, but it has not yet become a priority. Not to mention the likely substantial increase in store size once the much, much chattier kubelet traffic arrives in ES.