syslog / stderr -> Kafka -> centralized ELK Files / log rotation is completely the wrong approach because log entries are mostly innately structured, rich data

syslog / stderr -> Kafka -> centralized ELK Files / log rotation is completely the wrong approach because log entries are mostly innately structured, rich data that occurs at a specific time. Serializing and then parsing log lines again is wasted effort. Messaging is a better fit than lines in files which create log-management headaches like not rotating, losing messaging on rotation, compressing/decompressing and a lengthy soup of destructured data that fills up local disks. Logging to files on local disks is wrong and often creates privacy problems. Logging to cloud services is also expensive, a legal quagmire and raise data portability concerns.