4 ms·

> More recently, I was part of a discussion where Kubernetes has decided to roll its own logging infrastructure, while leaving important parts like log rotation

by alexk 9y ago

> More recently, I was part of a discussion where Kubernetes has decided to roll its own logging infrastructure, while leaving important parts like log rotation still undecided.... driving me even closer to the Docker ecosystem. Can you please refer to this particular discussion and link to it here, I would like to make sure that this indeed is what happened.

thockingoog 9y ago
I don;t think that is really what happened. :) As part of the runtime abstraction, kubernetes must necessarily decide how to handle stdout/stderr logs. Not having a decision for something at a given point in time is not exactly a bad thing.
- alexk 9y ago
  Thanks, this seemed to be quite unusual decision to be made by the Kubernetes dev community, that's why I've asked.
  - resouer 9y ago
    Hi, I'm from sig-node. We just don't have enough time/people to cover all log management in 1.6, this is just a temp workaronud so we can ship CRI in time. Log mgmt, together with some other missing parts will be definitely picked up after CRI fully released.
    - JdeBP 9y ago
      I recommend that you learn from the world around you. This is a long since addressed problem. This is the way that the daemontools world has been addressing logging since the middle 1990s, and there is a lot of design and implementation experience to draw upon and learn from. The ideas of decoupling logs from lifetimes of the the things being logged, desynchronized log-watcher services that ship logs to central off-machine log stores, logging chains, shims for things that talk to syslog over AF_LOCAL or UDP sockets, shims for things that talk to the systemd journal using its idiosyncratic logging protocol, compartmentalized logging services that are protected by the operating system from compromises to the logged services, and persistent log pipes that do not lose log data; have all been designed and implemented. * http://jdebp.eu./FGA/daemontools-family.html#Logging http://jdebp.eu./FGA/daemontools-family.html#Logging * http://jdebp.eu./FGA/do-not-use-logrotate.html http://jdebp.eu./FGA/do-not-use-logrotate.html * http://jdebp.eu./Softwares/nosh/guide/logging.html http://jdebp.eu./Softwares/nosh/guide/logging.html * http://skarnet.org/software/s6/s6-log.html#diesyslogdiedie http://skarnet.org/software/s6/s6-log.html#diesyslogdiedie * http://skarnet.org/software/s6/s6-log.html#loggingchain http://skarnet.org/software/s6/s6-log.html#loggingchain * http://tinydns.org/#ContributedLogging http://tinydns.org/#ContributedLogging
sandGorgon 9y ago
For independent reasons, I would rather not go into it because the discussion became borderline heated. My perspective was that it was inconsiderate to not give a path for kubernetes logging to work with journald which is now default on Redhat, Ubuntu, Debian and Coreos. I don't say journald should be the only supported format...But it's wasteful to have the pendulum swing the whole other way. The last known proposal is here https://github.com/kubernetes/community/blob/master/contributors/design-proposals/kubelet-cri-logging.md https://github.com/kubernetes/community/blob/master/contribu... You can check the rotation part. Btw, yes I am aware there was increased journald support added in 1.6 . And I'm hoping that more reasonable minds prevail. I get very worried when people say "the format+kubectl cli is more important. we will figure out the mechanics of easy stuff like logrotation later. until then you can use a cronjob. systemd is obviously a monopoly."

by alexk 9y ago

thockingoog 9y ago

I don;t think that is really what happened. :) As part of the runtime abstraction, kubernetes must necessarily decide how to handle stdout/stderr logs. Not having a decision for something at a given point in time is not exactly a bad thing.

alexk 9y ago
Thanks, this seemed to be quite unusual decision to be made by the Kubernetes dev community, that's why I've asked.
- resouer 9y ago
  Hi, I'm from sig-node. We just don't have enough time/people to cover all log management in 1.6, this is just a temp workaronud so we can ship CRI in time. Log mgmt, together with some other missing parts will be definitely picked up after CRI fully released.
  - JdeBP 9y ago
    I recommend that you learn from the world around you. This is a long since addressed problem. This is the way that the daemontools world has been addressing logging since the middle 1990s, and there is a lot of design and implementation experience to draw upon and learn from. The ideas of decoupling logs from lifetimes of the the things being logged, desynchronized log-watcher services that ship logs to central off-machine log stores, logging chains, shims for things that talk to syslog over AF_LOCAL or UDP sockets, shims for things that talk to the systemd journal using its idiosyncratic logging protocol, compartmentalized logging services that are protected by the operating system from compromises to the logged services, and persistent log pipes that do not lose log data; have all been designed and implemented. * http://jdebp.eu./FGA/daemontools-family.html#Logging http://jdebp.eu./FGA/daemontools-family.html#Logging * http://jdebp.eu./FGA/do-not-use-logrotate.html http://jdebp.eu./FGA/do-not-use-logrotate.html * http://jdebp.eu./Softwares/nosh/guide/logging.html http://jdebp.eu./Softwares/nosh/guide/logging.html * http://skarnet.org/software/s6/s6-log.html#diesyslogdiedie http://skarnet.org/software/s6/s6-log.html#diesyslogdiedie * http://skarnet.org/software/s6/s6-log.html#loggingchain http://skarnet.org/software/s6/s6-log.html#loggingchain * http://tinydns.org/#ContributedLogging http://tinydns.org/#ContributedLogging

sandGorgon 9y ago

For independent reasons, I would rather not go into it because the discussion became borderline heated. My perspective was that it was inconsiderate to not give a path for kubernetes logging to work with journald which is now default on Redhat, Ubuntu, Debian and Coreos. I don't say journald should be the only supported format...But it's wasteful to have the pendulum swing the whole other way. The last known proposal is here https://github.com/kubernetes/community/blob/master/contributors/design-proposals/kubelet-cri-logging.md https://github.com/kubernetes/community/blob/master/contribu... You can check the rotation part. Btw, yes I am aware there was increased journald support added in 1.6 . And I'm hoping that more reasonable minds prevail. I get very worried when people say "the format+kubectl cli is more important. we will figure out the mechanics of easy stuff like logrotation later. until then you can use a cronjob. systemd is obviously a monopoly."