5 ms·

> Ops is maybe trying to help developers and increase security (no login on the boxes needed anymore to look at logs). Can this be handled fine by having tail

by flukus 10y ago

> Ops is maybe trying to help developers and increase security (no login on the boxes needed anymore to look at logs). Can this be handled fine by having tail run as a daemon and forwarding all log entries to somewhere that the developers do have access? > Better workflow for (certain kind of) alerts, which may then get more complicated. This is likely to be highly specific as to what sort of alerts you want, but will tailing the log and running it through an awk script work? I assume you'll have to do something much like this with any alert management tool. Tools like kabana like nice, but this is one of those areas where I think people might be going for the shiny solution (that looks great to management) instead of analyzing what they really need.

chronid 10y ago
You can use whatever system you want (syslog can drop the logs in a single box, no need for horrible tail-based concoctions), as long as you mantain it and it's not Ops responsibility to fix your AWK scripts that look for events in logs from 3 different services for the same request and the same customer (and/or respond to the alerts those script generate at 2am after Bob forgot to update it to parse correctly a new log message - we have our own fires to fight already). I'm not saying one should go straight to ELK. There are other ways, but at the end of the day you are going to implement a similar stack, guaranteed, and you are going to regret using freeform logging instead of a sensible structured format.
- flukus 10y ago
  It sounds like Ops want all the control but to do none of the work? Maintaining scripts like this is there job.
  - chronid 10y ago
    I'm sorry: it's your application, your alerts, your responsibility. Not Ops.
    - flukus 10y ago
      If it's my responsibility then I need access to the systems, otherwise I can't perform excercise that responsibility. How does alert creation and maintenance not fall under ops? They sound like the sort of ops people that devops teams were created to replace.
JdeBP 10y ago
Forwarding, yes. tail, no. tail will lose log data. * http://jdebp.eu./FGA/do-not-use-logrotate.html#Problems http://jdebp.eu./FGA/do-not-use-logrotate.html#Problems

by flukus 10y ago

chronid 10y ago

You can use whatever system you want (syslog can drop the logs in a single box, no need for horrible tail-based concoctions), as long as you mantain it and it's not Ops responsibility to fix your AWK scripts that look for events in logs from 3 different services for the same request and the same customer (and/or respond to the alerts those script generate at 2am after Bob forgot to update it to parse correctly a new log message - we have our own fires to fight already). I'm not saying one should go straight to ELK. There are other ways, but at the end of the day you are going to implement a similar stack, guaranteed, and you are going to regret using freeform logging instead of a sensible structured format.

flukus 10y ago
It sounds like Ops want all the control but to do none of the work? Maintaining scripts like this is there job.
- chronid 10y ago
  I'm sorry: it's your application, your alerts, your responsibility. Not Ops.
  - flukus 10y ago
    If it's my responsibility then I need access to the systems, otherwise I can't perform excercise that responsibility. How does alert creation and maintenance not fall under ops? They sound like the sort of ops people that devops teams were created to replace.

JdeBP 10y ago

Forwarding, yes. tail, no. tail will lose log data. * http://jdebp.eu./FGA/do-not-use-logrotate.html#Problems http://jdebp.eu./FGA/do-not-use-logrotate.html#Problems