8 ms·

Show HN: A program to email log files using Go

cejast 8y ago
I thought this was what logrotate was for? https://linux.die.net/man/8/logrotate https://linux.die.net/man/8/logrotate
- submeta 8y ago
  My thought also. - I was wondering why this post made it to the first page of HN. Obviously many fellow HN users find it valuable / do not know that there are solutions to this.
  - dvfjsdhgfv 8y ago
    I'm baffled by this, too. Maybe it's interesting for people who are interested in learning Golang, but I think it's clear it's not the best piece of code to learn from.
    - muzzammildotxyz 8y ago
      Gee, thanks for the boost in confidence. :|
      - dvfjsdhgfv 8y ago
        I think you're self-confident enough.
  - deleted 8y ago
    [deleted]
  - muzzammildotxyz 8y ago
    Or maybe it's just that good :D jk...
- muzzammildotxyz 8y ago
  Just made it to see if I could :)
badrabbit 8y ago
A few concerns I have... 1) The logs are not sent encrypted, this exposes them for every smtp server and mitm party between unencrypted hops 2) you have to save a plain text version of your log mailer's smtp password to disk 3) Monitoring by email sucks and future compromise of any recipients' inbox exposes all historical logs I have written something similar in the past as well and seen email monitoring in real world scenarios. You're not doing it "wrong" per se ,but I think modern protocols allow better solutions. For instance,you can POST the logs to a server(ec2 instance or DO droplet) over TLS, have it generate a link and email that link. You can then control link expiry and encrypt the logs so that they are decrypted in-browser via Webcrypto. You can also do some sort of push monitoring(in addition to sending the link via email) by the server which lets you avoid the whole "email messages can be stuck on an smtp hop for 24-48 hours" and having to monitor for NDR's before resending the logs.
- muzzammildotxyz 8y ago
  Yes, these are valid concerns. 1) I don't know how to encrypt logs and send them. If you can help please open a PR on GitHub or walk me through it :) 2) How else should I store it? Should I encrypt the json file with a master key? 3) This is correct but I just wanted a quick way to do it :) Thanks for the concerns.
hestefisk 8y ago
You can get FreeBSD to email any log periodically using cron. No go program needed. This seems like an absolute overkill but it was probably fun writing the code.
- muzzammildotxyz 8y ago
  Yes, it was a great experience. Just made it to see if I could :)
dewey 8y ago
As this seems to be a beginner project to learn here's some thoughts on the code: 1) You should really not build your json by manually building the string. That's very error prone and hard to extend. fmt.Fprintf(f, "{\n\t\"from\": {\n\t\t\"name\": \"%s\", Use https://golang.org/pkg/encoding/json/ https://golang.org/pkg/encoding/json/ (https://gobyexample.com/json https://gobyexample.com/json) instead. Working with json in Go is really one of the nicer features of Go (It's Go, not "GoLang") 2) By setting / typoing a config option you could accidentally delete files from your computer, even your home directory err := os.Remove(jsonfile)
- jerf 8y ago
  A similar comment for the HTML portions of the message. Use html/template. It's got a couple of quirks (mostly around how it sort of weirdly conflates having one particular template with having a set of templates), but it's one of the safest template languages to just pick up and throw some stuff out; it's quite strong against injection attacks. To assist in using JSON, you'll want to declare a struct holding all your config, at which point it becomes quite surprisingly simple to load it from and save it to a file. Once you have that functionality, I suggest using it for your initial creation as well. I have a program here where I've been using the pattern that I create a default config sample within the program, and if I can't find the config you specify, I print out an error, and the sample JSON file that constitutes a full, legal config for the user's convenience. I wouldn't ship anything that I'd expect to end up in a Linux distro or something that way, but for an internal tool it seems a decent enough pattern. Finally, I'd suggest reddit.com/r/golang is a better sort of place for this sort of thing if you want a review, and there's probably even better places, as I'm not convinced that a karma-based voting site is all that great for reviews for beginners. (It's really easy for a post like that to pick up a couple of early downvotes, and consequently lose all visibility to the people who are willing to help.) But if you're going to use one, /r/golang would be better.
  - muzzammildotxyz 8y ago
    Thanks for the suggestion :D I am not shipping it to any distro or something like that. I was using this for myself and just made it open source. Thank you for taking your time and reviewing this. As for /r/golang, I will look into it. :)
liotier 8y ago
Any reason for not using Logwatch ? https://wiki.archlinux.org/index.php/Logwatch https://wiki.archlinux.org/index.php/Logwatch
- actionscripted 8y ago
  Looks like the author is taking a stab at working with Go and wanted to build something to fill a need. I would absolutely recommend Logwatch or, if you're fancy, Filebeat and Logstash.
  - muzzammildotxyz 8y ago
    actionscripted, you are a wizard! How did you guess that? :D
- muzzammildotxyz 8y ago
  Just made it to see if I could :)
dsr_ 8y ago
It's very nice that you can specify the period between sending logs in nanoseconds, microseconds or milliseconds, but also utterly useless. Even if there's a time library being used that has these options, it's appropriate to only advertise the units that make sense. Sadly, it does not include "days", although one can always specify the value in milliseconds. (The fifth hyperfactorial, as recently mentioned.)
- muzzammildotxyz 8y ago
  Well, you caught me. I copied and pasted that line from Go Source comments :D But hey, you can use 1h4ms6ns as intervals :D Maybe I should parse days... Thanks for the Idea.
heegemcgee 8y ago
Overall, i would say "using email for system tasks considered harmful". I have worked several devops / sysadmin jobs where my inbox took weeks to tame with filters because of rampant abuse of automated system emails. Every alert should be actionable. And email doesn't have good reliability or timeliness - it can take hours for me to get a push notification on my phone that there is email, and in the evenings, i really shouldn't be looking at email at all. So we should be using a proper alert system via SMS (pagerduty is pretty great for this, but i also like twilio, and amazon SNS is just fine too). More germaine to the topic at hand: I'd recommend a) setting up log monitors with Nagios, or Zabbix, or your favorite tool. You want to regex match on certain strings in the log file, like "Deadlock" or "out of memory". Pass that alert on to your monitoring system and get a proper, actionable alert. And b), aggregating the logs. As far as convenient access, i'd recommend Graylog (or ELK or Splunk) if you have more than a handful of nodes. This makes it easy to search through logs or review them without signing into all those nodes. You can also push them over to Amazon Cloudwatch Logs for archival and rudimentary search.
- AnIdiotOnTheNet 8y ago
  > Every alert should be actionable. How do you know the difference between "everything is working properly" and "the logging and/or monitoring has stopped working"?
  - dewey 8y ago
    You instrument your code instead of just logging. To see that your metric export is working you can regularly export a simple value and check for it’s existence.
  - heegemcgee 8y ago
    Who will watch the watchmen, right? :D It's a real concern. Personally, i have a monitoring agent running, and then i have the config management agent (puppet) validate that the monitoring agent is running. And what Dewey said is absolutely right - you can monitor the code / service itself through health checks. In the case of a reports service, perhaps your monitor asks the API for a very small report. Or you could implement a special endpoint / controller that calls on the core code. I recently implemented a monitor that emulates a typical user session, logging in, performing popular tasks, and logging out. If any step in that process has an error, i get an alert with the step listed, and i instantly have some idea of where things are jammed. In this manner, i don't need to have pre-defined log monitors for specific errors; i can catch novel error types by virtue of exercising the code and watching for the expected responses - 200 in the header, ability to perform tasks that are only available on login, checking for certain strings in the response, etc.
noponpop 8y ago
Have you considered syslog, which represents a standard for shipping logs?
- LinuxBender 8y ago
  I would add, that you can extend syslog to use queuing [0], RELP [1] and TLS [2] to protect your data in transit. [0] - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/s1-working_with_queues_in_rsyslog https://access.redhat.com/documentation/en-us/red_hat_enterp... [1] - https://en.wikipedia.org/wiki/Reliable_Event_Logging_Protocol https://en.wikipedia.org/wiki/Reliable_Event_Logging_Protoco... [2] - https://www.rsyslog.com/doc/v8-stable/tutorials/tls.html https://www.rsyslog.com/doc/v8-stable/tutorials/tls.html
- weavie 8y ago
  Syslog (at least on Solaris) seems to have a max length of 1024 characters. Highly annoying and seemingly arbitrary limitation.
  - muzzammildotxyz 8y ago
    Yup
- muzzammildotxyz 8y ago
  what @weavie said. :)

I thought this was what logrotate was for? https://linux.die.net/man/8/logrotate https://linux.die.net/man/8/logrotate

submeta 8y ago
My thought also. - I was wondering why this post made it to the first page of HN. Obviously many fellow HN users find it valuable / do not know that there are solutions to this.
- dvfjsdhgfv 8y ago
  I'm baffled by this, too. Maybe it's interesting for people who are interested in learning Golang, but I think it's clear it's not the best piece of code to learn from.
  - muzzammildotxyz 8y ago
    Gee, thanks for the boost in confidence. :|
    - dvfjsdhgfv 8y ago
      I think you're self-confident enough.
- deleted 8y ago
  [deleted]
- muzzammildotxyz 8y ago
  Or maybe it's just that good :D jk...
muzzammildotxyz 8y ago
Just made it to see if I could :)

A few concerns I have... 1) The logs are not sent encrypted, this exposes them for every smtp server and mitm party between unencrypted hops 2) you have to save a plain text version of your log mailer's smtp password to disk 3) Monitoring by email sucks and future compromise of any recipients' inbox exposes all historical logs I have written something similar in the past as well and seen email monitoring in real world scenarios. You're not doing it "wrong" per se ,but I think modern protocols allow better solutions. For instance,you can POST the logs to a server(ec2 instance or DO droplet) over TLS, have it generate a link and email that link. You can then control link expiry and encrypt the logs so that they are decrypted in-browser via Webcrypto. You can also do some sort of push monitoring(in addition to sending the link via email) by the server which lets you avoid the whole "email messages can be stuck on an smtp hop for 24-48 hours" and having to monitor for NDR's before resending the logs.

muzzammildotxyz 8y ago
Yes, these are valid concerns. 1) I don't know how to encrypt logs and send them. If you can help please open a PR on GitHub or walk me through it :) 2) How else should I store it? Should I encrypt the json file with a master key? 3) This is correct but I just wanted a quick way to do it :) Thanks for the concerns.

hestefisk 8y ago

You can get FreeBSD to email any log periodically using cron. No go program needed. This seems like an absolute overkill but it was probably fun writing the code.

muzzammildotxyz 8y ago
Yes, it was a great experience. Just made it to see if I could :)

dewey 8y ago

As this seems to be a beginner project to learn here's some thoughts on the code: 1) You should really not build your json by manually building the string. That's very error prone and hard to extend. fmt.Fprintf(f, "{\n\t\"from\": {\n\t\t\"name\": \"%s\", Use https://golang.org/pkg/encoding/json/ https://golang.org/pkg/encoding/json/ (https://gobyexample.com/json https://gobyexample.com/json) instead. Working with json in Go is really one of the nicer features of Go (It's Go, not "GoLang") 2) By setting / typoing a config option you could accidentally delete files from your computer, even your home directory err := os.Remove(jsonfile)

jerf 8y ago
A similar comment for the HTML portions of the message. Use html/template. It's got a couple of quirks (mostly around how it sort of weirdly conflates having one particular template with having a set of templates), but it's one of the safest template languages to just pick up and throw some stuff out; it's quite strong against injection attacks. To assist in using JSON, you'll want to declare a struct holding all your config, at which point it becomes quite surprisingly simple to load it from and save it to a file. Once you have that functionality, I suggest using it for your initial creation as well. I have a program here where I've been using the pattern that I create a default config sample within the program, and if I can't find the config you specify, I print out an error, and the sample JSON file that constitutes a full, legal config for the user's convenience. I wouldn't ship anything that I'd expect to end up in a Linux distro or something that way, but for an internal tool it seems a decent enough pattern. Finally, I'd suggest reddit.com/r/golang is a better sort of place for this sort of thing if you want a review, and there's probably even better places, as I'm not convinced that a karma-based voting site is all that great for reviews for beginners. (It's really easy for a post like that to pick up a couple of early downvotes, and consequently lose all visibility to the people who are willing to help.) But if you're going to use one, /r/golang would be better.
- muzzammildotxyz 8y ago
  Thanks for the suggestion :D I am not shipping it to any distro or something like that. I was using this for myself and just made it open source. Thank you for taking your time and reviewing this. As for /r/golang, I will look into it. :)

liotier 8y ago

Any reason for not using Logwatch ? https://wiki.archlinux.org/index.php/Logwatch https://wiki.archlinux.org/index.php/Logwatch

actionscripted 8y ago
Looks like the author is taking a stab at working with Go and wanted to build something to fill a need. I would absolutely recommend Logwatch or, if you're fancy, Filebeat and Logstash.
- muzzammildotxyz 8y ago
  actionscripted, you are a wizard! How did you guess that? :D
muzzammildotxyz 8y ago
Just made it to see if I could :)

dsr_ 8y ago

It's very nice that you can specify the period between sending logs in nanoseconds, microseconds or milliseconds, but also utterly useless. Even if there's a time library being used that has these options, it's appropriate to only advertise the units that make sense. Sadly, it does not include "days", although one can always specify the value in milliseconds. (The fifth hyperfactorial, as recently mentioned.)

muzzammildotxyz 8y ago
Well, you caught me. I copied and pasted that line from Go Source comments :D But hey, you can use 1h4ms6ns as intervals :D Maybe I should parse days... Thanks for the Idea.

heegemcgee 8y ago

Overall, i would say "using email for system tasks considered harmful". I have worked several devops / sysadmin jobs where my inbox took weeks to tame with filters because of rampant abuse of automated system emails. Every alert should be actionable. And email doesn't have good reliability or timeliness - it can take hours for me to get a push notification on my phone that there is email, and in the evenings, i really shouldn't be looking at email at all. So we should be using a proper alert system via SMS (pagerduty is pretty great for this, but i also like twilio, and amazon SNS is just fine too). More germaine to the topic at hand: I'd recommend a) setting up log monitors with Nagios, or Zabbix, or your favorite tool. You want to regex match on certain strings in the log file, like "Deadlock" or "out of memory". Pass that alert on to your monitoring system and get a proper, actionable alert. And b), aggregating the logs. As far as convenient access, i'd recommend Graylog (or ELK or Splunk) if you have more than a handful of nodes. This makes it easy to search through logs or review them without signing into all those nodes. You can also push them over to Amazon Cloudwatch Logs for archival and rudimentary search.

AnIdiotOnTheNet 8y ago
> Every alert should be actionable. How do you know the difference between "everything is working properly" and "the logging and/or monitoring has stopped working"?
- dewey 8y ago
  You instrument your code instead of just logging. To see that your metric export is working you can regularly export a simple value and check for it’s existence.
- heegemcgee 8y ago
  Who will watch the watchmen, right? :D It's a real concern. Personally, i have a monitoring agent running, and then i have the config management agent (puppet) validate that the monitoring agent is running. And what Dewey said is absolutely right - you can monitor the code / service itself through health checks. In the case of a reports service, perhaps your monitor asks the API for a very small report. Or you could implement a special endpoint / controller that calls on the core code. I recently implemented a monitor that emulates a typical user session, logging in, performing popular tasks, and logging out. If any step in that process has an error, i get an alert with the step listed, and i instantly have some idea of where things are jammed. In this manner, i don't need to have pre-defined log monitors for specific errors; i can catch novel error types by virtue of exercising the code and watching for the expected responses - 200 in the header, ability to perform tasks that are only available on login, checking for certain strings in the response, etc.

noponpop 8y ago

Have you considered syslog, which represents a standard for shipping logs?

LinuxBender 8y ago
I would add, that you can extend syslog to use queuing [0], RELP [1] and TLS [2] to protect your data in transit. [0] - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/s1-working_with_queues_in_rsyslog https://access.redhat.com/documentation/en-us/red_hat_enterp... [1] - https://en.wikipedia.org/wiki/Reliable_Event_Logging_Protocol https://en.wikipedia.org/wiki/Reliable_Event_Logging_Protoco... [2] - https://www.rsyslog.com/doc/v8-stable/tutorials/tls.html https://www.rsyslog.com/doc/v8-stable/tutorials/tls.html
weavie 8y ago
Syslog (at least on Solaris) seems to have a max length of 1024 characters. Highly annoying and seemingly arbitrary limitation.
- muzzammildotxyz 8y ago
  Yup
muzzammildotxyz 8y ago
what @weavie said. :)