8 ms·

Posting successful SSH logins to Slack

133 points by sandm 11y ago

deleted 11y ago
[deleted]
Kenp77 11y ago
Elegant. Thank you! Is there a way to extend it to override DND?
- sandm 11y ago
  That's a great idea. But I didn't find any docs that explain how to override DND. I opened a feature request instead, so we'll see how it goes :)
  - jerf 11y ago
    That kind of gets you into this sort of problem: https://blogs.msdn.microsoft.com/oldnewthing/20110310-00/?p=11253 https://blogs.msdn.microsoft.com/oldnewthing/20110310-00/?p=... If you start overriding DND, now the user is going to want super-DND. Which somebody will then want to override, and so on. The correct solution is that your users need to not set DND when they in fact need to be disturbed, and your systems shouldn't be disturbing unnecessarily, and to the extent that's a really hard problem, well, yes, it very much is, but an unboundedly-large hierarchy of "bother that person, no don't bother me, SUPER bother that person, no SUPER don't bother me, SUPER MEGA bother that person" isn't part of the solution set.
    - falcolas 11y ago
      The biggest problem with DND in my book is that when first introduced it was enabled automatically, and not super obvious that it was enabled. This caused more than a few missed announcements and made escalation hard for a bit.
- pavel_lishin 11y ago
  Is that a good idea? You don't necessarily want to wake up the entire team with an alert if there's a dedicated on-call person.
tinco 11y ago
Excellent. I've been thinking about having a SSHD keylogger post to slack (or some other log). It's crazy that sshd doesn't have this functionality built-in. It's so important to know what your admins are executing on your machines. Aside from the fact that they might have been compromised, it's just good to know what sort of general administration is being done.
- rogeryu 11y ago
  What about logwatch? That can do the same and a lot more. You can set it to mail you daily, which gives you an overview who logged in and how often. With a weekly mail you don't get these details, but it might be good enough.
- jmiserez 11y ago
  If you don't want to install a "real" solution like Snoopy Logger that works for all users/shells/edge cases, you could always adapt the bash prompt to write the command out. E.g. I personally use something like this in my .bashrc which logs everything per user, but you could easily adapt this to post to Slack instead: # Adapted from https://unix.stackexchange.com/questions/207813/how-to-log-every-command-typed-into-bash-and-every-file-operation export ETERNAL_AUDIT_LOGFILE=~/.bash_eternal_auditlog PROMPT_COMMAND='RET_VAL=$?; history -a; echo "$(who am i | sed -e "s/[[:space:]]\+/ /g") [$$]: $(history 1 | sed "s/^[ ]*[0-9]\+[ ]*//" ) [$RET_VAL]" >> $ETERNAL_AUDIT_LOGFILE' Output including return code and all parameters: ubuntu pts/0 2016-01-22 13:24 (example-loggedinuser-rdns.yourisp.com) [4379]: [2016-01-22 13:25:37] ps aux | grep python [0] If you assume no malicious users this will work just fine.
  - tinco 11y ago
    I've looked into both Snoopy and a bash script as you said. They both have merits, but I think for it to be really reliable there's no better way than to just log keyboard input. Assuming there are malicious users for me is a big part of the motivation.
    - deleted 11y ago
      [deleted]
_yy 11y ago
Is Slack really the right place for security-critical notifications?
- Klathmon 11y ago
  yes? I mean the next step is to have an automated phone call go out to people (which is what we do for critical alerts). Short of that, slack is on my desktop, laptop, and phone. If i don't have one of those around me at the time, you aren't getting ahold of me for any reason. So yeah i think it's perfectly valid for security-critical notifications. Plus this isn't as security critical as you'd think. I don't want klaxons going off every time someone sshs into a server... This can just be an additional layer of security.
  - plasticxme 11y ago
    Slack is terrible for auditing, though. What's wrong with email?
    - RubyPinch 11y ago
      people are more likely to be looking at chat windows as opposed to emails
- Tepix 11y ago
  Not in my book. Slack seems to be really cool but since it's not self-hosted and owned by a US entity, I'll stay clear.
  - pc86 11y ago
    It's an ssh login notification with a user and IP address. It's not notifying everyone what the new launch codes are. Let's not overstate it.
    - deleted 11y ago
      [deleted]
hoorayimhelping 11y ago
Love this, great idea! I've been trying to setup useful Slack integrations lately and this is a really clever use of them.
codercotton 11y ago
srvAudit also does this, though it's still early in development. srvaudit.com
CaptSpify 11y ago
I do the same thing, except I email the logins to myself with SEC: https://simple-evcorr.github.io/ https://simple-evcorr.github.io/
esseti 11y ago
I did a similar thing a couple of days ago. I just added this (with the correct values) in the `sshrc` file inside `/etc/ssh` and enabled a webhook. that's it. ip=`echo $SSH_CONNECTION | cut -d " " -f 1` curl -X POST --data-urlencode 'payload={"channel": "#<your channel>", "username": "SSH Login watcher", "text": "User '${USER}' just logged in from '${ip}'", "icon_emoji": ":robot_face:"}' https://hooks.slack.com/services/<rest https://hooks.slack.com/services/<rest of the webhook>
lazyant 11y ago
If you want to send email on login, add in /etc/profile : echo "`whoami` logged in at `date` from `echo $SSH_CLIENT`" | mail -s "`hostname` login" youremail@example.com Note that people can still ssh execute remotely etc.
- mscman 11y ago
  You'll definitely want to add a '&' at the end of that line so that you don't delay user logins if the network is down or mail barfs.
  - lazyant 11y ago
    right, actually I do have a delay in a server with no mail where it fails :-)
- taylorhughes 11y ago
  pam_exec seems to be a more robust solution to login notifications than /etc/profile http://blog.stalkr.net/2010/11/login-notifications-pamexec-scripting.html http://blog.stalkr.net/2010/11/login-notifications-pamexec-s...
rmdoss 11y ago
I recommend using something like OSSEC to watch your logs and also tie it to Slack/Pagerduty. This post explains how to set it up: https://blog.sucuri.net/2016/01/server-security-integrating-ossec-with-slack-and-pagerduty.html https://blog.sucuri.net/2016/01/server-security-integrating-...
duggan 11y ago
This is neat! Threw together an Ansible role for it: https://github.com/duggan/ansible-slack-notify-ssh https://github.com/duggan/ansible-slack-notify-ssh

133 points by sandm 11y ago

deleted 11y ago

[deleted]

Kenp77 11y ago

Elegant. Thank you! Is there a way to extend it to override DND?

sandm 11y ago
That's a great idea. But I didn't find any docs that explain how to override DND. I opened a feature request instead, so we'll see how it goes :)
- jerf 11y ago
  That kind of gets you into this sort of problem: https://blogs.msdn.microsoft.com/oldnewthing/20110310-00/?p=11253 https://blogs.msdn.microsoft.com/oldnewthing/20110310-00/?p=... If you start overriding DND, now the user is going to want super-DND. Which somebody will then want to override, and so on. The correct solution is that your users need to not set DND when they in fact need to be disturbed, and your systems shouldn't be disturbing unnecessarily, and to the extent that's a really hard problem, well, yes, it very much is, but an unboundedly-large hierarchy of "bother that person, no don't bother me, SUPER bother that person, no SUPER don't bother me, SUPER MEGA bother that person" isn't part of the solution set.
  - falcolas 11y ago
    The biggest problem with DND in my book is that when first introduced it was enabled automatically, and not super obvious that it was enabled. This caused more than a few missed announcements and made escalation hard for a bit.
pavel_lishin 11y ago
Is that a good idea? You don't necessarily want to wake up the entire team with an alert if there's a dedicated on-call person.

tinco 11y ago

Excellent. I've been thinking about having a SSHD keylogger post to slack (or some other log). It's crazy that sshd doesn't have this functionality built-in. It's so important to know what your admins are executing on your machines. Aside from the fact that they might have been compromised, it's just good to know what sort of general administration is being done.

rogeryu 11y ago
What about logwatch? That can do the same and a lot more. You can set it to mail you daily, which gives you an overview who logged in and how often. With a weekly mail you don't get these details, but it might be good enough.
jmiserez 11y ago
If you don't want to install a "real" solution like Snoopy Logger that works for all users/shells/edge cases, you could always adapt the bash prompt to write the command out. E.g. I personally use something like this in my .bashrc which logs everything per user, but you could easily adapt this to post to Slack instead: # Adapted from https://unix.stackexchange.com/questions/207813/how-to-log-every-command-typed-into-bash-and-every-file-operation export ETERNAL_AUDIT_LOGFILE=~/.bash_eternal_auditlog PROMPT_COMMAND='RET_VAL=$?; history -a; echo "$(who am i | sed -e "s/[[:space:]]\+/ /g") [$$]: $(history 1 | sed "s/^[ ]*[0-9]\+[ ]*//" ) [$RET_VAL]" >> $ETERNAL_AUDIT_LOGFILE' Output including return code and all parameters: ubuntu pts/0 2016-01-22 13:24 (example-loggedinuser-rdns.yourisp.com) [4379]: [2016-01-22 13:25:37] ps aux | grep python [0] If you assume no malicious users this will work just fine.
- tinco 11y ago
  I've looked into both Snoopy and a bash script as you said. They both have merits, but I think for it to be really reliable there's no better way than to just log keyboard input. Assuming there are malicious users for me is a big part of the motivation.
  - deleted 11y ago
    [deleted]

_yy 11y ago

Is Slack really the right place for security-critical notifications?

Klathmon 11y ago
yes? I mean the next step is to have an automated phone call go out to people (which is what we do for critical alerts). Short of that, slack is on my desktop, laptop, and phone. If i don't have one of those around me at the time, you aren't getting ahold of me for any reason. So yeah i think it's perfectly valid for security-critical notifications. Plus this isn't as security critical as you'd think. I don't want klaxons going off every time someone sshs into a server... This can just be an additional layer of security.
- plasticxme 11y ago
  Slack is terrible for auditing, though. What's wrong with email?
  - RubyPinch 11y ago
    people are more likely to be looking at chat windows as opposed to emails
Tepix 11y ago
Not in my book. Slack seems to be really cool but since it's not self-hosted and owned by a US entity, I'll stay clear.
- pc86 11y ago
  It's an ssh login notification with a user and IP address. It's not notifying everyone what the new launch codes are. Let's not overstate it.
  - deleted 11y ago
    [deleted]

hoorayimhelping 11y ago

Love this, great idea! I've been trying to setup useful Slack integrations lately and this is a really clever use of them.

codercotton 11y ago

srvAudit also does this, though it's still early in development. srvaudit.com

CaptSpify 11y ago

I do the same thing, except I email the logins to myself with SEC: https://simple-evcorr.github.io/ https://simple-evcorr.github.io/

esseti 11y ago

I did a similar thing a couple of days ago. I just added this (with the correct values) in the `sshrc` file inside `/etc/ssh` and enabled a webhook. that's it. ip=`echo $SSH_CONNECTION | cut -d " " -f 1` curl -X POST --data-urlencode 'payload={"channel": "#<your channel>", "username": "SSH Login watcher", "text": "User '${USER}' just logged in from '${ip}'", "icon_emoji": ":robot_face:"}' https://hooks.slack.com/services/<rest https://hooks.slack.com/services/<rest of the webhook>

lazyant 11y ago

If you want to send email on login, add in /etc/profile : echo "`whoami` logged in at `date` from `echo $SSH_CLIENT`" | mail -s "`hostname` login" youremail@example.com Note that people can still ssh execute remotely etc.

mscman 11y ago
You'll definitely want to add a '&' at the end of that line so that you don't delay user logins if the network is down or mail barfs.
- lazyant 11y ago
  right, actually I do have a delay in a server with no mail where it fails :-)
taylorhughes 11y ago
pam_exec seems to be a more robust solution to login notifications than /etc/profile http://blog.stalkr.net/2010/11/login-notifications-pamexec-scripting.html http://blog.stalkr.net/2010/11/login-notifications-pamexec-s...

rmdoss 11y ago

I recommend using something like OSSEC to watch your logs and also tie it to Slack/Pagerduty. This post explains how to set it up: https://blog.sucuri.net/2016/01/server-security-integrating-ossec-with-slack-and-pagerduty.html https://blog.sucuri.net/2016/01/server-security-integrating-...

duggan 11y ago

This is neat! Threw together an Ansible role for it: https://github.com/duggan/ansible-slack-notify-ssh https://github.com/duggan/ansible-slack-notify-ssh