3 ms·

Back when I ran a small hosting provider, I'd monitor outbound SSH connections and page someone if there were too many from any individual host. On a couple oc

Back when I ran a small hosting provider, I'd monitor outbound SSH connections and page someone if there were too many from any individual host. On a couple occasions we had to increase the threshold for certain users, but most of the time it was an immediate indication that a host had gotten compromised. Super effective.

BLKNSLVR 2y ago
What were your characteristics to define an SSH connection? Port only, or protocol(s), or traffic pattern matching? (I want know more!) I have flags on my firewall(s) for outbound connections to a list of ports that are commonly attacked, so I get notified if it looks as if attacks may be originating from my networks. It's purely based on port numbers though, nothing smarter than that.
- linsomniac 2y ago
  It was based on outbound connections per local IP address, but I don't remember exactly how I achieved it. It might have been port 22 SYN, or it might have been using the conntrack module. IIRC it used the rate limiting module to set the rate, and then just the log module to log a message which the monitoring system would pick up. My routers were Linux boxes running in an HA configuration (heartbeat), with quagga for BGP to the upstreams. I had more cores than interfaces so the system would never livelock due to interrupt storm if a burst of packets came through (interrupt mitigation wouldn't happen if all cores are saturated handling interrupts). Firewalling was kept pretty simple, to reduce load and end boxes all had their normal firewalls, but I did block users sending IP source addresses that they weren't assigned. I also had a custom kernel module that would account traffic on the transit interfaces, so customers were only charged for traffic outside our network, anything internal (backups, apt/yum updates to our local mirror, etc) were all free. I loved having a router that had more than enough memory to hold full BGP tables no problem (while others were worried about the growth of the tables blowing out their RAM), and I liked that my slow path and fast path were the same speed. :-)

BLKNSLVR 2y ago

What were your characteristics to define an SSH connection? Port only, or protocol(s), or traffic pattern matching? (I want know more!) I have flags on my firewall(s) for outbound connections to a list of ports that are commonly attacked, so I get notified if it looks as if attacks may be originating from my networks. It's purely based on port numbers though, nothing smarter than that.

linsomniac 2y ago
It was based on outbound connections per local IP address, but I don't remember exactly how I achieved it. It might have been port 22 SYN, or it might have been using the conntrack module. IIRC it used the rate limiting module to set the rate, and then just the log module to log a message which the monitoring system would pick up. My routers were Linux boxes running in an HA configuration (heartbeat), with quagga for BGP to the upstreams. I had more cores than interfaces so the system would never livelock due to interrupt storm if a burst of packets came through (interrupt mitigation wouldn't happen if all cores are saturated handling interrupts). Firewalling was kept pretty simple, to reduce load and end boxes all had their normal firewalls, but I did block users sending IP source addresses that they weren't assigned. I also had a custom kernel module that would account traffic on the transit interfaces, so customers were only charged for traffic outside our network, anything internal (backups, apt/yum updates to our local mirror, etc) were all free. I loved having a router that had more than enough memory to hold full BGP tables no problem (while others were worried about the growth of the tables blowing out their RAM), and I liked that my slow path and fast path were the same speed. :-)