4 ms·

I'm still seeing the same number of attempts on my public SFTP servers. As a funny side note I found that by going through the hardening steps on ssh-audit [1]

I'm still seeing the same number of attempts on my public SFTP servers. As a funny side note I found that by going through the hardening steps on ssh-audit [1], most of the bots can't even negotiate a connection. I only see them because I configured verbose logging. They seem to be using really old ssh libraries in the bot code that severely limit the ciphers available to them. Another interesting side effect is that my ssh negotiation is much faster on high latency connections after the hardening. [Edit]: I forgot to add that if testing the ssh hardening from that site, be sure to do this locally where one has console access and the same version of sshd or if on remote nodes make sure that remote consoles are working first to avoid being locked out of the remote nodes. Modifying the client ciphers first and testing remote connections prior to modifying the servers is the lowest risk. [1] - https://www.ssh-audit.com/ https://www.ssh-audit.com/

mmsnberbar66 4y ago
> Another interesting side effect is that my ssh negotiation is much faster on high latency connections after the hardening. why?
- LinuxBender 4y ago
  Watching the connection in verbose it appears the cipher list from both the client and server side being significantly shorter and matching on both sides speeds up the negotiation. There may also be other factors but that is what sticks out to me. I've not dug deep enough into this to see if utilizing different ciphers is also part of the speed-up.
  - chickin 4y ago
    Initially it sounded implausible that a small size increase could noticably raise connection establishment, but it's possible the small size increase causes MTU to be exceeded, and thus an additional roundtrip.
  - newaccount74 4y ago
    RFC 4253 says about the key exchange: > Each side MAY guess which algorithm the other side is using, and MAY send an initial key exchange packet according to the algorithm, if appropriate for the preferred method. If you only support one method, that guess will always be correct, and you save a round trip.
    - deleted 4y ago
      [deleted]
- loeg 4y ago
  Fewer roundtrips to exchange less data, multiplied by the latency per round trip => lower total time.
- mkj 4y ago
  Is it a slow bandwidth connection as well as low latency? It could be because curve25519/ed25519 are smaller packet sizes than other algorithms (and also just the length of the list of algorithms in the kexinit packet). I would have expected the round trip count at the SSH level to be the same, but there could be more TCP round trips for window adjustment. Plugging my own software, Dropbear's dbclient is a bit quicker at high latency - "ssh git@github.com" is 3 seconds (from Perth, Australia) vs 1.5 secs with dbclient. It makes use of the "guess kex" feature that most clients don't, and sends other packets sooner. Of course dbclient doesn't have all the useful features of OpenSSH, and isn't as performant at high bandwidth.
nousermane 4y ago
> [after] hardening steps [...] most of the bots can't even negotiate a connection Yep, same here, except I'm using [tinyssh], which organically does not support password-based auth, or any ciphers other than ssh-ed25519, curve25519-sha256, and chacha20-poly1305@openssh.com. [tinyssh] https://tinyssh.org/ https://tinyssh.org/
- vlan0 4y ago
  >which organically does not support password-based auth This is the way. Everything else is just gravy imo.
- Panino 4y ago
  I agree with your post and just want to add that TinySSH also supports sntrup761x25519-sha512@openssh.com for post-quantum security.
password4321 4y ago
Filtering by client works wonders in servers that support it. Nobody recompiles libssh.
technonerd 4y ago
I found disabling aes ciphers alone killed 90% of the bots with an occasional ECC only bot sneaking through. I only have 2-3 options for MAC, KEX and cipher. Certain mobile clients won't work (older embedded ssh library) but others support more modern configs. But I started my journey with [1] and have slowly tweaked and slimmed it down over the years. https://stribika.github.io/2015/01/04/secure-secure-shell.html https://stribika.github.io/2015/01/04/secure-secure-shell.ht...
jessaustin 4y ago
...be sure to do this locally where one has console access... I found I was able to restart sshd without bouncing my ssh connection. I think this is because sshd forks off children? Of course I made sure everything was working before killing the existing connection! Thanks for the link to the audit service! We went from "F" to "A+"...
nibbleshifter 4y ago
You are correct about the libraries not supporting modern ssh :) I ran a bunch of honeypots for a while and grabbed hassh (like ja3 but for ssh) signatures, most bots are using old fucking libssh/libssh2/paramiko that simply can't talk to modern hosts.
gerdesj 4y ago
Same for www. Ratchet up the TLS/SSL - https://ssl-config.mozilla.org/ https://ssl-config.mozilla.org/ - go for modern and you'll see a lot of failed connections from bots and scanners. Also, if you don't use any other IP block list, do use DROP from Spamhaus: https://www.spamhaus.org/drop/ https://www.spamhaus.org/drop/ - that is small enough that you can run it on the webserver if you don't have much control over your connection to the outside world.
- LinuxBender 4y ago
  I also do that and even take it a step further for my silly hobby sites. I only accept HTTP/2.0 which blocks 99% of the bots and most search engines. Bing is the only search engine that appears to support H2 in their crawler bots.
  - gerdesj 4y ago
    It is a violation of the old mantra exemplified by Sendmail - "be kind in what you accept but be strict in what you send" (or words to that effect). It also looks a bit like a nod to security through obscurity. However the world is a different place nowadays and rather nasty! I think we should insist on the best standards we have and not simply allow old stuff to carry on working through some form of misplaced altruism. I have come across some criticism of h2 but I don't know enough about the context and haven't researched that so can't comment but if dumping h1 works for you then I say crack on. I quietly enjoy watching my HA Proxy logs live and watching things bounce off. I have had an on prem MS Exchange (currently 2016) system that passes PCI-DSS for several years. Quite a few URLs will run really slowly from the outside world and then fail to work. Password testing on the OWA will quietly get handed off to a fake page that not only never lets you login, but gradually gets slower and slower and uses very few actual resources. The real OWA page is firing up all sorts of things in anticipation of you authenticating but the HA Proxy fake login is a C process and tiny. You call your sites "silly hobby" but they are still interweb facing and if you don't protect them they will cease to be just yours and instead be part of a botnet that's trying to hack me and my customers or stealing your electricity to run cryptocoin generation or used as part of the war against Ukraine. There is a good chance that a determined and skilled haxxor will run rings around both of us but let's crack on and make ourselves a bit trickier to crack than the average. If we make ourselves tough enough to crack then unless you are a valuable target then they will pass eventually but you will have slowed them down a bit.
MrRadar 4y ago
I tried that tool against my Raspberry Pi which should have a reasonably-secure SSH configuration and I got a lot of warnings about: > NIST P-curves are possibly back-doored by the U.S. National Security Agency Is there any evidence of this? This seems extremely paranoid.
- LinuxBender 4y ago
  Is there any evidence of this? This seems extremely paranoid. I have not seen any evidence, just discussions. [1][2] [1] - https://crypto.stackexchange.com/questions/10263/should-we-trust-the-nist-recommended-ecc-parameters https://crypto.stackexchange.com/questions/10263/should-we-t... [2] - https://arstechnica.com/information-technology/2015/01/nsa-official-support-of-backdoored-dual_ec_drbg-was-regrettable/ https://arstechnica.com/information-technology/2015/01/nsa-o...
- _jal 4y ago
  No smoking gun. But beyond Snowden's allegations, (a) the NSA has a history of very similar shenanigans and (b) there are readily available alternatives, so why risk it?
sillystuff 4y ago
I'm hoping that the folks running the bots use their bots' failure to negotiate a connection as a filter in a similar, but opposite, way that Nigerian scammers use ridiculously scammy, all caps, emails as a filter. It would significantly reduce log spam if each bot only ever connected once. Anyone who responds to a Nigerian scam email is much more likely to be able to be successfully scammed. It is more efficient, for the scammer, that less credulous people get filtered out in the first step, so as not to waste time on them. Similarly, any site where an ssh bot fails connection because the site operator removed e.g., weaker default ciphers is much less likely to support password auth for it to even be possible to have a password brute forced. It is more efficient for the bot operator to just move along to a more likely target.

mmsnberbar66 4y ago

> Another interesting side effect is that my ssh negotiation is much faster on high latency connections after the hardening. why?

LinuxBender 4y ago
Watching the connection in verbose it appears the cipher list from both the client and server side being significantly shorter and matching on both sides speeds up the negotiation. There may also be other factors but that is what sticks out to me. I've not dug deep enough into this to see if utilizing different ciphers is also part of the speed-up.
- chickin 4y ago
  Initially it sounded implausible that a small size increase could noticably raise connection establishment, but it's possible the small size increase causes MTU to be exceeded, and thus an additional roundtrip.
- newaccount74 4y ago
  RFC 4253 says about the key exchange: > Each side MAY guess which algorithm the other side is using, and MAY send an initial key exchange packet according to the algorithm, if appropriate for the preferred method. If you only support one method, that guess will always be correct, and you save a round trip.
  - deleted 4y ago
    [deleted]
loeg 4y ago
Fewer roundtrips to exchange less data, multiplied by the latency per round trip => lower total time.
mkj 4y ago
Is it a slow bandwidth connection as well as low latency? It could be because curve25519/ed25519 are smaller packet sizes than other algorithms (and also just the length of the list of algorithms in the kexinit packet). I would have expected the round trip count at the SSH level to be the same, but there could be more TCP round trips for window adjustment. Plugging my own software, Dropbear's dbclient is a bit quicker at high latency - "ssh git@github.com" is 3 seconds (from Perth, Australia) vs 1.5 secs with dbclient. It makes use of the "guess kex" feature that most clients don't, and sends other packets sooner. Of course dbclient doesn't have all the useful features of OpenSSH, and isn't as performant at high bandwidth.

nousermane 4y ago

> [after] hardening steps [...] most of the bots can't even negotiate a connection Yep, same here, except I'm using [tinyssh], which organically does not support password-based auth, or any ciphers other than ssh-ed25519, curve25519-sha256, and chacha20-poly1305@openssh.com. [tinyssh] https://tinyssh.org/ https://tinyssh.org/

vlan0 4y ago
>which organically does not support password-based auth This is the way. Everything else is just gravy imo.
Panino 4y ago
I agree with your post and just want to add that TinySSH also supports sntrup761x25519-sha512@openssh.com for post-quantum security.

password4321 4y ago

Filtering by client works wonders in servers that support it. Nobody recompiles libssh.

technonerd 4y ago

I found disabling aes ciphers alone killed 90% of the bots with an occasional ECC only bot sneaking through. I only have 2-3 options for MAC, KEX and cipher. Certain mobile clients won't work (older embedded ssh library) but others support more modern configs. But I started my journey with [1] and have slowly tweaked and slimmed it down over the years. https://stribika.github.io/2015/01/04/secure-secure-shell.html https://stribika.github.io/2015/01/04/secure-secure-shell.ht...

jessaustin 4y ago

...be sure to do this locally where one has console access... I found I was able to restart sshd without bouncing my ssh connection. I think this is because sshd forks off children? Of course I made sure everything was working before killing the existing connection! Thanks for the link to the audit service! We went from "F" to "A+"...

nibbleshifter 4y ago

You are correct about the libraries not supporting modern ssh :) I ran a bunch of honeypots for a while and grabbed hassh (like ja3 but for ssh) signatures, most bots are using old fucking libssh/libssh2/paramiko that simply can't talk to modern hosts.

gerdesj 4y ago

Same for www. Ratchet up the TLS/SSL - https://ssl-config.mozilla.org/ https://ssl-config.mozilla.org/ - go for modern and you'll see a lot of failed connections from bots and scanners. Also, if you don't use any other IP block list, do use DROP from Spamhaus: https://www.spamhaus.org/drop/ https://www.spamhaus.org/drop/ - that is small enough that you can run it on the webserver if you don't have much control over your connection to the outside world.

LinuxBender 4y ago
I also do that and even take it a step further for my silly hobby sites. I only accept HTTP/2.0 which blocks 99% of the bots and most search engines. Bing is the only search engine that appears to support H2 in their crawler bots.
- gerdesj 4y ago
  It is a violation of the old mantra exemplified by Sendmail - "be kind in what you accept but be strict in what you send" (or words to that effect). It also looks a bit like a nod to security through obscurity. However the world is a different place nowadays and rather nasty! I think we should insist on the best standards we have and not simply allow old stuff to carry on working through some form of misplaced altruism. I have come across some criticism of h2 but I don't know enough about the context and haven't researched that so can't comment but if dumping h1 works for you then I say crack on. I quietly enjoy watching my HA Proxy logs live and watching things bounce off. I have had an on prem MS Exchange (currently 2016) system that passes PCI-DSS for several years. Quite a few URLs will run really slowly from the outside world and then fail to work. Password testing on the OWA will quietly get handed off to a fake page that not only never lets you login, but gradually gets slower and slower and uses very few actual resources. The real OWA page is firing up all sorts of things in anticipation of you authenticating but the HA Proxy fake login is a C process and tiny. You call your sites "silly hobby" but they are still interweb facing and if you don't protect them they will cease to be just yours and instead be part of a botnet that's trying to hack me and my customers or stealing your electricity to run cryptocoin generation or used as part of the war against Ukraine. There is a good chance that a determined and skilled haxxor will run rings around both of us but let's crack on and make ourselves a bit trickier to crack than the average. If we make ourselves tough enough to crack then unless you are a valuable target then they will pass eventually but you will have slowed them down a bit.

MrRadar 4y ago

I tried that tool against my Raspberry Pi which should have a reasonably-secure SSH configuration and I got a lot of warnings about: > NIST P-curves are possibly back-doored by the U.S. National Security Agency Is there any evidence of this? This seems extremely paranoid.

LinuxBender 4y ago
Is there any evidence of this? This seems extremely paranoid. I have not seen any evidence, just discussions. [1][2] [1] - https://crypto.stackexchange.com/questions/10263/should-we-trust-the-nist-recommended-ecc-parameters https://crypto.stackexchange.com/questions/10263/should-we-t... [2] - https://arstechnica.com/information-technology/2015/01/nsa-official-support-of-backdoored-dual_ec_drbg-was-regrettable/ https://arstechnica.com/information-technology/2015/01/nsa-o...
_jal 4y ago
No smoking gun. But beyond Snowden's allegations, (a) the NSA has a history of very similar shenanigans and (b) there are readily available alternatives, so why risk it?

sillystuff 4y ago

I'm hoping that the folks running the bots use their bots' failure to negotiate a connection as a filter in a similar, but opposite, way that Nigerian scammers use ridiculously scammy, all caps, emails as a filter. It would significantly reduce log spam if each bot only ever connected once. Anyone who responds to a Nigerian scam email is much more likely to be able to be successfully scammed. It is more efficient, for the scammer, that less credulous people get filtered out in the first step, so as not to waste time on them. Similarly, any site where an ssh bot fails connection because the site operator removed e.g., weaker default ciphers is much less likely to support password auth for it to even be possible to have a password brute forced. It is more efficient for the bot operator to just move along to a more likely target.