2 ms·

From what I understand, it wasn't even the performance testing itself that caught the backdoor. The developer wanted to have the machine as quiescent as possibl

by cesarb 3y ago

From what I understand, it wasn't even the performance testing itself that caught the backdoor. The developer wanted to have the machine as quiescent as possible (so that nothing else running on it would interfere) before starting the performance tests, but sshd was using much more CPU than expected (and this could be observed with simple tools like "top"). My guess is that the usual "backscatter" of password guessing ssh login attempts from all over the Internet normally uses very little CPU time in sshd before being rejected, but the backdoor made each login attempt use a significant amount of CPU time to check whether it was an encrypted request from the attacker (and this particular machine had its sshd open to the Internet because it was a cloud machine being accessed via ssh through the open Internet).

anarazel 3y ago
Nearly spot on. I indeed was seeing sshd usage via top. > but the backdoor made each login attempt use a significant amount of CPU time to check whether it was an encrypted request from the attacker Absurdly enough, the high cpu usage is well before it even tries to figure that out. Due to avoiding "suspicious" strings in both the binary and memory, it has a more expensive string matching routine. Finding all the symbols it from the in-memory symbol tables ends up slow due to that. That's why even sshd -h (i.e. printing help) is slow when started in the right environment. There's not enough visibility into other things that early during process startup (this happens long before main() is called), so they couldn't check what key is being presented or such. They really "should" have deferred much more of their initialization until after the ed448 check happened. > (and this particular machine had its sshd open to the Internet because it was a cloud machine being accessed via ssh through the open Internet) Unfortunately not. It's a machine I have at home, with some port of my public IP redirected to it, as I often need it when not at home. Oddly enough, my threat model did not include getting attacked with something as sophisticated as this, so I thought that was fine (only pubkey, no root).
- cesarb 3y ago
  > Unfortunately not. It's a machine I have at home, with some port of my public IP redirected to it, as I often need it when not at home. Wow, that must have really sucked. It's one thing to have a rented office downtown (a VPS or similar) be backdoored, but to find a burglar broke into your home while it was under construction (or renovation in case it was a distro upgrade) and added a secret passage to it from the outside? What else did the burglar mess with while you weren't looking? > Oddly enough, my threat model did not include getting attacked with something as sophisticated as this, so I thought that was fine (only pubkey, no root). Most people's threat model is "everything except sshd is risky; openssh is from these cool paranoid people at OpenBSD and, as long as nobody can password guess, it's safe from non-authenticated attackers". That is, often sshd is the only open port (which helps explain why the attacker fixated on it).

by cesarb 3y ago

anarazel 3y ago

Nearly spot on. I indeed was seeing sshd usage via top. > but the backdoor made each login attempt use a significant amount of CPU time to check whether it was an encrypted request from the attacker Absurdly enough, the high cpu usage is well before it even tries to figure that out. Due to avoiding "suspicious" strings in both the binary and memory, it has a more expensive string matching routine. Finding all the symbols it from the in-memory symbol tables ends up slow due to that. That's why even sshd -h (i.e. printing help) is slow when started in the right environment. There's not enough visibility into other things that early during process startup (this happens long before main() is called), so they couldn't check what key is being presented or such. They really "should" have deferred much more of their initialization until after the ed448 check happened. > (and this particular machine had its sshd open to the Internet because it was a cloud machine being accessed via ssh through the open Internet) Unfortunately not. It's a machine I have at home, with some port of my public IP redirected to it, as I often need it when not at home. Oddly enough, my threat model did not include getting attacked with something as sophisticated as this, so I thought that was fine (only pubkey, no root).

cesarb 3y ago
> Unfortunately not. It's a machine I have at home, with some port of my public IP redirected to it, as I often need it when not at home. Wow, that must have really sucked. It's one thing to have a rented office downtown (a VPS or similar) be backdoored, but to find a burglar broke into your home while it was under construction (or renovation in case it was a distro upgrade) and added a secret passage to it from the outside? What else did the burglar mess with while you weren't looking? > Oddly enough, my threat model did not include getting attacked with something as sophisticated as this, so I thought that was fine (only pubkey, no root). Most people's threat model is "everything except sshd is risky; openssh is from these cool paranoid people at OpenBSD and, as long as nobody can password guess, it's safe from non-authenticated attackers". That is, often sshd is the only open port (which helps explain why the attacker fixated on it).