3 ms·

Absolutely nothing likely. Honestly, this is a weird fucking hack. The update went out signed by Solar Winds. Somehow, they got this payload into the build-cha

Absolutely nothing likely. Honestly, this is a weird fucking hack. The update went out signed by Solar Winds. Somehow, they got this payload into the build-chain their developers use.

vsareto 6y ago
Maybe they were just automatically signing it as a a CI/CD step?
- EvanAnderson 6y ago
  I would love to see a postmortem for how they were handling their code signing cert. Right now I get a "PFX file on a network share and everybody knows the password" kind of feeling.
- dane-pgp 6y ago
  Maybe after signing it (but before releasing it), the CI/CD should upload the signed hash to an independent third-party publicly-accessible append-only log (like Certificate Transparency). Assuming the binary is reproducibly buildable, different internal teams can try rebuilding the source and checking that the signature matches what's in the public log, and that each entry in the log has a corresponding commit in the source repo.
rossjudson 6y ago
I'm looking for info on this same topic. If the update was signed, how did that happen? The other thing that occurs to me is that this Solar Winds update goes out to a huge number of organizations, with a tremendous diversity of capabilities. Why was FireEye the first one to notice? Was the Solar Winds code exempted from monitoring, because it is monitoring? How big is this blind spot?
- EvanAnderson 6y ago
  My understanding was that FireEye noticed because this backdoor was used to attack them. They worked it backwards and found the backdoor. It would be interesting to know how many "softer" targets the attacker went after and were completely undetected. It's worth noting that FireEye didn't find the backdoor until after it had been exploited. Nobody can vet modern software. The complexity is too high, let alone accounting for the ridiculous update "velocity" that's coming out of most commercial software today. Just getting software "manufacturers" to itemize and document the normal communication mechanisms used by their software, so that threat hunters aren't reverse engineering communications protocols, would go a long way. If SolarWinds had provided a list of domains their software needed to communicate with that would have gone a long way to stopping this kind of attack. (I can't be the only person who demands vendors provide this info, and who puts third-party software behind MiTM proxies and hard-assed ACLs.)
  - octoberfranklin 6y ago
    New doctrine: update velocity is a vulnerability.
    - EvanAnderson 6y ago
      I've always seen it as a good indicator of unreliability and poor QA practices. "We update our software frequently" has never seemed like a good thing to me. I read it as "You're paying licenses fees to be our testers."
- octoberfranklin 6y ago
  > If the update was signed, how did that happen? https://news.ycombinator.com/item?id=25462894 https://news.ycombinator.com/item?id=25462894 https://news.ycombinator.com/item?id=25463034 https://news.ycombinator.com/item?id=25463034

Absolutely nothing likely. Honestly, this is a weird fucking hack. The update went out signed by Solar Winds. Somehow, they got this payload into the build-chain their developers use.

vsareto 6y ago

Maybe they were just automatically signing it as a a CI/CD step?

EvanAnderson 6y ago
I would love to see a postmortem for how they were handling their code signing cert. Right now I get a "PFX file on a network share and everybody knows the password" kind of feeling.
dane-pgp 6y ago
Maybe after signing it (but before releasing it), the CI/CD should upload the signed hash to an independent third-party publicly-accessible append-only log (like Certificate Transparency). Assuming the binary is reproducibly buildable, different internal teams can try rebuilding the source and checking that the signature matches what's in the public log, and that each entry in the log has a corresponding commit in the source repo.

rossjudson 6y ago

I'm looking for info on this same topic. If the update was signed, how did that happen? The other thing that occurs to me is that this Solar Winds update goes out to a huge number of organizations, with a tremendous diversity of capabilities. Why was FireEye the first one to notice? Was the Solar Winds code exempted from monitoring, because it is monitoring? How big is this blind spot?

EvanAnderson 6y ago
My understanding was that FireEye noticed because this backdoor was used to attack them. They worked it backwards and found the backdoor. It would be interesting to know how many "softer" targets the attacker went after and were completely undetected. It's worth noting that FireEye didn't find the backdoor until after it had been exploited. Nobody can vet modern software. The complexity is too high, let alone accounting for the ridiculous update "velocity" that's coming out of most commercial software today. Just getting software "manufacturers" to itemize and document the normal communication mechanisms used by their software, so that threat hunters aren't reverse engineering communications protocols, would go a long way. If SolarWinds had provided a list of domains their software needed to communicate with that would have gone a long way to stopping this kind of attack. (I can't be the only person who demands vendors provide this info, and who puts third-party software behind MiTM proxies and hard-assed ACLs.)
- octoberfranklin 6y ago
  New doctrine: update velocity is a vulnerability.
  - EvanAnderson 6y ago
    I've always seen it as a good indicator of unreliability and poor QA practices. "We update our software frequently" has never seemed like a good thing to me. I read it as "You're paying licenses fees to be our testers."
octoberfranklin 6y ago
> If the update was signed, how did that happen? https://news.ycombinator.com/item?id=25462894 https://news.ycombinator.com/item?id=25462894 https://news.ycombinator.com/item?id=25463034 https://news.ycombinator.com/item?id=25463034