18 ms·
Highly Evasive Attacker Leverages SolarWinds Supply Chain
- virtuallynathan 6y ago"In observed traffic these HTTP response bodies attempt to appear like benign XML related to .NET assemblies, but command data is actually spread across the many GUID and HEX strings present. Commands are extracted from HTTP response bodies by searching for HEX strings using the following regular expression: "\{[0-9a-f-]{36}\}"|"[0-9a-f]{32}"|"[0-9a-f]{16}". Command data is spread across multiple strings that are disguised as GUID and HEX strings. All matched substrings in the response are filtered for non HEX characters, joined together, and HEX-decoded. The first DWORD value shows the actual size of the message, followed immediately with the message, with optional additional junk bytes following. The extracted message is single-byte XOR decoded using the first byte of the message, and this is then DEFLATE decompressed. The first character is an ASCII integer that maps to the JobEngine enum, with optional additional command arguments delimited by space characters." Dang, that's pretty sneaky. This is one heck of a hack.
- markus_zhang 6y agoI wonder how someone manages to figure this out...
- tgsovlerkhgsel 6y agoOnce they find the software, they can reverse-engineer it. Finding it the first time is the difficult part. That can happen based on a tip-off (e.g. every security team in the world will be checking their network for these processes, domain names etc. now), a lucky find (e.g. if the backdoor triggered weird crashes and someone took a look), or because it's linked to some other bad activity (e.g. someone finds some form of known malware and investigates how it got there). In this case, it's likely that they somehow found the backdoor in SolarWinds' network and realized what was happening that way.
- markus_zhang 6y agoThanks. This is voodoo magic to me. I understand the basic of computer programming but this is way more fascinating than the stuffs that I know and touch. I doubt this stuff is taught in schools, maybe the basic like network programming etc., but definitely this is highly sophiscated. Kudos to whoever designed the scheme and who found it!
- nitrogen 6y agoMy CS program included a class that required us to exploit compiled code. Phrack Magazine and loads of other public resources probably have ideas like this for concealing data. Kids I knew in Junior high and high school were writing password stealers for Windows that would just iterate over every HWND (or whatever Windows 9x called handles) looking for inputs of type password, and concealing the app and the results. It doesn't take a great deal of sophistication to come up with some of these things, just a bit of cleverness and exposure to the possibility of cleverness.
- markus_zhang 6y agoThanks for the resources!
- shanusmagnus 6y agoYour phrasing is very wise -- "cleverness and exposure to the possibility of cleverness." I'm going to ruminate on that one.
- sk5t 6y agoOdds are, if you're a programmer, that you'd have come up with a very similar scheme, given knowledge of the kinds of messages the software is expected to send or receive. I.e. leave the envelope plausible-looking and stash the payload in the random-seeming bits.
- markus_zhang 6y agoAlthough not a professional programmer, I do agree with what you said. But the whole scheme also includes execution on other fronts (e.g. how did they plant the payload).
- redis_mlc 6y agoYou know in advance that it's tbe persistent vector you want, because it's on millions of servers, so then you analyze it. And since the vendor's motivation to write an agent is to "get 'er done ASAP", not necessarily securely, voila. See my comment in the other submission thread on another vendor's agents.
- marcan_42 6y agoThis only works for casual untrained observers. A proper infosec analyst looking at this would first wonder why there is HTTP traffic related to .NET assemblies at all (this sounds weird). Then comparing two requests or so would likely show a suspicious pattern of all the hex changing randomly while the rest of the payload is cookie cutter the same. Unless the outer layer is a legitimate service that's actually in use, this kind of thing only fools people who'd dismiss this traffic as "something I don't know about, but which is probably benign". Then there's the whole figuring out what IPs this is going to part, which would raise more alarm bells. Cute, but I think you could do better. Hiding from someone looking at your traffic is very hard. The more important part is how well you hide from dumb automated tools that people rely on for initial detection. Then again, a huge portion of the auditing/"infosec" market nowadays are untrained random people running automated scanners who actually have zero reverse engineering or proper security research experience, so I'm sure it'd work well against those.
- nl 6y ago> A proper infosec analyst looking at this would first wonder why there is HTTP traffic related to .NET assemblies at all (this sounds weird). This isn't accurate. If you look at the Snort rules used to block it[1], it is masquerading as traffic to .solarwinds.com (ie, the vendor) to URLs looking like: swip/upd/SolarWinds.CortexPlugin.Components.xml Unless you knew the software isn't supposed to do that, it isn't suspicious at all. [1] https://github.com/fireeye/sunburst_countermeasures/blob/main/all-snort.rules https://github.com/fireeye/sunburst_countermeasures/blob/mai...
- marcan_42 6y agoHitting a vendor web server on plaintext HTTP? That right there is a massive red flag. If this were legitimate traffic, that'd be enough reason to drop the vendor right then and there. If this were HTTPS then it wouldn't need the obfuscation to pass by undetected. And then there isn't much you could do at that point to find it via traffic analysis, assuming the uncompromised app makes similar HTTPS connections, other than perhaps going deeper into traffic pattern analysis if you're lucky. Once the threat is identified some other way, it might be possible to develop blocking rules that work at the ciphertext layer (e.g. from packet size patterns exhibited only by the backdoor requests).
- simias 6y agoRemember that a group of Minecraft players managed to reverse-engineer the seed of a map based on a single low res screenshot using, among other things, the shape of clouds. Human ingenuity is really impressive sometimes.
- saargrin 6y agoweaponized autism at its best
- matwood 6y agoExactly. Human ingenuity at scale can figure out wild things. I'm reminded of back when I played MMOs. No matter how hard the company tried to 'balance' the characters, it would only take a couple of days before players figured out optimal solutions that the company often didn't take into account.
- anigbrowl 6y agoThis paper sheds some interesting perspective on the development and detection challenges attached to this sort of signalling, although it's not about computing as such. https://www.nature.com/articles/s41598-018-22926-1 https://www.nature.com/articles/s41598-018-22926-1
- wgx 6y agoIIRC the Snowden leaks revealed the NSA's data "exfiltration" practices which used a similar technique.
- baq 6y agowhy quotes around exfiltration? https://en.wikipedia.org/wiki/Exfiltration https://en.wikipedia.org/wiki/Exfiltration
- SECProto 6y agoI'm guessing they used quotes around "exfiltration" because, per your link, the word is used as a near-synonym for data theft.
- dkarl 6y agoMaybe because exfiltration had an established meatspace usage before it was applied to data, and for a lot of people, applying it to data might be an extension (though a straightforward one) of the usage they're accustomed to to. Probably less necessary on HN than it would be in a forum that isn't explicitly technical.
- bayindirh 6y agoThis is number-stations level obfuscation. Wow.
- DoofusOfDeath 6y agoMaybe I'm being dense, but the article left me confused regarding how the "sunburst" backdoor made it into a piece of software signed by the vendor. Can anyone explain?
- jeffmcjunkin 6y agoIt strongly implies that the vendor was thoroughly compromised, in order to insert backdoors into their software (possibly amongst other attacker actions).
- tootie 6y agoYeah, seems like a buried lede. SolarWinds was owned pretty badly and the attack that lead to that isn't described. Once they had access they have free reign to send malware to any one of their clients masquerading as routine patches. Sounds like they went the extra mile to deliver an extremely subtle exploit to avoid detection.
- jiveturkey 6y agoironic that you spelled ‘lede’ correctly but not ‘led’
- dxdm 6y agoAre you the guy doing youtube vids on submarine stuff? I like those, but it's unexpected to find you here, snarking over people's spelling. If you're not the guy, I recommend looking him up if you're interested in this sort of thing, it's a nice way to spend time online. :)
- deleted 6y ago[deleted]
- jiveturkey 6y agoare you the righteous indignation police? sometimes a simple “i found it amusing/ironic/delightful/serendipitous “ comment is just that, you know. i’m not sure why you think my comment was snarky, or holier than thou, or worthy of a criticism. the sub guy was amusing. as my own comment was intended. i’m honored by the comparison.
- numbsafari 6y agoIt sounds like this is how FireEye were compromised last week. Though I don’t see that referenced in their write up, the WSJ article seems to imply that’s the case, and that this has been weaponized against the federal government. Gonna be a busy week for a lot of teams.
- iJohnDoe 6y agoDamn, this seems extremely sophisticated. Someone or a team went through a ton of thought and work on this. It be would interesting to know how it was originally detected/discovered. The knowledge of how it works and the level detailed shared in this particular case is also interesting. It’s almost like they are burning this one on purpose.
- elliekelly 6y agoI’d also love to know how long it took to plan. Was this a years in the making kind of thing? Or just months? Was there a perfect storm that presented the opportunity and so they seized it or was this a concerted and determined effort to infiltrate some specific agency no matter what it took and this happened to be the weak point they could exploit?
- heartbreak 6y agoThe other top post on HN right now is full of commenters making fun of the attribution to “sophisticated” and “with resources of a nation-state.” Oops. As for the write-up itself, I honestly can’t believe they published this much detail. The feds and FireEye must be very concerned that folks are exposed to this right now and those folks don’t know it.
- akhilcacharya 6y ago> “sophisticated” and “with resources of a nation-state.” Oops I've noticed this since the NK hacks in 2014 and it's bothered me. Why is this the reaction even in technologically savvy quarters?
- creato 6y agoThis has several factors that lead people to make such comments: Tech nerds like to describe things as boring, obvious, and easy to do, it makes them feel smart. Certain politics lead people to assume that government is incompetent and therefore obviously it was an unsophisticated attack that breached them. A lot of IT workers think every instance of a hack is an example of underfunding the IT department, or the IT department wastes the budget they do have.
- syshum 6y ago>>Certain politics lead people to assume that government is incompetent I would say history leads people to assume that government is incompetent. We have plenty of evidence on which to form this conclusion >A lot of IT workers think every instance of a hack is an example of underfunding the IT department, Again this is also drawn from a place of experience, many of us have seen first hand organization refusing to put in the money needed to properly secure systems until AFTER their is a compromise, and then the money is only allocate for a few months to resolve the exact compromise that impacted the organization never really changing the security posture of the company / organization. This pattern is repeated over and over and over again
- acdha 6y ago
- jaxxstorm 6y agoNation state attack targeted across multiple companies and federal agencies. Your threat model just got....complicated..
- deleted 6y ago[deleted]
- EvanAnderson 6y agoSolarWinds has a major presence in the "MSP" market with their "n-Able" product. Lots and lots of PCs and servers have the n-Able agent installed. I am aware of MSPs that use their Customer-facing n-Able installations for the management of their internal systems. Credential harvesting potential alone would be massive. If somebody can backdoor one of their products it would stand to reason they can backdoor others too.
- rixed 6y agoI wonder how frequently this agent has been forced upon users by the security team.
- octoberfranklin 6y agoLess often than the Intel Manglement Engine.
- iJohnDoe 6y agoIt’s mind blowing how relevant this comment is. IME needs to go away.
- EvanAnderson 6y agoHad a chance to look at an n-Able / n-Central installation this morning. Turns out that the "Probe" component (installed on machines in remote networks to act as a centralized data collector within that network) does use the affected Orion DLL, but the version appears to be older than the malicious version. If this hadn't been spotted sooner I wonder if the affected component would have ended up being bundled into n-Able and being shipped that way too.
- 1970-01-01 6y agoSigned malware always indicates a very high quality of craftsmanship, but it looks like SUNBURST takes it to another level. The larger story here is how SolarWinds was just the stepping stone. The actual pivot was/is the entire USA.
- jtchang 6y agoHoly this is one hell of a sophisticated attack. Most of the SolarWinds tools run on Windows only. Some of the obfuscation they use reminds me of how complicated PSD is to parse. Finding that you've been infected with this malware should trigger a critical response. I wouldn't even feel safe until I've nuked all the servers and rolled as many credentials as I could. And even then who knows how much data the attacker could have already exfiltrated. Frankly this level of sophistication is terrifying because you really never know for sure how deep they managed to get in.
- arthurcolle 6y agoSorry for the aside, but I really find it truly mind-blowing that such a major, publicly traded company with products that cost hundreds of thousands of dollars a year to use can be a "Windows exclusive" in the realm of network security/technology infrastructure.
- Godel_unicode 6y agoWhy is that so mind blowing? Microsoft has a gigantic market share in basically every vertical. There are tons of software companies who make Windows-only products. The world is bigger than (and in fact is largely not) FAANG and startups.
- arthurcolle 6y agoYeah, I suppose that's true. It just seems like a company that focuses on another company's products seems... unpoetic. Not sure how else to phrase it. ¯\_(ツ)_/¯
- webmaven 6y ago> Yeah, I suppose that's true. It just seems like a company that focuses on another company's products seems... unpoetic. Not sure how else to phrase it. ¯\_(ツ)_/¯ The word you're looking for is probably "prosaic".
- 6y ago
- umanghere 6y agoHere’s a list of SolarWinds’s customers: https://www.solarwinds.com/company/customers https://www.solarwinds.com/company/customers I’m not in favour of having public client lists, especially when you’re a critical software vendor — but this list is just terrifying. There are a lot of big there, and I won’t be surprised to hear of more incidents in the coming days.
- stonecraftwolf 6y ago“ More than 425 of the US Fortune 500 All ten of the top ten US telecommunications companies All five branches of the US Military The US Pentagon, State Department, NASA, NSA, Postal Service, NOAA, Department of Justice, and the Office of the President of the United States All five of the top five US accounting firms” What’s the opposite of security through obscurity?
- Godel_unicode 6y ago> What’s the opposite of security through obscurity? Marketing
- vlovich123 6y agoSoftware monoculture
- boulos 6y agoPwned due to hubris?
- onion2k 6y agoWhat’s the opposite of security through obscurity? Security.
- jimmySixDOF 6y agoEschew Obfuscation
- simtel20 6y agoAny fortune 500 company that's been around for more than a decade probably has one of every enterprise software product running somewhere. When I worked at a big bank, when we acquired any company, large or small, the software stack they used usually just got bottled up where they were, and the client list on the vendor's website just got updated to the new company name. I mean that company list has "smith barney" which doesn't exist anymore.
- Scoundreller 6y agoI wish there was a way for trusted users to undead comments. Anywho: > umanghere 1 minute ago [dead] [–] > Here’s a list of SolarWinds’s customers: > https://www.solarwinds.com/company/customers https://www.solarwinds.com/company/customers > I’m not in favour of having public client lists, especially when you’re a critical software vendor — but this list is just terrified. There are a lot of big there, and I won’t be surprised to hear of more incidents in the coming days.
- coryrc 6y agoThere's a vouch button, if you click on the time.
- dang 6y ago> I wish there was a way for trusted users to undead comments. That's exactly what vouching is for: https://news.ycombinator.com/newsfaq.html#cvouch https://news.ycombinator.com/newsfaq.html#cvouch. coryrc already pointed this out but I want to hammer the point :) (Users have already vouched for the comment so it's no longer dead.)
- onlydeadheroes 6y agoHow did it get dead?
- skinkestek 6y agoSome people "flagged" it. To flag a comment your account needs a minimum score, I'm not sure, but I think it is 50 (or 100 or something). To flag something you click on the timestamp on top of the comment (typically something like: <x> hours ago). This opens a view of only that comment and replies to it. If you have enough reputation/points/karma, you can now see a flag link on top of the comment. If you click flag it ends up in a moderation queue. If multiple users click flag it might bypass the moderation queue and become flagged immediately without further intervention from mods.
- dang 6y ago
- resfirestar 6y agoThis looks like the real nightmare scenario for a supply chain attack. How long has SolarWinds has been breached? Were the attackers after a small number of US government targets + Fireeye or will we (more likely) discover extensive breaches at other companies worldwide as a result of this disclosure? Interesting times for those working in security.
- octoberfranklin 6y agoNo dude, metrology cutouts are the nightmare scenario for supply chain attacks: https://news.ycombinator.com/item?id=25043465 https://news.ycombinator.com/item?id=25043465
- NotChina 6y agoHow are they so sure it's Russia? It always seems like there is another country with much more interests in these companies. Given partnerships and the global economic status of Russia. It does not pass the sniff test.
- stupendousyappi 6y agoGiven that your account is seven minutes old and has one comment, you don't pass the sniff test.
- dang 6y agoPlease don't attack other users this way. The damage it does to the community is not worth the chance that you're right. It's much better to follow the site guidelines, which are designed to sustain an interesting community. In this case that means (a) respond with a better argument and/or (b) if you feel the comment breaks the site guidelines, flag it. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- youresameasme 6y agoWhere in the article does it say it's Russia?
- rixed 6y agoAttribution is indeed always the part that make me doubt about entire security related articles. Each and every time I looked into how such attacks were attributed to this or that country the evidence, if not the reasonning, were severely lacking at best. There is too much at stake in those affairs that it is safer to assume that only fabricated stories and counter fires are told about it all.
- lawnchair_larry 6y agoThe part you’re missing is that they don’t tell you how they did the attribution, and there are good reasons for this. You’re assuming that you know what they know.
- mcphilip 6y agoGood overview of what’s known so far about the situation: https://www.kxan.com/news/local/austin/austin-based-solarwinds-at-center-of-massive-us-government-hack/ https://www.kxan.com/news/local/austin/austin-based-solarwin...
- StartupTree 6y ago"Our European visitors are important to us" blah blah blah
- IfOnlyYouKnew 6y agoIt's funny how common that practice is among a single, well-defined segment of websites, and nowhere else: the hillbilly newspaper.
- rorykoehler 6y agoA quick cost benefit analysis will reveal why.
- kubanczyk 6y agohttps://web.archive.org/web/20201214100001/https://www.kxan.com/news/local/austin/austin-based-solarwinds-at-center-of-massive-us-government-hack/ https://web.archive.org/web/20201214100001/https://www.kxan....
- ocdtrekkie 6y agoA big takeaway here is that telemetry is a security threat. If your application is pushing opaque regular traffic out for not adequate reasons you are providing a good cover for malware activity. Even a fresh Windows install these days has so much regular network activity for telemetry and other services, it's trivial to hide bad behavior.
- deleted 6y ago[deleted]
- gauravphoenix 6y agoMany detection tools do leverage the fact that if there is spike in NXDOMAIN DNS responses, it could be a malware. However such signals are also exhibited by infra issues. This is sort of analogous to cancer detection. Cancers cells are really good at disguising themselves.
- inetknght 6y ago> spike in NXDOMAIN DNS responses NXDOMAIN is watched because known malware has been observed to use it. But I tell you: malware can just as easily utilize a valid DNS response.
- duskwuff 6y agoDomain-generation algorithms work along the lines of: seed a random number generator with the current date. Spit out a handful of domain names, and try contacting each of those a couple times each day. This way, the malware isn't tied down to a specific domain or set of domains; the bot herder just has to register any one of the domains in the current day's set, and if they lose the domain, they can repeat the process the next day. The downside is that all the bots will generate a lot of DNS traffic checking for those domains. That's what's being detected here -- the NXDOMAIN responses don't carry any desired data, but the malware can't avoid generating them while it's looking for its owner.
- octoberfranklin 6y ago
- dang 6y agoAnother current thread is https://news.ycombinator.com/item?id=25409416 https://news.ycombinator.com/item?id=25409416. Both discussions are good, but I don't think we'll merge them, because this one is oriented around the more specific information in the fireeye.com article.
- lovecg 6y agoThe question I have is how does something like this end up undetected in the officially signed update? I would imagine internally there would be a source control system with auditable commits, and the commit hash would be signed with the executable — is that not standard practice?
- gauravphoenix 6y agoNope. Theoretically it should be happening but it rarely happens. In fact, given the sophistication of this particular adversary, they would have just compromised the build server(and resign the binary) and I doubt anyone goes to the length of verifying the build server builds against some reference.
- numbsafari 6y agoMaybe we should be, in response to knowing the viability and use of CI hacks? Ensure you have a reproducible build, then randomly build on a different machine and compare file signatures of the results. Do it every so often with a “clean room” machine. Probably no need to run parallel infra that’s just as likely to be hacked.
- jcims 6y agoWhile it’s absolutely not a requirement, my first guess would be an inside job. They hire for locations all over the world, and once you’re in it would be much easier to spend your time finding and/or creating the scenario for this functionality to get into the build without detection.
- thu2111 6y agoNo way. So far there have been zero APT attacks traced back to inside jobs. That's not how modern spy agencies work anymore. Code signing hardly means much. All you have to do is compromise CI to make an organisation sign a binary that isn't based on reviewed code. How hard is that? I think for the vast majority of us we have to admit our CI would be very easy to compromise. Especially if you can get administrator credentials as easily as these guys apparently can. But even without that, CI executes and runs arbitrary code that's intended to only come from trusted individuals. Step 1. Find a third party package or library that is being downloaded by a package manager onto a CI box. Step 2. Compromise the open source dev's laptop by phishing them or something else really basic. Step 3. Put a back door into that package. Step 4. Wait for it to be downloaded to and executed on SolarWinds CI. Step 5. Insert a rootkit into CI. Now that rootkit can just systematically insert the final back door into any build of that file it sees using link time editing. Every build will have the back door and the owners will eventually sign the binary and push it out to production. In many orgs CI can actually request signing of any binary it wants, so you don't even have to do that. You can just hack the CI machine, grab the credential used to authenticate to the HSM, sign your preferred binary and then ensure users are downloading it.
- gauravphoenix 6y agoI would have hoped that the sensitive govt networks are locked down enough to not allow just any outbound traffic.
- jonstewart 6y agoGiven that there’s evidence of the attacker replacing their tools after gaining credentialed access and how long the supply chain has been in place, it’s going to be difficult to rule out compromise if you’re a SolarWinds customer. Logs that would record relevant activity of the trojan likely won’t go back all that long.
- TecoAndJix 6y agoSolarWinds Security Advisory: https://www.solarwinds.com/securityadvisory https://www.solarwinds.com/securityadvisory
- ahupp 6y agoI wonder if this will be a "Reflections on Trusting Trust" situation; it's easy to spot malicious code in source control, but if your build machine was compromised to insert malicious code at compile time that would be easy to overlook.
- hedora 6y agoI don’t see how to word this politely. What is SolarWinds, and why are all these organizations using it? Clearly, it’s a giant single point of failure, but other than that, I’ve never heard of them. Their marketing says they do network monitoring, etc. Do they have a legitimate product, or is this just another case of enterprise checkbox security theater gone awry?
- rootsudo 6y agoVery common in public sector, think of just a fancy graphite/splunk that deals mostly on SNMP and creates reports that fits 99% of the network/sysadmin needs for gov/compliance.
- jcims 6y agoThe company has been around for a while and has a nice and comprehensive set of products for managing systems and networks.
- oars 6y agoIf we replaced SolarWinds with Splunk would you be reacting the same way?
- joncrane 6y agoNot apples to apples. Splunk can do way more than Solarwinds but is also slower to ingest the data and produce an alert.
- bosswipe 6y agoIt's one of the bigger names in the network monitoring space. Most network and systems people that work for a big enough company will have at least heard of them.
- vermilingua 6y agoSo many organisations use it because they made the mistake of letting a SW sales rep get their contact details /s
- 6y ago
- deleted 6y ago[deleted]
- filleokus 6y agoReally impressive! It'll be interesting to see how the malware got signed by SolarWinds... I wonder what the cost of an attack like this is. It doesn't feel impossible for a small group (< 10) of very smart and motivated people to maybe achieve this in 12-18 months of work? Of course they could be working on behalf of some nation state, but compared to let's say an ICBM attack, this is probably not out of the realm of non-nation-state actors to pull of. An entrepreneurial crime syndicate would probably have the resources to do this (or outsource to some mercenary attackers).
- nl 6y agoSome (pretty interesting) details here: https://blogs.microsoft.com/on-the-issues/2020/12/13/customers-protect-nation-state-cyberattacks/ https://blogs.microsoft.com/on-the-issues/2020/12/13/custome... They used a an on-premises compromise to gain access to an organization’s trusted SAML token- signing certificate. This enables them to forge SAML tokens that impersonate any of the organization’s existing users and accounts, including highly privileged accounts Note the on-premises compromise (!!) Microsoft credits it to a nation state.
- thu2111 6y agoI think that just means they hacked a machine running in the offices, not that they were literally physically on-premises. There's a rather large opaque black hole in the middle of this assertion: how are the attackers obtaining administrator credentials to the SSO systems? There seems to be a pretty big leap here from "get code inside the firewall via Orion" and "oops, now your SSO ticket private key is pwned". They say administrator credential compromise, OK, that's a pretty general class of techniques. The details would be good because otherwise there seems to be a missing link here.
- syshum 6y agoI am sure since they are classing in as a "nation state hack" the government will move quickly to classify the details for "national security". So then the question becomes was it really a nation state, or just incompetence being covered by claims of "nation state" hacking
- drevil-v2 6y agoThe Microsoft blog post on this incident has more details on how the attackers might have managed to get their malware digitally signed by Solarwinds https://blogs.microsoft.com/on-the-issues/2020/12/13/customers-protect-nation-state-cyberattacks/ https://blogs.microsoft.com/on-the-issues/2020/12/13/custome... > An intruder using administrative permissions acquired through an on-premises compromise to gain access to an organization’s trusted SAML token- signing certificate. This enables them to forge SAML tokens that impersonate any of the organization’s existing users and accounts, including highly privileged accounts.
- p932 6y agoYour quote seems to be describing what Microsoft observed from the attackers on the compromised machines with SolarWinds Orion not how the software was compromised in the first place. See detailed version: https://msrc-blog.microsoft.com/2020/12/13/customer-guidance-on-recent-nation-state-cyber-attacks/ https://msrc-blog.microsoft.com/2020/12/13/customer-guidance... > In actions observed at the Microsoft cloud, attackers have either gained administrative access using compromised privileged account credentials (e.g. stolen passwords) or by forging SAML tokens using compromised SAML token signing certificates. > Although we do not know how the backdoor code made it into the library, from the recent campaigns, research indicates that the attackers might have compromised internal build or distribution systems of SolarWinds, embedding backdoor code into a legitimate SolarWinds library with the file name SolarWinds.Orion.Core.BusinessLayer.dll.
- dane-pgp 6y ago> the attackers might have compromised internal build or distribution systems of SolarWinds It should probably become a requirement (for both open and closed source software) that any updates be not just signed but have their hashes available in a Binary Transparency log[0]. When you first install a piece of software, you might need to calculate the hash locally and manually search for it in a log's web interface, but after that, its software-update routine should check that the new version it is downloading has had its hash published in a known place. That way, software publishers can check an append-only independently-run log to see what has been signed with their keys. I suppose there is a risk that an attacker could prevent users from receiving security updates by DoS'ing the transparency logs, but that should be harder than just DoS'ing the servers that host the software updates themselves. Large organisations could also maintain mirrors of these logs on their internal networks, which would help with privacy/latency/availability, and the logs should ideally be available as Tor hidden services too. For non-critical updates, the log checking routine should require that the update's hash had been in the log for a certain period of time, long enough for the software publisher to notice and raise the alarm to their users. Updates marked as critical should default to stopping the software from running until the necessary period had elapsed, for which the workaround would be a fresh install of the newer version by whomever has the admin privileges to do that. [0] https://wiki.mozilla.org/Security/Binary_Transparency https://wiki.mozilla.org/Security/Binary_Transparency
- johndoe42377 6y agoSimilar to the attack on Ukrainian systems, which was also thorough a centralised update system being compromised. It is that simple.
- sivm 6y agodoron gordon
- wryroybatty 6y agoHow much data could have been exfiltrated? Surely not GB worth?
- _wldu 6y agoI know why orgs install central management systems (ease of maintenance at scale). But when one system can be used to compromise all the nodes in an org, is it really worth it? Diversity is good for security. I understand that Solarwinds is a reputable company with all the right security auditing and compliance credentials. That's why they are used by US federal and state government agencies. Perhaps the criteria (MBA/Audit/Management driven security) that governments use to judge vendors is useless against real world attacks? If I wanted or needed to be compliant, I would buy that software (like all the other gov agencies), but if I wanted to be secure, I would not.
- bostik 6y ago> I know why orgs install central management systems (ease of maintenance at scale). But when one system can be used to compromise all the nodes in an org, is it really worth it? If you think about it, the modern cloud-native approaches are logically aiming for the same. You have a centralised management system (the cloud provider's engine), with programmatic configuration (infrastructure-as-code and indeed everything-as-code). We even call these modern best practices. Just instead of a webgui management console, we have git repos, and in place of two-person change protocol we have mandatory code reviews enforced by tooling. A single point of failure is one side of the coin. The side you're not looking at is single point of control.
- jabroni_salad 6y agoQuick question -- If you can't certify what exactly your computers have been doing, can you actually say that you are secure? It seems like not using NMS is just fast tracking yourself to become one of those orgs that gets owned for months and never even knows about it.
- _wldu 6y agoThis compromise supposedly goes back to March 2020. We're just learning about it now.
- deleted 6y ago[deleted]
- driverdan 6y agoI hope this results in a massive class action lawsuit against SolarWinds. Companies need to be held accountable for their security practices. The fact that the binary was signed and distributed through their official channels undetected for this long seems like malfeasance.
- pgo 6y agohttps://twitter.com/vinodsparrow/status/1338431183588188160?s=08 https://twitter.com/vinodsparrow/status/1338431183588188160?...
- SV_BubbleTime 6y agoGet ready for a bunch of new accusations about the election. Dominion Voting systems uses SolarWinds according to their login pages, while the hack was still unknown. I shudder to think of the crazy conspiracy theories that are coming from a certain someone on twitter now... smh.
- User23 6y agoWhy would it be crazy for a state level black hat actor to want to hack votes in the USA’s elections? I’m not saying that happened, but it’s not prima facie crazy. In fact I recall hearing that Russia somehow hacked the election for the past 4 years. Motive and now means is there. Creating opportunity is within a state actor’s capabilities too.
- SV_BubbleTime 6y ago>not prima facie crazy. No, You're right. It's not. But I mean, it's totally different that a hack to Solarwinds and FireEye that was before the election and just assume a correlation knowing that all these counties and election systems suppliers use SolarWinds but only might be contributed to Russia should be considered the same as the Russian disinformation campaign carried out on Facebook ads in 2016. I mean, one would be election shattering but there is no proof, and the other we are pretty sure happened even though nothing came of it at all.
- hikerclimb 6y agoHopefully more hacks happen against us companies