13 ms·
The problem is that Equifax put within webserver's reach information that had no business being there. Apache Strut's vulnerability is unfortunate, but it shoul
by mevile 9y ago
The problem is that Equifax put within webserver's reach information that had no business being there. Apache Strut's vulnerability is unfortunate, but it shouldn't have been the keys to the kingdom, where the kingdom is the personal information of nearly the entire US adult population with a credit history. If I knew a service relied only on the security of a web server to protect deeply personal information such as my name and SSN I'd never sign up. We didn't have that choice with Equifax.
Having said that, definitely keep up on the vulnerabilities of software you use. It's hard though, especially when you're relying on a great deal of dependencies. A company the size of Equifax should have had a team dedicated to this. A team. It doesn't seem like they had anyone who knew anything about basic security at all.
- tptacek 9y agoThis is a problem, yes, but I think it's worth keeping the perspective that it's a practically universal problem. That doesn't mean we should be letting Equifax off the hook. But nobody should pretend that RCE on an on-prem webserver wouldn't be game-over, for the entire internal network, across the majority of the Fortune 100. We have as a society chosen to trade the security of our personal information for greater and cheaper access to products and services. Nobody on HN will like that fact (at least, as stated bluntly like that; plenty of them do like the increase to their earnings capability that results from that tradeoff), but it's generally true.
- justicezyx 9y agoYou claim that you keep my data safe. You missed it. It's not a trade off, it's a failure on your part...
- tptacek 9y agoIt very much is a trade-off, because the expense that would go in to making a large enterprise network reliably secure would be extreme, and would drive prices up, retard economic growth, and most likely result in software engineers making substantially less money. I'm not saying you should like the tradeoff, or that the existence of the tradeoff doesn't have implications for how we should regulate banks and credit ratings agencies. I'm just saying that if you think there's some other CRA that has reliably protected its IT assets from threats like this, you're wrong.
- abiox 9y ago> It very much is a trade-off this seems like goalpost moving. first you said 'we as a society', yet there isn't much 'we' in this. it doesn't seem to me that people (the general public) are properly aware at just how vulnerable they are. they haven't therefore 'decided' that this is okay. > the expense that would go in to making a large enterprise network reliably secure would be extreme, and would drive prices up, retard economic growth what is this based on?
- tptacek 9y agoPart of living in a society is recognizing that you still belong to it even when it makes decisions you don't agree with. To more directly answer your question, though, consider a regime of general commercial liability for security incidents caused by failures of engineering (which Equifax is an example of). What would it be like to bring new products to market under such a regime? How much more difficult would it be to enter the field? How much more difficult to hire qualified people? What would that do to the prices of things made possible largely through software engineering work? When the prices of things increase, they're put out of reach of more consumers; markets constrict. What does that do for employment? What does reduced employment in a sector do to wages? I'm not saying this to justify insecurity. I'm ambivalent about the tradeoffs our society has made on these issues. I just don't think we tend to grapple with them honestly on HN.
- chris_wot 9y agoIf that new product exposes scads of private data, then it's rather irrelevant. These credit scores aren't something most people signed up for, they are a product for financial lenders and not for the direct consumer.
- tptacek 9y agoLike I said, the fact of the tradeoff probably does have implications for how we regulate companies like Equifax.
- tedmiston 9y ago"Safe" is full of grey area, just like security isn't binary.
- noir_lord 9y ago> But nobody should pretend that RCE on an on-prem webserver wouldn't be game-over, The system/network I inherited at new job it absolutely would be, it won't be in a year though. We aren't a fortune 500 but there are simple, practical things you can do to limit the damage an RCE on a webserver can actually do.
- tptacek 9y agoEverybody can lock down a small network. Empirically, very few security teams can manage that for large networks with large numbers of stakeholders.
- chris_wot 9y agoThen systems of governance are utterly broken and management should step in to enforce remedial actions, ASAP. But I disagree, security teams can and should manage large networks with large numbers of stakeholders.
- tptacek 9y agoIf they can, why don't they? How many annual site-wide pentests have you heard of that failed hard? And those are the people savvy enough to conduct annual site-wide pentests!
- extrapickles 9y agoA common pentest isn't geared towards discovering architectural weaknesses like all Americans PII is one popped webserver away from disclosure. What's needed is an architectural security review, which is even rarer than people conducting an annual pentest.
- tptacek 9y agoWhat's your point? I'm saying, sitewide pentests almost always succeed.
- harshreality 9y ago> But nobody should pretend that RCE on an on-prem webserver wouldn't be game-over, for the entire internal network, across the majority of the Fortune 100. You certainly have more experience with this than the vast majority of the rest of us, but with that kind of data on millions of people, shouldn't it be, and why isn't it, standard practice to put servers storing that data behind an extremely limited API, with a firewall in place to monitor queries and responses and severely rate-limit or stop additional traffic if something suspicious happens, until the monitoring people can take a look?
- tptacek 9y agoBecause that not only adds operational expense but also requires engineering teams to have a level of systems engineering expertise and diligence that virtually no line-of-business software project has.
- ethbro 9y ago*adds operational expense without a counterparty willing to pay for that expense Those with data on Equifax's servers aren't Equifax's customers, so other than "not having a PR cluster&#@_" economic incentives are not aligned to prevent this.
- sillysaurus3 9y agoIt's probably best to treat Equifax specially rather than making an example out of them for the rest of the industry to follow.
- concede_pluto 9y agoIf they aren't qualified to design with least privilege, what are they qualified for, beyond maybe cat videos?
- tptacek 9y ago
- bradleyankrom 9y ago> We have as a society chosen to trade the security of our personal information for greater and cheaper access to products and services. Agree, and it's one of the reasons I generally try to avoid using products that are low-cost or free (the other reason being questionable sustainability, regardless of funding). At some point, products need to break even or they will go out of business. As a customer, you are always paying for the product in some form or another. For something that's free, you might be paying with your personal information now... or later if running on a low budget has caused the product developer to cut some corners when it comes to infosec.
- nodamage 9y agoShouldn't we hold credit reporting agencies to a higher standard than the rest of the Fortune 100 though? After all, they store sensitive personal information about the vast majority of US adults.
- tptacek 9y agoMaybe? I've said that elsewhere on the thread. But I'm not sure. Every big company tends to hold all sorts of hugely sensitive information.
- swixmix 9y agoThis was a reaction to Quartz Media's (QM) article that made Apache Strut appear to be incompetent. [0]. QM updated the article. [1]. Equifax was vulnerable for at least two and half months, and database access wasn't detected. [2]. [0]: https://web.archive.org/web/20170908214344/https://qz.com/1073221/the-hackers-who-broke-into-equifax-exploited-a-nine-year-old-security-flaw/ https://web.archive.org/web/20170908214344/https://qz.com/10... [1]: https://qz.com/1073221/the-hackers-who-broke-into-equifax-exploited-a-nine-year-old-security-flaw/ https://qz.com/1073221/the-hackers-who-broke-into-equifax-ex... [2]: https://baird.bluematrix.com/docs/pdf/dbf801ef-f20e-4d6f-91c1-88e55503ecb0.pdf https://baird.bluematrix.com/docs/pdf/dbf801ef-f20e-4d6f-91c...
- upstarter 9y ago> The problem is that Equifax put within webserver's reach information that had no business being there. Can you please link to your preferred tutorials (or other sources) which explain how to set up the most secure server architecture, with regards to this story/subject, for those of us who would like to do a better job? Thanks a lot!
- sillysaurus3 9y agoIt doesn't really exist. You "can't" because some computer still needs access to the data. You can do layered security, but how? Think about a customer service rep. They need access to ~everybody's data at a moment's notice because they're a CS rep. That's what they do. That means every CS rep machine has access to all the data. I'm not sure this is solvable. SSN's are the problem, not the way that we store SSNs. They still need you to read off your SSN when you call in, so you can't just encrypt it. Think about Equifax in particular. They're a credit bureau. Of course they need all the data available all the time -- that's what a credit bureau does. The question is, how would you partition the data so that a failure in one place wouldn't lead to exfiltrating all the data everywhere? The trouble is, it wasn't just "they popped Struts and then ran an scp." Even if you partition the data, they popped Struts! They have RCE inside your network. It's absurdly easy to pivot from one machine to another in a given network. You keep a helpful ~/.bash_history file that says exactly what's important and where everything is installed. Stuff like this is why I argued for deleting it, since it was the #1 most helpful tool I had that helped me pivot inside a target network. But then you get people yelling about taking away their bash history. You can imagine how terrible this problem becomes in practice. Nobody here gives a crap about Equifax; they just want their heads to roll. Unfortunately you'll be next on the chopping block because no one has solved these issues. As soon as the world goes crazy over the lack of security, it's game over for us all because it leads to a highly regulated world that won't be much safer. See how resistant we all were to the mere mention of increasing security procedures: https://news.ycombinator.com/item?id=14104156 https://news.ycombinator.com/item?id=14104156 I was snarky in that thread which certainly didn't help, but you have an uphill battle if you want to implement anything on a large scale.
- 9y ago
- g051051 9y agoI'd guess that it didn't rely on the security of a web server, but the entry point was the "thin end of the wedge" for probing and deploying more sophisticated attacks. At some point, the front end of any web application has to go to the back end database to get information. So there's just no way to cleanly and completely isolate the front facing parts of an application from possible breach.
- deejaybog 9y agoThere are definitely methods of limiting the damage of a front end being taken over. A proper threat modeling process was obviously never performed.
- bognition 9y agoThis just isn't true. With proper controls and an internal authentication stack you absolutely can guard against an edge getting owned.
- smegel 9y agoMost transactional websites need access to the core customer database, otherwise they could not function. Just DMZing everything is not always possible. Maybe segmentation is the only possible defence - customers A-C served out of this DB with a different stack to limit the damage. But once they get in, they get in, right. Its just a matter of finding the next vulnerability. As long as you are providing a rich customer experience, with the ability to generate reports based on historical information, you have risk.
- jtakkala 9y agoI completely agree about having a dedicated team, and I'd expect a company of their nature to be at the forefront of security best practices. I just checked some Equifax domains against SSL Labs, and while their Canadian site (https://www.econsumer.equifax.ca https://www.econsumer.equifax.ca) scores an A-, it has no forward secrecy. I'm surprised to see a modern web server not supporting FS today. Worse, the main entry point to their Canadian site (http://www.consumer.equifax.ca http://www.consumer.equifax.ca) as indexed by Google does not redirect to a TLS enabled page, although they do seem to have a TLS endpoint for that domain -- but not sure how people are expected to get to it. Edited to add: The first link is only accessible through a redirect by clicking on the "Get Started" button on their main Canadian site. Furthermore, even selecting Canada from the drop-down on https://www.equifax.com/personal/ https://www.equifax.com/personal/ redirects to the insecure non-TLS site.
- FabHK 9y agoWouldn't an extension like HTTPS Everywhere do it for you? (Provided you're savvy enough to have it installed; in other words, they should still put in that redirection...) https://www.eff.org/https-everywhere https://www.eff.org/https-everywhere
- jtakkala 9y agoI've got HTTPS Everywhere running and it doesn't appear to have a rule since I'm still hitting the non-TLS site.
- jlgaddis 9y agoThe results of a scan of equifaxsecurity2017.com by Mozilla's "Observatory" do not inspire confidence either: https://observatory.mozilla.org/analyze.html?host=equifaxsecurity2017.com https://observatory.mozilla.org/analyze.html?host=equifaxsec...
- avip 9y agoMy rule-of-thumb for assessing security mindset is STS header. Which they don't have. That tiny Norton secured icon does not make me extra confident.
- matchagaucho 9y agoWe won't really know until Equifax releases the full RCA (root cause analysis). The attack could be way more sophisticated and involve several hacks through the stack. Injecting a JSON blob to the Struts REST deserializer that effectively results in executing payload code would indicate high investment in the attack. The hacker(s) possibly had insider information. They possibly used social hacking techniques to acquire data decryption keys. Given Equifax stores more than 143M records, the initial report would imply some sort of data segmentation limited the scope of damage. If the doors were wide open, why didn't the hackers take 100% of the records? None of this excuses Equifax or removes liability (I've personally initiated a credit freeze and will likely need a LifeLock Plus account for the rest of my life thanks to this breach).
- solomatov 9y ago>Injecting a JSON blob to the Struts REST deserializer that effectively results in executing payload code would indicate high investment in the attack. Why does it indicate high investment? IMO it's similar to SQL injection attack. People, just take a look at points where complicated data can be deserialized and check whether adequate checks can be overcome. As I see from their report here: https://cwiki.apache.org/confluence/display/WW/S2-052 https://cwiki.apache.org/confluence/display/WW/S2-052 they use XStream to deserialize REST data. XStream is basically Java serialization/deserialization to XML instead of binary. As far as I remember from my experience of working with XStream, XStream can deserialize almost any object, even the ones which aren't serializable by default. It's quite trivial to go from here to execution of arbitrary code.
- matchagaucho 9y ago> It's quite trivial to go from here to execution of arbitrary code. Code that does what? Dumps all environment vars? OK, maybe that produces a DB2 connection string. Refactor code, re-inject, does the connection work? OK. Refactor code, re-inject and dump all tables. etc... This attack requires an iterative trial-and-error probing process, typically done behind a proxy for many weeks/months. From Equifax's PR: "the unauthorized access occurred from mid-May through July 2017.". That's a reasonable amount of time to probe all systems involved and develop sophisticated code.
- tmp3001 9y agoEquifax is in the business of selling this information to other companies. They are offering services, which allow banks and other institutions to check this information through web service interfaces. It is kind of a core functionality that this information is available through a web server. More old fashioned way would have been to expose this information through some obscure protocol, but I don't think that would make it any more secure. Now a different thing is that should this information be available through a server that is accessible directly through the Internet (and I don't know if it was) or if they should require for example a dedicated VPN connection.
- perlgeek 9y ago> Having said that, definitely keep up on the vulnerabilities of software you use. It's hard though, especially when you're relying on a great deal of dependencies. A company the size of Equifax should have had a team dedicated to this. A team. It doesn't seem like they had anyone who knew anything about basic security at all. If you don't have a full team for this, another strategy is to rely on a vendor, and try to obtain as many dependencies as possible from this vendor. Of course, you still need a process for updating the vendor-supplied components as quickly as possible.