21 ms·

Microsoft Chose Profit over Security, Whistleblower Says

676 points by tyleroconnell 2y ago

fredgrott 2y ago
oh lets put fonts in the user space rather then the kernel space what could ever go wrong? this not new its a major feature of how MS works
1vuio0pswjnm7 2y ago
Sounds like the same Microsoft culture as has always been. Like a cult. It can do no wrong. The conversation with Microsoft businesspeople at conferences was always the same: Microsoft has no deficiencies, there is nothing it isn't working on and it has a solution for every possible problem. Other sources of software do not exist. There is only Microsoft. Total illusion put forth by delusional employees. The outside world can be ignored because life in the cult is good.
- Drakim 2y ago
  Don't worry, Microsoft has big ambitions and huge plans about how to truly present themselves as more safety oriented in the future.
  - NekkoDroid 2y ago
    AI Safety. Code is run through AI to check for vulnerabilities. Files are analyzed by AI to ensure they aren't malware. Every instruction is run through AI to ensure nothing maliciously is happening (mostly enforcing DRM :). Every pixel is output by AI to ensure you see nothing not intended for your precious eyes.
    - freedomben 2y ago
      You joke, but (the DRM part at least) is the future I fear is coming. It could hit us from so many angles (not forgetting Chrome's Web Environment Integrity and Apple's Private Access Tokens), and with all the money and power behind it (big tech plus big copyright), and the complete apathy of the average user towards this, it seems inevitable.
      - NekkoDroid 2y ago
        The DRM part wasn't really part of the joke, just a sad truth that is being worked on more and more that sadly fit into the joke.
- eganist 2y ago
  If memory over the last two decades serves, this is a relatively recent degradation. Microsoft's security reputation prior to the recent (5ish years?) failures was largely built up on top of the work stemming from the Trustworthy Computing memo. https://www.wired.com/2002/01/bill-gates-trustworthy-computing/ https://www.wired.com/2002/01/bill-gates-trustworthy-computi...
  - diggan 2y ago
    Bill Gates in 2002: "So now, when we face a choice between adding features and resolving security issues, we need to choose security." Satya Nadella in 2024: "If you’re faced with the tradeoff between security and another priority, your answer is clear: Do security." Microsoft in 2024: Run this software on your computer so we can take a screenshot of everything you do, index it and we promise Security is still, and have always been, the priority. And yes, we do store data unencrypted on your disk, why are you asking?
    - NekkoDroid 2y ago
      > And yes, we do store data unencrypted on your disk, why are you asking? But don't worry, you need to be an administrator to open the file. What? your average person daily drives an administrator account? How should we have known that???
- freedomben 2y ago
  This comment sounds hyperbolic, but it really isn't. It's really bad. This has been my experience with Microsoft employees also. In my experience, what makes for bad software is PM and engineering hubris. You definitely need some vision and confidence as just following user feedback is a recipe for terrible software as well. The key is to find the right balance and straddle that line. If it's been long enough for insiders to tell the story of Windows Phone and the eventual cancellation, I'd be fascinated to hear the story of that (from inception to death) and how that went internally given the culture.
  - tracker1 2y ago
    Just wanted to say that I thought the Windows Phone (the last version of such) was relatively nice. It had a decent developer experience, but was pretty much an also ran and didn't have enough market share to overcome mindshare for first party apps. When so many first apps were iOS first and Android later, throwing a third option in the mix just missed the mark more often than not. I was already in the Android ecosystem and far less cynical at that point about Google.
    - Ylpertnodi 2y ago
      I didn't get a Windows phone because i don't trust Microsoft. A friend had one and it was really ok, but no way for me.
      - tracker1 2y ago
        In retrospect, I don't trust Apple or Google either...
        fsflover 2y ago
        So you need a GNU/Linux phone?
  - fingerlocks 2y ago
    Can confirm, it is 100% hubris based on my limited time of working at Microsoft. There is pervasive NIH syndrome, re-inventing the wheel, and massive amounts of over engineering and unnecessary abstraction caused by chasing the endless "But what if...?" dragon. This behavior is justified, and critics are silenced, by the "But we're an enterprise company!" cop-out
- jan3024 2y ago
  [dead]
- diggan 2y ago
  To be honest, that sounds like every company that suffers from delusions of grandeur and wants to conquer the planet, one way or another. What you're saying is equally true for Apple, Google, Amazon and most other public companies today. You're never gonna get "Use this Microsoft product" as an answer from Apple support/engineer even if that product would solve your particular problem better.
  - 1vuio0pswjnm7 2y ago
    The conferences always included representatives from other large, public companies in the same or similar industries, e.g., Apple or Amazon, as well as other, different industries. But there was always something cult-like about the folks from Microsoft, their level of BS and (deliberate?) ignorance, that I never experienced with the others. To be clear, I could not make the same comment about Apple or Amazon businesspeople. While they may exhibit their own stigmatic qualities, they are, IME, different. Nothing like Microsoft. Microsoft does not suffer from "delusions of grandeur". It achieved grandeur a long time ago, and then became delusional. Currently, it is either #1 or #2 on the list of the world's wealthiest companies. Comments suggesting that the company has "changed", and such comments have been popular on HN in recent years, are quite amusing.
- surfingdino 2y ago
  I'll give you a 4-letter word... Zune /s
- magicalhippo 2y ago
  What Microsoft can provide are lots of nice stickers saying they conform to this or that security standard, making security folks in IT departments all warm and fuzzy. At least that's how it appears from our POV, selling B2B applications. They don't seem to care that much about actualities as long as the security checklist passes.
deleted 2y ago
[deleted]
everdrive 2y ago
I'm not defender of Microsoft, but I don't know if I could point to any company which does not put profit over security.
- diggan 2y ago
  I guess the issue becomes when they say security is the top priority (and have been for two decades), yet all actions point towards it not being so. > Bill Gates in 2002: "So now, when we face a choice between adding features and resolving security issues, we need to choose security." https://www.wired.com/2002/01/bill-gates-trustworthy-computing/ https://www.wired.com/2002/01/bill-gates-trustworthy-computi... > Satya Nadella in 2024: "If you’re faced with the tradeoff between security and another priority, your answer is clear: Do security." https://www.theverge.com/24148033/satya-nadella-microsoft-security-memo https://www.theverge.com/24148033/satya-nadella-microsoft-se...
  - ls65536 2y ago
    Obviously, nobody is going to outright admit they put profits above security; indeed, they will often state the opposite. But their closely-held beliefs will shine through when it comes time to make decisions and the outcomes of those decisions are exposed to their customers and to the public.
    - lesuorac 2y ago
      Does Bill or Satya write code anymore? It could very well be that they consider security the top priority but it's a moot point because they're so removed from operations. Although I would suspect that you're effectively right in that they either don't have it as a top priority or think they do but have a reveal preference of they don't. For example, an engineer that does rigorous security testing and finds nothing as well as launches one project gets promoted less often than an engineer that launches two projects and doesn't do rigorous security testing.
      - deleted 2y ago
        [deleted]
hypeatei 2y ago
Execs should absolutely be held responsible, but the human factor is always there. Many times people will take the easy route and get worn down by security practices or roadblocks. I think it's too easy to go "alright focus on security" and then expect it trickle down and figure itself out.
cellu 2y ago
surprisedpikachu.jpg
ThinkBeat 2y ago
This whole article seems a bit odd to me. What is "the product" ? Presumably this is not related to earlier problems with SolarWinds. Did MS screw up. Yes. However, all things have bugs. I takes one person finding one bug and exploiting it. and there are enormous resources going into finding one, and I am certain that this is the only one. I am sure the NSA is sitting on a pile of them. Whereas the developers have to think about everything that can happen and protect against it. Does this make Microsoft different from its competitors? I think Microsofts strategy is somewhat similar to Linus: Where security patches are often not part of new releases due to the burden of establishing what the consequences of bigger changes would be, and the fact that security people dont do sane things. (But you can of course pull them and make it part of an in-house distro. https://lkml.iu.edu/hypermail/linux/kernel/1711.2/01357.html https://lkml.iu.edu/hypermail/linux/kernel/1711.2/01357.html
- pgraf 2y ago
  It is true that nothing is 100% secure. Sitting on a major security vulnerability internally with a motivated employee pushing to fix it and doing nothing for business reasons is not negligence, but malice. People in the chain of command need to be held accountable for this.
- latexr 2y ago
  > However, all things have bugs. There are bugs and there are critical flaws you’ve been warned about. This is the latter. The fact that this was known by Microsoft but not fixed is the story.
- SuchAnonMuchWow 2y ago
  > Harris said he pleaded with the company for several years to address the flaw in the product, a ProPublica investigation has found. But at every turn, Microsoft dismissed his warnings, telling him they would work on a long-term alternative — leaving cloud services around the globe vulnerable to attack in the meantime. That is not a screw-up, that is a deliberate decision.
- shkkmo 2y ago
  Microsoft had a known, high consequence, security flaw that they did not acknowledge or fix, they had evidence that indicated it had already been exploited and they knew they had limited to no ability monitor for exploitation. This choice lead directly to the SolarWinds hack that happened in 2019 was discovered in late 2020 and acknowledged by the USG in early 2021. Many companies make bad choices around security for profit, however that factors I listed above make this extremely egregious. I would seriously question any use of Microsoft products in any security conscious organization after this reveal. I also hope that anyone negatively effected by the Solar Winds sue Microsoft for knowing about the vulnerability for years without fixing it or disclosing it.
outside1234 2y ago
Unless that other priority is laying people off. Then the layoffs are more important.
minisooftwin 2y ago
> “If you’re faced with the tradeoff between security and another priority, your answer is clear: Do security,” the company’s CEO, Satya Nadella, told employees. Satya's model of making security a priority at Microsoft: - Cram ads in every nook and corner of Windows. Left, right, centre, back, front, everywhere. What else is an operating system for? - Install a recorder which records everything you do. For the benefit of users of course - you know, what if a user missed an ad and wants to go back and see what they missed. - Send a mail to your employees and tell them "Do security". Mission accomplished - Microsoft is now the most secure platform.
- VyseofArcadia 2y ago
  The Microsoft bribes scandal broke not too long after I had to take the "hey don't do bribes" training at Microsoft. That event really drove home for me the fact that all of the trainings, emails, processes, etc. are mostly plausible deniability. There are people who care about security at MS. I know, I've met them, but for the most part all of this exists so that Satya can plausibly say in court or in front of congress, "well we told them to do security better. This is clearly the fault of product teams or individual contributors, not Microsoft policy and incentives."
  - montjoy 2y ago
    I dunno, that’s a pretty cynical take. Isn’t it just as plausible that they became aware of the bribes internally and were trying to curtail them when the scandal broke out? Or maybe the “don’t do bribes” training actually worked enough for someone to whistleblow even if official internal channels failed? Those who are doing wrong often try to stymie others from making positive changes out of fear, greed, etc. Edit: I just want to add that there are things to be cynical about - I’m not completely naive. If it’s your legal department heading up the training then you can be pretty sure that there was a cause for it.
    - blowski 2y ago
      Yes, massive companies are a nest of conflicting priorities. The sales team wants to do whatever it takes to win the deal, and the legal team wants everyone to behave ethically at all times. The board wants to be shocked(!) when it turns out those goals are in conflict, with the ethical side sometimes losing out, to remove any personal risk to themselves.
spydum 2y ago
As per usual, executive platitudes around "security first" don't matter. If you pay and promote people for features, and don't reward security culture, people are not dumb: they and the management layers will optimize for that. I don't know how to design incentives to solve for this, but this is always going to be the way it is.
- olivierduval 2y ago
  I think that it could be "security as a feature" Usually, a feature is included in a product if the marketing show that it will grow the business more than the cost of the feature. Maybe we can try the same idea ? "We identified this vulnerability, and it will impact X % of our customer and Y % will leave (+ reputation damage) so we will loose BIGNUMBER $. However, we can correct it for SMALLNUMBER $ in Z days. Decision ?"
  - deleted 2y ago
    [deleted]
  - nicce 2y ago
    Real security cannot be feature. Your complete system design and other features should be based on the idea of ”security first”, if you really want to build secure systems.
    - hulitu 2y ago
      > Your complete system design and other features should be based on the idea of ”security first”, if you really want to build secure systems. One can argue that the most secure system is the one turned off and not used. And i am not talking about devices with builtin batteries.
      - nicce 2y ago
        One can always argue that, but, fundamentally security is about limiting the systems' use for its purpose and eliminate all unwanted scenarios. If you need to use the system, you cannot turn it off or not to use it.
pgraf 2y ago
Imagine a major bridge that was built by a contractor. A internal safety inspector repeatedly warned his supervisors of structural deficiencies that could lead to the collapse of the bridge. Furthermore, in the pass of time two external sources publicly warned about the issue, but the company downplayed the importance. Finally, the bridge collapses. It becomes evident that the company did nothing about the issue because it didn‘t want to loose contracts selling more flawed bridges. The public would justifiably go nuts, and there would be legal consequences for everyone involved. What is different in our industry that companies (and managers) get away with such malice?
- deleted 2y ago
  [deleted]
- red_admiral 2y ago
  Wasn't there something a bit like that with the Morandi bridge that collapsed in Italy? (There was definitely something like that with the Mottarone cable car that had been running for years with the safety catch disabled. When the tow-rope snapped, wiht no catch, the cabin rushed down and killed everyone on board.)
- natsucks 2y ago
  I don't understand how this doesn't destroy a company. They willfully ingored a serious risk and it had major national security implications.
  - dfedbeef 2y ago
    Have you tried to use Google customer support
- magicalhippo 2y ago
  Here in Norway a bridge built with known structural deficiencies did in fact collapse[1], and basically nothing has happened except tax payers get to pay even more for a new bridge. Unless enough lives are lost, people generally don't care that much it seems. [1]: https://www.nrk.no/innlandet/statens-vegvesen-legg-fram-rapport-etter-at-tretten-bru-kollapsa-1.16721805 https://www.nrk.no/innlandet/statens-vegvesen-legg-fram-rapp...
xyst 2y ago
> “If you’re faced with the tradeoff between security and another priority, your answer is clear: Do security,” corporate morality is a Potemkin village. It's all about the profit and appeasing the shareholder, baby! is anybody honestly surprised at this point? The abbreviation of "M$" is well deserved despite small OSS contributions and attempts to PR their way out of previous history (ie, United States v. Microsoft Corp. [2001])
- winocm 2y ago
  The original sins: https://www.nytimes.com/1970/09/13/archives/a-friedman-doctrine-the-social-responsibility-of-business-is-to.html https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr... https://www.sciencedirect.com/science/article/pii/0304405X7690026X https://www.sciencedirect.com/science/article/pii/0304405X76...
erand293 2y ago
I worked at Microsoft the entire time period covered in this article. I've interacted with MSRC on multiple issues with the product team I'm on. My experience inside Microsoft is not at all consistent with what the whisteblower is saying, for what it's worth. It's completely unrecognizable, in fact - and I'm having a hard time reconciling what I'm seeing with what I'm reading here.
execveat 2y ago
I work in infosec, and this sounds like a communication failure on the whistleblower's part. Contrary to what many people believe, the profits should be prioritized over security for the most companies, that's only natural (after all, they don't generate any profits themselves, typically). The key is finding the right balance for this tradeoff. Business leaders are the ones that are responsible for figuring out the acceptable risk level. They already deal with that every day, so it's nonsensical to claim they aren't capable of understanding risk. InfoSec's role for the most part is being a good translator, by identifying the technical issues (vulnerabilities, threats, missing best practices) that go beyond the acceptable risk profile and to present these findings to the business stakeholders, using the language they understand. Either the guy wasn't convincing enough, or he failed to figure out the things business cares about & present the identified risk in these terms.
- civilized 2y ago
  Why not go even further? Why not say that the whistleblower was wrong and Microsoft business leadership was right? Maybe their profits from ignoring this issue have been fantastic, and the externalities from e.g. mass theft of national security secrets are not Microsoft's problem.
  - execveat 2y ago
    Well, because as a security person I can only evaluate his actions from the point of security. Evaluating actions of MS business leadership is beyond my expertise. I highly doubt that the senior leadership would willingly accept this kind of liability. But you need to put it into right terms for them to understand. Politics play important role at that level as well. There are ways of putting additional pressure on the c-suite, such as making sure certain keywords are used in writing, triggering input from legal or forcing stakeholders to formally sign off on a presented risk. Without insight knowledge, it's impossible to figure out what went wrong here, so I'm not assigning blame to the whistleblower, just commenting that way too often techies fail to communicate risks effectively.
- cplat 2y ago
  During my Master's, security was one of the subjects I took. It started with an equation that related risk (how much you'd lose if something bad happened), the probability of that risk, and the cost of mitigating that risk. The instruction being, one tries to find a mitigation that costs less than the exploitation of the risk. And note here that "cost" does not refer to just money, but could be computational cost, energy consumed, etc.
hooverd 2y ago
Security first, but security from whom?
JohnMakin 2y ago
The misaligned incentives between security and profit, especially in public companies, is not really a fixable problem without a massive cultural shift. I'm not sure at this point what could even trigger one. I've always dabbled in cybersecurity, taking on the hat in various roles over the years but have refused to go full time into it due to what I have personally seen in the industry - an overwhelming focus on compliance rather than actual good security practices, and the compliance standards are either very lacking or poorly enforced.
- graemep 2y ago
  This is exactly it. There is no incentive to prioritise security. It is not visible to customers, except in terms of compliance, most likely a check-list approach. I think it needs a massive cultural shift, but from customers. If customers were willing to evaluate security (consumers cannot, but enterprise can) properly, demand binding assurances, and make buying choices accordingly industry would respond. Of course MS is too strongly entrenched in the desktop market for this to be completely effective.
  - hulitu 2y ago
    > If customers were willing to evaluate security (consumers cannot, but enterprise can) Where i work, IT is outsourced and decision to buy most of the SW is made by managers who have no idea about computers.
  - wyldberry 2y ago
    When I first left offensive security consulting and joined an internal defensive team, a wise ex-agency person said to me "In product development, the first things to often get axed are security, and performance. They are invisible to the user, until they aren't, and rarely do failures in those areas end a company." Granted this was prior to ransomware really blowing up, but even that itself is a different threat model that doesn't mean your product has to be good at security.
  - fuzzfactor 2y ago
    The purpose of using Microsoft products in an office environment is so that your office can be run with as much personal computer enhancement as you originally realized when you first effectively replaced the traditional office machines or more-labor-intensive tasks with software-powered substitutes. Which all occurred way before any of the things like "single-sign-on" got popular among those who didn't seem to know any better. The second this appeared it was easily recognized as one of the many consumer/entertainment features that must be disabled across every bit of any serious corporate network. Also best disabled on any home computer before it is allowed to touch the internet. There was no forthcoming mitigation, all Microsoft leadership could do was throw up their hands, after all there were unsurmountable reasons why such a threat could not be overcome. >it required customers to turn off one of Microsoft’s most convenient and popular features: Like any other office no-brainer: >the ability to access nearly every program used at work with a single logon. Duh.
thiagoharry 2y ago
Like any other private company? All choices are to maximize profit, even when they spend resources in security, it is to maximize profit.
ChrisMarshallNY 2y ago
I think that when companies sell to the government, there is so much money to be made, and such a huge PR boost, that they are incentivized to cover up the naughty bits (a certain airframe manufacturer, comes to mind). It can mean anything from concealing slightly embarrassing stuff, to massive, systemic, deliberate, fraud; sometimes, the whole spectrum, over time. It often seems to encourage a basic corrosion of Integrity and Ethics, at a fundamental cultural level. When leaders say "Make Security|Quality a priority," but don't actually incentivize it, they set the stage. For example, routinely (as in what is done every day) rewarding or punishing, based on monetary targets, vs. punishing one or two low-level people, every now and then (when caught), says it all. They are serious about money, and not serious at all, about Security|Quality. If you want to meet a goal, you need to incentivize it. Carrots work better than sticks. Sales people get a lot of stress, and can get fired easily, but they can also make a great deal of money, if they succeed. Security people don't get fired, if they succeed, and get fired, if they don't. Often, the result of good work is ... nothing ... No breaches, no disasters, no drama. Hard to measure, as well. How to quantify an absence? Sales: Lots of carrot, and the same stick as everyone else gets. Easy to measure, too. Security: No carrot. All stick. The stick can be a really big stick, too; with nails driven through it. I'm really not sure what the answer is, but it's cultural, and cultural change is always the most difficult thing to change.
- slashtom 2y ago
  I think this is sort of it but I don't think it's the carrot that's the problem here. I believe it's the process and yeah ultimately the culture. I don't think you want sales concerned about security, their focus should and only be on growth. The problem is if you don't give jurisdiction and power to the other side to actually say no this priority (security fix) goes in before work is done on this new feature, then you have an imbalanced system. If the project manager who is incentivized toward growth is the decision-maker for deciding what is prioritized, well of course naturally you'll have the PM choosing growth over security. Process needs fixing, give more agency and jurisdiction to the other side to effect change. It's not like security doesn't see what the issues are, it's just the fixes are not prioritized and the culture and process isn't balanced between both.
- 2y ago
ckozlowski 2y ago
There's a pretty big caveat in this story which I feel is being looked over: "Disabling seamless SSO would have widespread and unique consequences for government employees, who relied on physical “smart cards” to log onto their devices. Required by federal rules, the cards generated random passwords each time employees signed on. Due to the configuration of the underlying technology, though, removing seamless SSO would mean users could not access the cloud through their smart cards. To access services or data on the cloud, they would have to sign in a second time and would not be able to use the mandated smart cards." The U.S. Government (USG) is one of MSFT's largest (if not the largest) customers. The user base is enormous, and the AD footprint equally so. I have experience working in this space; the user and roles management is a nightmare with comprimised credentials, locked out accounts, and the like. Given the nature of their work, it's a constant target. The USG has been attempting to move everyone to smart card auth to help mitigate some of these issues. Removing passwords and turning everyone to two-factor auth would greatly reduce their attack surface. They've been pursuing this for years. So along comes this guy, and he says that, as part of this fix, just tell all of their customers to turn this off. I don't dispute the danger of the original SAML flaw. But I think Harris is unfairly judging the rest of MSFT's reaction here. He's asking them to turn off two-factor auth across entire agencies. I might as well hand an attacker a set of credentials because that's the amount of effort and time they would need to phish a set off someone. To reiterate, the flaw in AD FS was bad and needed immeditate attention. But the short term mitigation Harris proposes would drastically hurt their security and open tons of customers to attacks of the very sort they were trying to prevent. This story is spun as another instance of a company not caring about security, but I see a "whistleblower" who had a very narrow view of their customers overall security posture, and threw a fit when this was pointed out to him. "To access services or data on the cloud, they would have to sign in a second time and would not be able to use the mandated smart cards. Harris said Morowczynski rejected his idea, saying it wasn’t a viable option." I would fully expect most government agency Info Sec Systems Managers (ISSMs) to say the same.
- foota 2y ago
  I mean... I guess the issue here is more that Microsoft didn't make customers aware of this flaw, and continued to sell the service. Which... is exactly the articles point. They knew there was no secure way to administer it, and yet sold it anyway.
stratigos 2y ago
This comes off like a study being published that shows tobacco is harmful to the lungs.
photochemsyn 2y ago
Yes, it's called investment capitalism - as long as the consequences of actions one demanded are never felt by oneself, due to limited liability of the financiers and shareholders, then such behavior will never change. The solutions are well known - the corporate death penalty is a good one, which dissolves the legal and financial structures of the company (the real assets such as factories are unharmed by this, and may simply be sold to a new more reliable set of financiers and shareholders, or may be nationalized and managed by the state, or may be handed over to the workers who run the place to see if they can form an employee-owned company or not, etc.). This isn't such a radical viewpoint, even many venture capitalists agree that this is the right way to go, e.g. on the airlines: https://www.cnbc.com/video/2020/04/13/government-should-let-airlines-fail-venture-capitalist-chamath-palihapitiya.html https://www.cnbc.com/video/2020/04/13/government-should-let-...
SebFender 2y ago
We needed an article to make sure this was clear.
nonrandomstring 2y ago
Surely, a whistleblower is someone who reveals a truth that nobody knows?
skilled 2y ago
The hearing will be streamed on YouTube in ten minutes from this comment: https://www.youtube.com/watch?v=kB2GCmasH4c https://www.youtube.com/watch?v=kB2GCmasH4c
- qintl55 2y ago
  Gosh, I'm watching this now... the amount of bullshit from Smith is ...wow!
say_it_as_it_is 2y ago
Business decisions involve profits against everything, not just security. Delaying shipments to make a product more secure can affect revenue targets.
MattSteelblade 2y ago
So...Golden SAML isn't a vulnerability, as the CyberArk article quoted in the post reiterates, it's a type of attack that requires completely comprising the box before using. Unless I am misunderstanding something, I don't see any particular flaw, per se. As Microsoft (mocked in the article) would say, it's not crossing a security boundary. SSO will ALWAYS have this particular tradeoff. If your SSO infrastructure is compromised, everything that uses it is at risk of being compromised.
- spdgg 2y ago
  Sounds like the vulnerability was one within AD FS and that exposed the private key, making golden SAML possible.
  - MattSteelblade 2y ago
    It was the SolarWinds hack that gave internal access and potential admin rights. It's no different than if a domain controller gets compromised. The attacker has gained control of the keys to kingdom; it's an inherent risk to SSO.
- duncans 2y ago
  Yes, it requires getting admin to the AD FS server https://www.netwrix.com/golden_saml_attack.html https://www.netwrix.com/golden_saml_attack.html which is kind of glossed over but surely is the real "hack"?
- worik 2y ago
  > If your SSO infrastructure is compromised, everything that uses it is at risk of being compromised. Really? Can you not think of any approach that gives SSO and accountability? I think there are
- artjomb 2y ago
  Exactly! AD FS is part of Tier 0 in the same way as Active Directory itself and needs to be treated and secured as such. Of course, security goes a long way when it's part of a holistic approach like zero trust. Mitigation is also not really possible when using SSO. One way would be to require the target service to require a second factor in addition to a valid SAML token, but then each user needs to keep current its second factor, whatever it might be, in each target service. This get unmanageable quite quick not to mention that there are basically no SaaS or self-hosted applications out there that support SSO and a second factor at the same time.
mihaaly 2y ago
I observed they chose profit over usability and user needs as well (the list is toooo long, I save all of us from pouring all here, let's say I am contemplating getting a completely different job where I do not have to run circles around the way Windows is corrupted), so this fits into the big picture afterall.
mikeegg1 2y ago
Will the whistle blower end up the same way as Boeing whistle blowers? "See something; say something."
dogman144 2y ago
Not unique to MSFT. I’m a security engineer. If you want sanity paired with outcomes in the career, work at places that are technical and have a strong regulatory incentive and related funding, or a strong threat model closely tied to profits to care about security culturally. Main examples for me that hit that are: - pre-IPO startups that want to pass SOC2 etc to go public: have the reg and profit incentive and pay to buy a security team from scratch - crypto: has the threat model and profit incentive due to key theft and so on. Pays well too and great risk space to test out sec skills - public tech cos providing a lot of critical infra: to an extent, some can veer into Too Big to Fail like MSFT, some have stronger internal sec teams like Google/Project Zero, Verizon/Paranoids, Cloudflare seems good. - Banking is maybe: they have funds, more risk-averse culture, heavily regulated. But healthcare is also heavily regulated and id never work in it due to the volume of exploits and lack of care. So ya, don’t work at MSFT as a sec eng IMO unless you’re on the DART team and want to see a lot of diverse incident response with legit threat actors, or want to do really low level OS sec. No idea about Apple sec eng work, on this note. This is also why the avg tenure in security careers +/- 10 years. Your sanity runs out and often pay is good enough where you can save up and do something else with your life by 30/40.
mikl 2y ago
Not the most earth-shattering revelation. Given how they’ve always needed to be dragged kicking and screaming to adopt more secure protocols, that should be obvious. For example, it’s been known that NTLM is easily cracked for over a decade, but they’re only getting around to phasing it out now.
pdimitar 2y ago
RE title: and somebody also said -- "The sun is still rising from the east". And after pulling their white beard, said wisely: "And water is still wet".
chucke1992 2y ago
I mean, I see the crappy level of security across the whole industry. And I am working at the bank. I am not sure what exactly the reason - even if the profit aside - but I suspect that there are not many people who are actually competent developer and security engineers.
shortsunblack 2y ago
The solution is complete zero trust and distrusting the network in organizations. You should treat the internal network as external -- hostile. Google does this. They were the first ones to widely adopt zero trust with BeyondCorp and there has not been a Google internal organizational breach since Aurora (which made them adopt BeyondCorp, what they call zero trust). You have completely managed endpoints, strong hardening of the endpoint and complete inventorization of all the resources in the organization. You have certificates installed onto each device. You have an ACL engine that determines whether a user should get access to a particular resource. You can use deterministic lists and also incorporate heuristics to detect anomalies (working hours, etc). All Google internal apps are internet-facing. You can open them, get redirected to the SSO portal. Come and do try to get in. You will not. Many of these security problems are solved. You just need to implement the solutions.
- orf 2y ago
  > You have completely managed endpoints, strong hardening of the endpoint and complete inventorization of all the resources in the organization. You have certificates installed onto each device. You have an ACL engine that determines whether a user should get access to a particular resource. None of those are “solved” for any mid or large-sized enterprise where tech isn’t their code competency. In fact, I’d say most of these are insurmountably hard. This is a “draw the whole owl” kind of response. Its very well to say that you can do things differently, but imagine Shaw Industries (22k employees, largest carpet/flooring manufacturer in the USA) doing any of that.
  - alkonaut 2y ago
    I think I agree with this conclusion. But I work in a Shaw-like enterprise (only the products are more mundane than flooring). What are the hurdles we’d see if we tried it? What processes and practices are we likely using, that would break under the zero trust model?
    - zach_miller 2y ago
      I think the hard part of trying it isn't using it, it's implementing it.
m463 2y ago
I think it's the nature of Microsoft. They've done things quick instead of well. This has served them well, and during the time microsoft has been around, most of the competitors that have reversed that equation have gone by the wayside. I vaguely recall they were in the same boat decades ago with the win 3.x and win 9x - windows was a virus and bluescreen laden garbage heap. I'm not sure what OS they started really cleaning up and validating the API calls. I think windows 2000 was a major step away from shared memory space to cure some of it.
6031769 2y ago
Bear shat in woods, whistleblower says.
madrox 2y ago
I don't see a future here that doesn't involve significant legislation over network security and include jail time for major offenses. Every time something like this happens, there's always that organizational Cassandra (usually the CISO) that saw it all coming but was ignored. Sooner or later someone will get burned badly enough that the consensus will be that tech cannot regulate itself on security. We've already got this a little bit for the most egregious cases, but it's still about as secure as banks in the 1920s. A less actionable gripe I have is that we have so few players that even if the US government loses trust in Microsoft's cloud...where else will they go? There aren't a lot of players here that could handle that scale. It's like if there were only 3 banks in the world.
- Zelphyr 2y ago
  I can think of one solution to the "too few players" problem. Break them up.
  - madrox 2y ago
    I, too, wouldn't mind living in that timeline, but that's just not going to happen for a number of reasons that are so obvious as to not be worth enumerating. Dwelling on unrealistic solutions prevents you from perceiving the possible.
ederamen 2y ago
Obviously
userbinator 2y ago
Of course when Microsoft does choose "security", it's often in the name of securing things against users.
worik 2y ago
Is it possible that insurance companies will play a role in this endless succession of insecure, critical, software systems? Can you get insurance against software system failure? If you cn, surely the providers of said insurance will take a keen interest?
Splice9160 2y ago
Shocked! I am SHOCKED....
animanoir 2y ago
Well, that’s what a company like Microsoft will always choose, so no wrong-doing here—business as usual
AdeptusAquinas 2y ago
"[Enter Company Name] Chose Profit over Security, Whistleblower Says". Any company, you pick.
o999 2y ago
Capitalism ate my face
exmsrc 2y ago
"I was very interested in that question. And one of the places that I focused on was the MSRC, which is short for Microsoft Security Response Center. This center is like a clearing house for reports of security bugs, and it was Harris' very first stop when he began warning colleagues of the flaw that he discovered. But the issue is that the center itself was understaffed and underresourced. And one employee who used to work there told me that staff is trained to think of cases in terms of how can I get to won't fix. So this center also clashed with the product teams." I used to work for the MSRC. He's right that it was understaffed and underresourced. It's one of the reasons I quit, same for many of my ex-colleagues. But I disagree with his characterization of us trying to find any way to get cases to Won't Fix. The fact is, we got many, many reports that were genuinely not vulns, and therefore shouldn't be prioritized for fixing from a security standpoint. Yes, occasionally reports may be incorrectly analyzed but that's not because we were trying to get them to Won't Fix. It's just people making mistakes now and then. "And, you know, another big issue there is that they're clashing with the product teams that they need to fix the actual issues. So they would bring a security vulnerability to a product group. They'd say, you need to fix this flaw. But those groups were often unmotivated to act fast, if at all, because compensation is tied to the release of new products and features." That's true in part, but it varies wildly between product teams. Some were incredibly responsive and knowledgeable, some were clueless about security, some just didn't prioritize it. Sometimes the fix was insufficient. When I was there, MSRC wouldn't check if the fix did what it was supposed to do, except in occasional cases where we were explicitly asked to check or if it was a particularly risky case that needed the extra scrutiny. But like he says, we were understaffed and underresourced, we simply didn't have the time to do this for every case.
mike503 2y ago
Boeing chose profits over safety. All companies choose profits over <literally anything> That's unfettered capitalism.
flybarrel 2y ago
> Product managers had little motivation to act fast, if at all, since compensation was tied to the release of new, revenue-generating products and features Only if I get rewarded that way lolllllll
ViktorRay 2y ago
”They saw it differently, Harris said. The federal government was preparing to make a massive investment in cloud computing, and Microsoft wanted the business. Acknowledging this security flaw could jeopardize the company’s chances, Harris recalled one product leader telling him. The financial consequences were enormous. Not only could Microsoft lose a multibillion-dollar deal, but it could also lose the race to dominate the market for cloud computing.” There is something fundamentally broken about an organization’s culture when this type of thinking is pervasive in the organization.

676 points by tyleroconnell 2y ago

oh lets put fonts in the user space rather then the kernel space what could ever go wrong? this not new its a major feature of how MS works

1vuio0pswjnm7 2y ago

Sounds like the same Microsoft culture as has always been. Like a cult. It can do no wrong. The conversation with Microsoft businesspeople at conferences was always the same: Microsoft has no deficiencies, there is nothing it isn't working on and it has a solution for every possible problem. Other sources of software do not exist. There is only Microsoft. Total illusion put forth by delusional employees. The outside world can be ignored because life in the cult is good.

Drakim 2y ago
Don't worry, Microsoft has big ambitions and huge plans about how to truly present themselves as more safety oriented in the future.
- NekkoDroid 2y ago
  AI Safety. Code is run through AI to check for vulnerabilities. Files are analyzed by AI to ensure they aren't malware. Every instruction is run through AI to ensure nothing maliciously is happening (mostly enforcing DRM :). Every pixel is output by AI to ensure you see nothing not intended for your precious eyes.
  - freedomben 2y ago
    You joke, but (the DRM part at least) is the future I fear is coming. It could hit us from so many angles (not forgetting Chrome's Web Environment Integrity and Apple's Private Access Tokens), and with all the money and power behind it (big tech plus big copyright), and the complete apathy of the average user towards this, it seems inevitable.
    - NekkoDroid 2y ago
      The DRM part wasn't really part of the joke, just a sad truth that is being worked on more and more that sadly fit into the joke.
eganist 2y ago
If memory over the last two decades serves, this is a relatively recent degradation. Microsoft's security reputation prior to the recent (5ish years?) failures was largely built up on top of the work stemming from the Trustworthy Computing memo. https://www.wired.com/2002/01/bill-gates-trustworthy-computing/ https://www.wired.com/2002/01/bill-gates-trustworthy-computi...
- diggan 2y ago
  Bill Gates in 2002: "So now, when we face a choice between adding features and resolving security issues, we need to choose security." Satya Nadella in 2024: "If you’re faced with the tradeoff between security and another priority, your answer is clear: Do security." Microsoft in 2024: Run this software on your computer so we can take a screenshot of everything you do, index it and we promise Security is still, and have always been, the priority. And yes, we do store data unencrypted on your disk, why are you asking?
  - NekkoDroid 2y ago
    > And yes, we do store data unencrypted on your disk, why are you asking? But don't worry, you need to be an administrator to open the file. What? your average person daily drives an administrator account? How should we have known that???
freedomben 2y ago
This comment sounds hyperbolic, but it really isn't. It's really bad. This has been my experience with Microsoft employees also. In my experience, what makes for bad software is PM and engineering hubris. You definitely need some vision and confidence as just following user feedback is a recipe for terrible software as well. The key is to find the right balance and straddle that line. If it's been long enough for insiders to tell the story of Windows Phone and the eventual cancellation, I'd be fascinated to hear the story of that (from inception to death) and how that went internally given the culture.
- tracker1 2y ago
  Just wanted to say that I thought the Windows Phone (the last version of such) was relatively nice. It had a decent developer experience, but was pretty much an also ran and didn't have enough market share to overcome mindshare for first party apps. When so many first apps were iOS first and Android later, throwing a third option in the mix just missed the mark more often than not. I was already in the Android ecosystem and far less cynical at that point about Google.
  - Ylpertnodi 2y ago
    I didn't get a Windows phone because i don't trust Microsoft. A friend had one and it was really ok, but no way for me.
    - tracker1 2y ago
      In retrospect, I don't trust Apple or Google either...
      - fsflover 2y ago
        So you need a GNU/Linux phone?
- fingerlocks 2y ago
  Can confirm, it is 100% hubris based on my limited time of working at Microsoft. There is pervasive NIH syndrome, re-inventing the wheel, and massive amounts of over engineering and unnecessary abstraction caused by chasing the endless "But what if...?" dragon. This behavior is justified, and critics are silenced, by the "But we're an enterprise company!" cop-out
jan3024 2y ago
[dead]
diggan 2y ago
To be honest, that sounds like every company that suffers from delusions of grandeur and wants to conquer the planet, one way or another. What you're saying is equally true for Apple, Google, Amazon and most other public companies today. You're never gonna get "Use this Microsoft product" as an answer from Apple support/engineer even if that product would solve your particular problem better.
- 1vuio0pswjnm7 2y ago
  The conferences always included representatives from other large, public companies in the same or similar industries, e.g., Apple or Amazon, as well as other, different industries. But there was always something cult-like about the folks from Microsoft, their level of BS and (deliberate?) ignorance, that I never experienced with the others. To be clear, I could not make the same comment about Apple or Amazon businesspeople. While they may exhibit their own stigmatic qualities, they are, IME, different. Nothing like Microsoft. Microsoft does not suffer from "delusions of grandeur". It achieved grandeur a long time ago, and then became delusional. Currently, it is either #1 or #2 on the list of the world's wealthiest companies. Comments suggesting that the company has "changed", and such comments have been popular on HN in recent years, are quite amusing.
surfingdino 2y ago
I'll give you a 4-letter word... Zune /s
magicalhippo 2y ago
What Microsoft can provide are lots of nice stickers saying they conform to this or that security standard, making security folks in IT departments all warm and fuzzy. At least that's how it appears from our POV, selling B2B applications. They don't seem to care that much about actualities as long as the security checklist passes.

deleted 2y ago

[deleted]

everdrive 2y ago

I'm not defender of Microsoft, but I don't know if I could point to any company which does not put profit over security.

diggan 2y ago
I guess the issue becomes when they say security is the top priority (and have been for two decades), yet all actions point towards it not being so. > Bill Gates in 2002: "So now, when we face a choice between adding features and resolving security issues, we need to choose security." https://www.wired.com/2002/01/bill-gates-trustworthy-computing/ https://www.wired.com/2002/01/bill-gates-trustworthy-computi... > Satya Nadella in 2024: "If you’re faced with the tradeoff between security and another priority, your answer is clear: Do security." https://www.theverge.com/24148033/satya-nadella-microsoft-security-memo https://www.theverge.com/24148033/satya-nadella-microsoft-se...
- ls65536 2y ago
  Obviously, nobody is going to outright admit they put profits above security; indeed, they will often state the opposite. But their closely-held beliefs will shine through when it comes time to make decisions and the outcomes of those decisions are exposed to their customers and to the public.
  - lesuorac 2y ago
    Does Bill or Satya write code anymore? It could very well be that they consider security the top priority but it's a moot point because they're so removed from operations. Although I would suspect that you're effectively right in that they either don't have it as a top priority or think they do but have a reveal preference of they don't. For example, an engineer that does rigorous security testing and finds nothing as well as launches one project gets promoted less often than an engineer that launches two projects and doesn't do rigorous security testing.
    - deleted 2y ago
      [deleted]

hypeatei 2y ago

Execs should absolutely be held responsible, but the human factor is always there. Many times people will take the easy route and get worn down by security practices or roadblocks. I think it's too easy to go "alright focus on security" and then expect it trickle down and figure itself out.

cellu 2y ago

surprisedpikachu.jpg

ThinkBeat 2y ago

This whole article seems a bit odd to me. What is "the product" ? Presumably this is not related to earlier problems with SolarWinds. Did MS screw up. Yes. However, all things have bugs. I takes one person finding one bug and exploiting it. and there are enormous resources going into finding one, and I am certain that this is the only one. I am sure the NSA is sitting on a pile of them. Whereas the developers have to think about everything that can happen and protect against it. Does this make Microsoft different from its competitors? I think Microsofts strategy is somewhat similar to Linus: Where security patches are often not part of new releases due to the burden of establishing what the consequences of bigger changes would be, and the fact that security people dont do sane things. (But you can of course pull them and make it part of an in-house distro. https://lkml.iu.edu/hypermail/linux/kernel/1711.2/01357.html https://lkml.iu.edu/hypermail/linux/kernel/1711.2/01357.html

pgraf 2y ago
It is true that nothing is 100% secure. Sitting on a major security vulnerability internally with a motivated employee pushing to fix it and doing nothing for business reasons is not negligence, but malice. People in the chain of command need to be held accountable for this.
latexr 2y ago
> However, all things have bugs. There are bugs and there are critical flaws you’ve been warned about. This is the latter. The fact that this was known by Microsoft but not fixed is the story.
SuchAnonMuchWow 2y ago
> Harris said he pleaded with the company for several years to address the flaw in the product, a ProPublica investigation has found. But at every turn, Microsoft dismissed his warnings, telling him they would work on a long-term alternative — leaving cloud services around the globe vulnerable to attack in the meantime. That is not a screw-up, that is a deliberate decision.
shkkmo 2y ago
Microsoft had a known, high consequence, security flaw that they did not acknowledge or fix, they had evidence that indicated it had already been exploited and they knew they had limited to no ability monitor for exploitation. This choice lead directly to the SolarWinds hack that happened in 2019 was discovered in late 2020 and acknowledged by the USG in early 2021. Many companies make bad choices around security for profit, however that factors I listed above make this extremely egregious. I would seriously question any use of Microsoft products in any security conscious organization after this reveal. I also hope that anyone negatively effected by the Solar Winds sue Microsoft for knowing about the vulnerability for years without fixing it or disclosing it.

outside1234 2y ago

Unless that other priority is laying people off. Then the layoffs are more important.

minisooftwin 2y ago

> “If you’re faced with the tradeoff between security and another priority, your answer is clear: Do security,” the company’s CEO, Satya Nadella, told employees. Satya's model of making security a priority at Microsoft: - Cram ads in every nook and corner of Windows. Left, right, centre, back, front, everywhere. What else is an operating system for? - Install a recorder which records everything you do. For the benefit of users of course - you know, what if a user missed an ad and wants to go back and see what they missed. - Send a mail to your employees and tell them "Do security". Mission accomplished - Microsoft is now the most secure platform.

VyseofArcadia 2y ago
The Microsoft bribes scandal broke not too long after I had to take the "hey don't do bribes" training at Microsoft. That event really drove home for me the fact that all of the trainings, emails, processes, etc. are mostly plausible deniability. There are people who care about security at MS. I know, I've met them, but for the most part all of this exists so that Satya can plausibly say in court or in front of congress, "well we told them to do security better. This is clearly the fault of product teams or individual contributors, not Microsoft policy and incentives."
- montjoy 2y ago
  I dunno, that’s a pretty cynical take. Isn’t it just as plausible that they became aware of the bribes internally and were trying to curtail them when the scandal broke out? Or maybe the “don’t do bribes” training actually worked enough for someone to whistleblow even if official internal channels failed? Those who are doing wrong often try to stymie others from making positive changes out of fear, greed, etc. Edit: I just want to add that there are things to be cynical about - I’m not completely naive. If it’s your legal department heading up the training then you can be pretty sure that there was a cause for it.
  - blowski 2y ago
    Yes, massive companies are a nest of conflicting priorities. The sales team wants to do whatever it takes to win the deal, and the legal team wants everyone to behave ethically at all times. The board wants to be shocked(!) when it turns out those goals are in conflict, with the ethical side sometimes losing out, to remove any personal risk to themselves.

spydum 2y ago

As per usual, executive platitudes around "security first" don't matter. If you pay and promote people for features, and don't reward security culture, people are not dumb: they and the management layers will optimize for that. I don't know how to design incentives to solve for this, but this is always going to be the way it is.

olivierduval 2y ago
I think that it could be "security as a feature" Usually, a feature is included in a product if the marketing show that it will grow the business more than the cost of the feature. Maybe we can try the same idea ? "We identified this vulnerability, and it will impact X % of our customer and Y % will leave (+ reputation damage) so we will loose BIGNUMBER $. However, we can correct it for SMALLNUMBER $ in Z days. Decision ?"
- deleted 2y ago
  [deleted]
- nicce 2y ago
  Real security cannot be feature. Your complete system design and other features should be based on the idea of ”security first”, if you really want to build secure systems.
  - hulitu 2y ago
    > Your complete system design and other features should be based on the idea of ”security first”, if you really want to build secure systems. One can argue that the most secure system is the one turned off and not used. And i am not talking about devices with builtin batteries.
    - nicce 2y ago
      One can always argue that, but, fundamentally security is about limiting the systems' use for its purpose and eliminate all unwanted scenarios. If you need to use the system, you cannot turn it off or not to use it.

pgraf 2y ago

Imagine a major bridge that was built by a contractor. A internal safety inspector repeatedly warned his supervisors of structural deficiencies that could lead to the collapse of the bridge. Furthermore, in the pass of time two external sources publicly warned about the issue, but the company downplayed the importance. Finally, the bridge collapses. It becomes evident that the company did nothing about the issue because it didn‘t want to loose contracts selling more flawed bridges. The public would justifiably go nuts, and there would be legal consequences for everyone involved. What is different in our industry that companies (and managers) get away with such malice?

deleted 2y ago
[deleted]
red_admiral 2y ago
Wasn't there something a bit like that with the Morandi bridge that collapsed in Italy? (There was definitely something like that with the Mottarone cable car that had been running for years with the safety catch disabled. When the tow-rope snapped, wiht no catch, the cabin rushed down and killed everyone on board.)
natsucks 2y ago
I don't understand how this doesn't destroy a company. They willfully ingored a serious risk and it had major national security implications.
- dfedbeef 2y ago
  Have you tried to use Google customer support
magicalhippo 2y ago
Here in Norway a bridge built with known structural deficiencies did in fact collapse[1], and basically nothing has happened except tax payers get to pay even more for a new bridge. Unless enough lives are lost, people generally don't care that much it seems. [1]: https://www.nrk.no/innlandet/statens-vegvesen-legg-fram-rapport-etter-at-tretten-bru-kollapsa-1.16721805 https://www.nrk.no/innlandet/statens-vegvesen-legg-fram-rapp...

xyst 2y ago

> “If you’re faced with the tradeoff between security and another priority, your answer is clear: Do security,” corporate morality is a Potemkin village. It's all about the profit and appeasing the shareholder, baby! is anybody honestly surprised at this point? The abbreviation of "M$" is well deserved despite small OSS contributions and attempts to PR their way out of previous history (ie, United States v. Microsoft Corp. [2001])

winocm 2y ago
The original sins: https://www.nytimes.com/1970/09/13/archives/a-friedman-doctrine-the-social-responsibility-of-business-is-to.html https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr... https://www.sciencedirect.com/science/article/pii/0304405X7690026X https://www.sciencedirect.com/science/article/pii/0304405X76...

erand293 2y ago

I worked at Microsoft the entire time period covered in this article. I've interacted with MSRC on multiple issues with the product team I'm on. My experience inside Microsoft is not at all consistent with what the whisteblower is saying, for what it's worth. It's completely unrecognizable, in fact - and I'm having a hard time reconciling what I'm seeing with what I'm reading here.

execveat 2y ago

I work in infosec, and this sounds like a communication failure on the whistleblower's part. Contrary to what many people believe, the profits should be prioritized over security for the most companies, that's only natural (after all, they don't generate any profits themselves, typically). The key is finding the right balance for this tradeoff. Business leaders are the ones that are responsible for figuring out the acceptable risk level. They already deal with that every day, so it's nonsensical to claim they aren't capable of understanding risk. InfoSec's role for the most part is being a good translator, by identifying the technical issues (vulnerabilities, threats, missing best practices) that go beyond the acceptable risk profile and to present these findings to the business stakeholders, using the language they understand. Either the guy wasn't convincing enough, or he failed to figure out the things business cares about & present the identified risk in these terms.

civilized 2y ago
Why not go even further? Why not say that the whistleblower was wrong and Microsoft business leadership was right? Maybe their profits from ignoring this issue have been fantastic, and the externalities from e.g. mass theft of national security secrets are not Microsoft's problem.
- execveat 2y ago
  Well, because as a security person I can only evaluate his actions from the point of security. Evaluating actions of MS business leadership is beyond my expertise. I highly doubt that the senior leadership would willingly accept this kind of liability. But you need to put it into right terms for them to understand. Politics play important role at that level as well. There are ways of putting additional pressure on the c-suite, such as making sure certain keywords are used in writing, triggering input from legal or forcing stakeholders to formally sign off on a presented risk. Without insight knowledge, it's impossible to figure out what went wrong here, so I'm not assigning blame to the whistleblower, just commenting that way too often techies fail to communicate risks effectively.
cplat 2y ago
During my Master's, security was one of the subjects I took. It started with an equation that related risk (how much you'd lose if something bad happened), the probability of that risk, and the cost of mitigating that risk. The instruction being, one tries to find a mitigation that costs less than the exploitation of the risk. And note here that "cost" does not refer to just money, but could be computational cost, energy consumed, etc.

hooverd 2y ago

Security first, but security from whom?

JohnMakin 2y ago

The misaligned incentives between security and profit, especially in public companies, is not really a fixable problem without a massive cultural shift. I'm not sure at this point what could even trigger one. I've always dabbled in cybersecurity, taking on the hat in various roles over the years but have refused to go full time into it due to what I have personally seen in the industry - an overwhelming focus on compliance rather than actual good security practices, and the compliance standards are either very lacking or poorly enforced.

graemep 2y ago
This is exactly it. There is no incentive to prioritise security. It is not visible to customers, except in terms of compliance, most likely a check-list approach. I think it needs a massive cultural shift, but from customers. If customers were willing to evaluate security (consumers cannot, but enterprise can) properly, demand binding assurances, and make buying choices accordingly industry would respond. Of course MS is too strongly entrenched in the desktop market for this to be completely effective.
- hulitu 2y ago
  > If customers were willing to evaluate security (consumers cannot, but enterprise can) Where i work, IT is outsourced and decision to buy most of the SW is made by managers who have no idea about computers.
- wyldberry 2y ago
  When I first left offensive security consulting and joined an internal defensive team, a wise ex-agency person said to me "In product development, the first things to often get axed are security, and performance. They are invisible to the user, until they aren't, and rarely do failures in those areas end a company." Granted this was prior to ransomware really blowing up, but even that itself is a different threat model that doesn't mean your product has to be good at security.
- fuzzfactor 2y ago
  The purpose of using Microsoft products in an office environment is so that your office can be run with as much personal computer enhancement as you originally realized when you first effectively replaced the traditional office machines or more-labor-intensive tasks with software-powered substitutes. Which all occurred way before any of the things like "single-sign-on" got popular among those who didn't seem to know any better. The second this appeared it was easily recognized as one of the many consumer/entertainment features that must be disabled across every bit of any serious corporate network. Also best disabled on any home computer before it is allowed to touch the internet. There was no forthcoming mitigation, all Microsoft leadership could do was throw up their hands, after all there were unsurmountable reasons why such a threat could not be overcome. >it required customers to turn off one of Microsoft’s most convenient and popular features: Like any other office no-brainer: >the ability to access nearly every program used at work with a single logon. Duh.

thiagoharry 2y ago

Like any other private company? All choices are to maximize profit, even when they spend resources in security, it is to maximize profit.

ChrisMarshallNY 2y ago

I think that when companies sell to the government, there is so much money to be made, and such a huge PR boost, that they are incentivized to cover up the naughty bits (a certain airframe manufacturer, comes to mind). It can mean anything from concealing slightly embarrassing stuff, to massive, systemic, deliberate, fraud; sometimes, the whole spectrum, over time. It often seems to encourage a basic corrosion of Integrity and Ethics, at a fundamental cultural level. When leaders say "Make Security|Quality a priority," but don't actually incentivize it, they set the stage. For example, routinely (as in what is done every day) rewarding or punishing, based on monetary targets, vs. punishing one or two low-level people, every now and then (when caught), says it all. They are serious about money, and not serious at all, about Security|Quality. If you want to meet a goal, you need to incentivize it. Carrots work better than sticks. Sales people get a lot of stress, and can get fired easily, but they can also make a great deal of money, if they succeed. Security people don't get fired, if they succeed, and get fired, if they don't. Often, the result of good work is ... nothing ... No breaches, no disasters, no drama. Hard to measure, as well. How to quantify an absence? Sales: Lots of carrot, and the same stick as everyone else gets. Easy to measure, too. Security: No carrot. All stick. The stick can be a really big stick, too; with nails driven through it. I'm really not sure what the answer is, but it's cultural, and cultural change is always the most difficult thing to change.

slashtom 2y ago
I think this is sort of it but I don't think it's the carrot that's the problem here. I believe it's the process and yeah ultimately the culture. I don't think you want sales concerned about security, their focus should and only be on growth. The problem is if you don't give jurisdiction and power to the other side to actually say no this priority (security fix) goes in before work is done on this new feature, then you have an imbalanced system. If the project manager who is incentivized toward growth is the decision-maker for deciding what is prioritized, well of course naturally you'll have the PM choosing growth over security. Process needs fixing, give more agency and jurisdiction to the other side to effect change. It's not like security doesn't see what the issues are, it's just the fixes are not prioritized and the culture and process isn't balanced between both.
2y ago

ckozlowski 2y ago

There's a pretty big caveat in this story which I feel is being looked over: "Disabling seamless SSO would have widespread and unique consequences for government employees, who relied on physical “smart cards” to log onto their devices. Required by federal rules, the cards generated random passwords each time employees signed on. Due to the configuration of the underlying technology, though, removing seamless SSO would mean users could not access the cloud through their smart cards. To access services or data on the cloud, they would have to sign in a second time and would not be able to use the mandated smart cards." The U.S. Government (USG) is one of MSFT's largest (if not the largest) customers. The user base is enormous, and the AD footprint equally so. I have experience working in this space; the user and roles management is a nightmare with comprimised credentials, locked out accounts, and the like. Given the nature of their work, it's a constant target. The USG has been attempting to move everyone to smart card auth to help mitigate some of these issues. Removing passwords and turning everyone to two-factor auth would greatly reduce their attack surface. They've been pursuing this for years. So along comes this guy, and he says that, as part of this fix, just tell all of their customers to turn this off. I don't dispute the danger of the original SAML flaw. But I think Harris is unfairly judging the rest of MSFT's reaction here. He's asking them to turn off two-factor auth across entire agencies. I might as well hand an attacker a set of credentials because that's the amount of effort and time they would need to phish a set off someone. To reiterate, the flaw in AD FS was bad and needed immeditate attention. But the short term mitigation Harris proposes would drastically hurt their security and open tons of customers to attacks of the very sort they were trying to prevent. This story is spun as another instance of a company not caring about security, but I see a "whistleblower" who had a very narrow view of their customers overall security posture, and threw a fit when this was pointed out to him. "To access services or data on the cloud, they would have to sign in a second time and would not be able to use the mandated smart cards. Harris said Morowczynski rejected his idea, saying it wasn’t a viable option." I would fully expect most government agency Info Sec Systems Managers (ISSMs) to say the same.

foota 2y ago
I mean... I guess the issue here is more that Microsoft didn't make customers aware of this flaw, and continued to sell the service. Which... is exactly the articles point. They knew there was no secure way to administer it, and yet sold it anyway.

stratigos 2y ago

This comes off like a study being published that shows tobacco is harmful to the lungs.

photochemsyn 2y ago

Yes, it's called investment capitalism - as long as the consequences of actions one demanded are never felt by oneself, due to limited liability of the financiers and shareholders, then such behavior will never change. The solutions are well known - the corporate death penalty is a good one, which dissolves the legal and financial structures of the company (the real assets such as factories are unharmed by this, and may simply be sold to a new more reliable set of financiers and shareholders, or may be nationalized and managed by the state, or may be handed over to the workers who run the place to see if they can form an employee-owned company or not, etc.). This isn't such a radical viewpoint, even many venture capitalists agree that this is the right way to go, e.g. on the airlines: https://www.cnbc.com/video/2020/04/13/government-should-let-airlines-fail-venture-capitalist-chamath-palihapitiya.html https://www.cnbc.com/video/2020/04/13/government-should-let-...

SebFender 2y ago

We needed an article to make sure this was clear.

nonrandomstring 2y ago

Surely, a whistleblower is someone who reveals a truth that nobody knows?

skilled 2y ago

The hearing will be streamed on YouTube in ten minutes from this comment: https://www.youtube.com/watch?v=kB2GCmasH4c https://www.youtube.com/watch?v=kB2GCmasH4c

qintl55 2y ago
Gosh, I'm watching this now... the amount of bullshit from Smith is ...wow!

say_it_as_it_is 2y ago

Business decisions involve profits against everything, not just security. Delaying shipments to make a product more secure can affect revenue targets.

MattSteelblade 2y ago

So...Golden SAML isn't a vulnerability, as the CyberArk article quoted in the post reiterates, it's a type of attack that requires completely comprising the box before using. Unless I am misunderstanding something, I don't see any particular flaw, per se. As Microsoft (mocked in the article) would say, it's not crossing a security boundary. SSO will ALWAYS have this particular tradeoff. If your SSO infrastructure is compromised, everything that uses it is at risk of being compromised.

spdgg 2y ago
Sounds like the vulnerability was one within AD FS and that exposed the private key, making golden SAML possible.
- MattSteelblade 2y ago
  It was the SolarWinds hack that gave internal access and potential admin rights. It's no different than if a domain controller gets compromised. The attacker has gained control of the keys to kingdom; it's an inherent risk to SSO.
duncans 2y ago
Yes, it requires getting admin to the AD FS server https://www.netwrix.com/golden_saml_attack.html https://www.netwrix.com/golden_saml_attack.html which is kind of glossed over but surely is the real "hack"?
worik 2y ago
> If your SSO infrastructure is compromised, everything that uses it is at risk of being compromised. Really? Can you not think of any approach that gives SSO and accountability? I think there are
artjomb 2y ago
Exactly! AD FS is part of Tier 0 in the same way as Active Directory itself and needs to be treated and secured as such. Of course, security goes a long way when it's part of a holistic approach like zero trust. Mitigation is also not really possible when using SSO. One way would be to require the target service to require a second factor in addition to a valid SAML token, but then each user needs to keep current its second factor, whatever it might be, in each target service. This get unmanageable quite quick not to mention that there are basically no SaaS or self-hosted applications out there that support SSO and a second factor at the same time.

mihaaly 2y ago

I observed they chose profit over usability and user needs as well (the list is toooo long, I save all of us from pouring all here, let's say I am contemplating getting a completely different job where I do not have to run circles around the way Windows is corrupted), so this fits into the big picture afterall.

mikeegg1 2y ago

Will the whistle blower end up the same way as Boeing whistle blowers? "See something; say something."

dogman144 2y ago

Not unique to MSFT. I’m a security engineer. If you want sanity paired with outcomes in the career, work at places that are technical and have a strong regulatory incentive and related funding, or a strong threat model closely tied to profits to care about security culturally. Main examples for me that hit that are: - pre-IPO startups that want to pass SOC2 etc to go public: have the reg and profit incentive and pay to buy a security team from scratch - crypto: has the threat model and profit incentive due to key theft and so on. Pays well too and great risk space to test out sec skills - public tech cos providing a lot of critical infra: to an extent, some can veer into Too Big to Fail like MSFT, some have stronger internal sec teams like Google/Project Zero, Verizon/Paranoids, Cloudflare seems good. - Banking is maybe: they have funds, more risk-averse culture, heavily regulated. But healthcare is also heavily regulated and id never work in it due to the volume of exploits and lack of care. So ya, don’t work at MSFT as a sec eng IMO unless you’re on the DART team and want to see a lot of diverse incident response with legit threat actors, or want to do really low level OS sec. No idea about Apple sec eng work, on this note. This is also why the avg tenure in security careers +/- 10 years. Your sanity runs out and often pay is good enough where you can save up and do something else with your life by 30/40.

mikl 2y ago

Not the most earth-shattering revelation. Given how they’ve always needed to be dragged kicking and screaming to adopt more secure protocols, that should be obvious. For example, it’s been known that NTLM is easily cracked for over a decade, but they’re only getting around to phasing it out now.

pdimitar 2y ago

RE title: and somebody also said -- "The sun is still rising from the east". And after pulling their white beard, said wisely: "And water is still wet".

chucke1992 2y ago

I mean, I see the crappy level of security across the whole industry. And I am working at the bank. I am not sure what exactly the reason - even if the profit aside - but I suspect that there are not many people who are actually competent developer and security engineers.

shortsunblack 2y ago

The solution is complete zero trust and distrusting the network in organizations. You should treat the internal network as external -- hostile. Google does this. They were the first ones to widely adopt zero trust with BeyondCorp and there has not been a Google internal organizational breach since Aurora (which made them adopt BeyondCorp, what they call zero trust). You have completely managed endpoints, strong hardening of the endpoint and complete inventorization of all the resources in the organization. You have certificates installed onto each device. You have an ACL engine that determines whether a user should get access to a particular resource. You can use deterministic lists and also incorporate heuristics to detect anomalies (working hours, etc). All Google internal apps are internet-facing. You can open them, get redirected to the SSO portal. Come and do try to get in. You will not. Many of these security problems are solved. You just need to implement the solutions.

orf 2y ago
> You have completely managed endpoints, strong hardening of the endpoint and complete inventorization of all the resources in the organization. You have certificates installed onto each device. You have an ACL engine that determines whether a user should get access to a particular resource. None of those are “solved” for any mid or large-sized enterprise where tech isn’t their code competency. In fact, I’d say most of these are insurmountably hard. This is a “draw the whole owl” kind of response. Its very well to say that you can do things differently, but imagine Shaw Industries (22k employees, largest carpet/flooring manufacturer in the USA) doing any of that.
- alkonaut 2y ago
  I think I agree with this conclusion. But I work in a Shaw-like enterprise (only the products are more mundane than flooring). What are the hurdles we’d see if we tried it? What processes and practices are we likely using, that would break under the zero trust model?
  - zach_miller 2y ago
    I think the hard part of trying it isn't using it, it's implementing it.

m463 2y ago

I think it's the nature of Microsoft. They've done things quick instead of well. This has served them well, and during the time microsoft has been around, most of the competitors that have reversed that equation have gone by the wayside. I vaguely recall they were in the same boat decades ago with the win 3.x and win 9x - windows was a virus and bluescreen laden garbage heap. I'm not sure what OS they started really cleaning up and validating the API calls. I think windows 2000 was a major step away from shared memory space to cure some of it.

6031769 2y ago

Bear shat in woods, whistleblower says.

madrox 2y ago

I don't see a future here that doesn't involve significant legislation over network security and include jail time for major offenses. Every time something like this happens, there's always that organizational Cassandra (usually the CISO) that saw it all coming but was ignored. Sooner or later someone will get burned badly enough that the consensus will be that tech cannot regulate itself on security. We've already got this a little bit for the most egregious cases, but it's still about as secure as banks in the 1920s. A less actionable gripe I have is that we have so few players that even if the US government loses trust in Microsoft's cloud...where else will they go? There aren't a lot of players here that could handle that scale. It's like if there were only 3 banks in the world.

Zelphyr 2y ago
I can think of one solution to the "too few players" problem. Break them up.
- madrox 2y ago
  I, too, wouldn't mind living in that timeline, but that's just not going to happen for a number of reasons that are so obvious as to not be worth enumerating. Dwelling on unrealistic solutions prevents you from perceiving the possible.

ederamen 2y ago

Obviously

userbinator 2y ago

Of course when Microsoft does choose "security", it's often in the name of securing things against users.

worik 2y ago

Is it possible that insurance companies will play a role in this endless succession of insecure, critical, software systems? Can you get insurance against software system failure? If you cn, surely the providers of said insurance will take a keen interest?

Splice9160 2y ago

Shocked! I am SHOCKED....

animanoir 2y ago

Well, that’s what a company like Microsoft will always choose, so no wrong-doing here—business as usual

AdeptusAquinas 2y ago

"[Enter Company Name] Chose Profit over Security, Whistleblower Says". Any company, you pick.

o999 2y ago

Capitalism ate my face

exmsrc 2y ago

"I was very interested in that question. And one of the places that I focused on was the MSRC, which is short for Microsoft Security Response Center. This center is like a clearing house for reports of security bugs, and it was Harris' very first stop when he began warning colleagues of the flaw that he discovered. But the issue is that the center itself was understaffed and underresourced. And one employee who used to work there told me that staff is trained to think of cases in terms of how can I get to won't fix. So this center also clashed with the product teams." I used to work for the MSRC. He's right that it was understaffed and underresourced. It's one of the reasons I quit, same for many of my ex-colleagues. But I disagree with his characterization of us trying to find any way to get cases to Won't Fix. The fact is, we got many, many reports that were genuinely not vulns, and therefore shouldn't be prioritized for fixing from a security standpoint. Yes, occasionally reports may be incorrectly analyzed but that's not because we were trying to get them to Won't Fix. It's just people making mistakes now and then. "And, you know, another big issue there is that they're clashing with the product teams that they need to fix the actual issues. So they would bring a security vulnerability to a product group. They'd say, you need to fix this flaw. But those groups were often unmotivated to act fast, if at all, because compensation is tied to the release of new products and features." That's true in part, but it varies wildly between product teams. Some were incredibly responsive and knowledgeable, some were clueless about security, some just didn't prioritize it. Sometimes the fix was insufficient. When I was there, MSRC wouldn't check if the fix did what it was supposed to do, except in occasional cases where we were explicitly asked to check or if it was a particularly risky case that needed the extra scrutiny. But like he says, we were understaffed and underresourced, we simply didn't have the time to do this for every case.

mike503 2y ago

Boeing chose profits over safety. All companies choose profits over <literally anything> That's unfettered capitalism.

flybarrel 2y ago

> Product managers had little motivation to act fast, if at all, since compensation was tied to the release of new, revenue-generating products and features Only if I get rewarded that way lolllllll

ViktorRay 2y ago

”They saw it differently, Harris said. The federal government was preparing to make a massive investment in cloud computing, and Microsoft wanted the business. Acknowledging this security flaw could jeopardize the company’s chances, Harris recalled one product leader telling him. The financial consequences were enormous. Not only could Microsoft lose a multibillion-dollar deal, but it could also lose the race to dominate the market for cloud computing.” There is something fundamentally broken about an organization’s culture when this type of thinking is pervasive in the organization.