8 ms·
Cloudflare mitigated yet another Okta compromise
- uncomputation 3y ago> Recommendations for Okta Seems a bit haughty to publicly chastise another company. The tone of this article is a bit off-putting for me personally.
- kalupa 3y agohaughty? https://news.ycombinator.com/item?id=37959904 https://news.ycombinator.com/item?id=37959904
- uncomputation 3y agoYes. No one likes a sore winner. Providing your customers with assurances? Good. Providing tips to Okta customers? Sure. Publicly chastising another company you do business with? Unnecessary. That should be kept private. Just my opinion
- toomuchtodo 3y agoI am responsible for spending several hundred thousand dollars a year with Cloudflare (out of my budget). I like this style. Don’t want to get called out, get your org fixed. This is somewhere between the third and fifth breach, depending on how you’re counting.
- landemva 3y agoAre you going to move your spend, or is having a 3rd party sling words good enough for you?
- toomuchtodo 3y agoEdit: removed for subthread cleanup.
- landemva 3y agoMy bad ... cf not okta.
- deleted 3y ago[deleted]
- tchbnl 3y agoThis is the _second_ time this has happened, and it's clear Octa hasn't learned any lessons. So Cloudflare is right to call them out, and Okta should be embarrassed. What surprised me about this post is that they didn't say they were dropping them. Okta is a vulnerability to any organization.
- forkerenok 3y ago> Publicly chastising another company you do business with? Unnecessary. I think this makes more sense for strategic business partners. In the Cloudflare-Okta case I'd wager that their relationship is fairly transactional.
- StressedDev 3y agoI am not sure I would call CloudFlare a “winner” in this case. They did not win anything by getting hacked.
- Kinrany 3y agoThey do win some points on having better security than a popular security product, considering Cloudflare's own security posture is also quite important to their customers.
- StressedDev 3y agoAgrees - CloudFlare and its employees did outstanding work. My main point was calling CloudFlare a sore winner did not make sense because they did not win anything. Also, I think CloudFlare’s blog post was very good.
- djbusby 3y agoBut the recommendations are good?
- codegeek 3y agoNot really. Here is one of the recommendations: "Take any report of compromise seriously and act immediately to limit damage; in this case Okta was first notified on October 2, 2023 by BeyondTrust but the attacker still had access to their support systems at least until October 18, 2023." It is good to call Okta out here as it impacts Cloudflare's business as well and if you can't fix a critical issue for 16 days, that is bad. Remember we are talking about Auth here. A breach impacts everything.
- Bijou_x7 3y ago[flagged]
- zmgsabst 3y agoOkay — and? Do we have anything to suggest CloudFlare is factually wrong? — or was that just random conversational chaff from a brand new account distracting from the stunning incompetence of Okta in ignoring a breach for two weeks? CloudFlare has more than enough reputation to make such an allegation — and Okta should be cut from any production usage. Two weeks of failing to address auth compromise is unprofessional conduct by both Okta leadership and engineers.
- sophacles 3y agoTo be fair, it's also the second time this has happened in 2 years - I don't mean okta breaches in general, I mean it's the second time the support system has been compromized to get access to customer accounts.
- madeofpalk 3y agoSee also, https://www.beyondtrust.com/blog/entry/okta-support-unit-breach https://www.beyondtrust.com/blog/entry/okta-support-unit-bre... > We raised our concerns of a breach to Okta on October 2nd. Having received no acknowledgement from Okta of a possible breach, we persisted with escalations within Okta until October 19th when Okta security leadership notified us that they had indeed experienced a breach and we were one of their affected customers.
- StressedDev 3y agoFirst, Okta got hacked and that hack allowed CloudFlare to get hacked. That is bad. Second, one of Okta’s other customers reported the hack and Okta either ignored the report, or investigated the report and did not find the hack. That is not good. Third, Cloud Flare’s response was professional. They asked a company providing a very important service to improve because that company’s product and practices endangered CloudFlare. If Okta does not want its customers to publically complain about its actions, Okta needs to improve and do better. In particular, if someone says they have been hacked, listen to them and keep digging until you find the problem.
- deleted 3y ago[deleted]
- cybersister 3y agoAgree. CF wont have the inside scoop and they use another company's statement to bolster own thoughts. I wonder about the BeyondTrust statement too. This just doesn't sound right....and, so far, although it could happen this week, there have been no SEC filings by Okta - which would have to happen if this was a bad situation for them.
- Reubend 3y agoI'm left a bit perplexed here. It sounds to me like Cloudflare did the right thing, but I don't understand why they still want to use Okta moving forward. Surely instead of "recommending" a better security posture, they should tell Okta that they won't continue to use them as a vendor unless they follow a stricter security policy?
- StressedDev 3y agoFirst, you are assuming Okta is worse than any other vender. It is very very very possible that other authentication venders are either just as bad or even worse. Remember, no one knows how to build an unhackable computer, service, or system. Second, moving venders is VERY hard. It can take years to do it. I have worked at another cloud provider and moving from an old system to a new one is a huge undertaking. You do not just say, “See you, I am out of here.”. It can take years of work and planning. Third, you do not know what CloudFlare is doing internally. My guess is they have already spoken to Okta several times about the problems they have had. At this point, CloudFlare may actually be moving off Okta or thinking about moving off of Okta.
- dartos 3y ago[flagged]
- RiOuseR 3y ago[dead]
- pard68 3y agoSo this is now the second time Okta was compromised, dragged their feet for weeks, and then almost got a third-party/client compromised? The tone of this is fine. What isn't fine is that Cloudflare isn't dropping Okta. I might consider dropping Okta.
- rapfaria 3y agoCurrent org (thousands of IT folks) has opened an internal issue about alternatives and a migration plan
- StressedDev 3y agoA couple of points. First, Okta should get credit for publicly acknowledging the compromise. Second, expecting any company or organization to never get hacked is unrealistic. Organizations which are transparent and acknowledge their security breaches should get credit for it. Organizations which cover things up should be avoided. Third, no one knows how to build secure software or services. The closest anyone has gotten is Open BSD but it can still be hacked once you install software on it. Forth, the number of publicly acknowledged breaches does not tell us anything about Okta’s security. Here is why: - We do not know how many breaches each authentication provider has had - We do not know how many breaches were never detected by the authentication provider - We do not know why the breaches occurred. Breach causes can range from gross incompetence to “WOW, that attacker was really clever and found a new class of security bugs”. - We do not know how Okta responded to the breach. We also don’t know how its competitors respond to their breaches. My main point is security is hard and measuring security is also hard. We cannot use simple metrics to determine if an organization is a good organization or a bad one.
- hiddencost 3y ago"First, Okta should get credit for publicly acknowledging the compromise" What???? They would reasonably face civil suits if they didn't.
- StressedDev 3y ago
- kypro 3y agoThat title reads as very passive aggressive. I assume that was intentional.
- epolanski 3y agoThe whole post is borderline aggressive and nothing short of PR. If you want to communicate with okta you send them an email, you don't shame them publicly.
- HL33tibCe7 3y agoShaming them publicly might be the best way to get them to actually get off their arses and do something.
- Kinrany 3y agoI mean, aggressiveness is a form of communication. "Get your shit together, or else."
- FridgeSeal 3y agoGiven that Okta’s response in the past to being hacked amounted to a weeks-worth of “nuh uhhh we didn’t, you’re lying” before then quibbling about whether it was really an issue, I think a public shaming is definitely in order.
- NicoJuicy 3y agoThat's BS. - 2nd time Okta was breached - Cloudflare shared/detected? it earlier. How long till more Okta customers were breached because they were silent? - Cloudflare considers authentication as very dangerous. There was a post recently: "hackers login" ( can't find the post anymore?) which means they are very consistent about these types of attacks.
- darkarmani 3y ago"yet another Okta compromise" sounds passive to you?
- dsadsadsawdqdwq 3y ago[flagged]
- vosper 3y ago> A good bit of today's fraud is conducted on cloudflare How are people using Cloudflare for fraud / how does Cloudflare enable fraud? Genuine question :)
- wredue 3y agoPuts on idiot hat Don’t you know cloudfare bans ~~terrorism~~ alternate opinions. (It’s funny how fast meal team 5 is able to find and down vote people satirizing their comments)
- j16sdiz 3y agoAFAIK, it happens when the site claims cloudfare endorse them, which is not true... They were just yet another pay customer, cloudfare endorsed nobody..
- StressedDev 3y agoWow First, CloudFlare is not responsible for people misusing its service. It is not feasible for ANY cloud provider (or company) to determine which customers are legitimate and which are crooks. Second, reports of fraud do not mean there is any fraud going on. Remember, not every report is legitimate or correct. Lots of people file false reports because they dislike a person or group (sad but true).
- depr 3y agoThe title of the blog post starts with "How" but the only actual how that is in the post seems to be basically: "we used Cloudflare products X Y and Z which mitigated the issue". Unfortunate.
- OJFord 3y agoCloudflare 'dog-foods' its own products, and that saved the day; what's the problem? There might be/come a longer form technical write-up with more detail of how those products helped, if that's what you're after, CF often seems to follow that approach, sort of announcement + detail, or press + technical.
- depr 3y agoThe problem is, if you asked me how did you bake that cake, and I said: I went to my kitchen and used up my ingredients and my methods, then I would not have answered your question. >There might be/come a longer form technical write-up with more detail of how those products helped There won't be, they just used their blog to pressure Okta to improve certain things, which is their right I suppose but it makes for very boring reading.
- TheNewsIsHere 3y agoI get what you’re saying, but Cloudflare isn’t trying to keep their methods a secret. They’ve openly written before about the key takeaway: how phishing-resistant MFA (particularly security keys) have saved them in the past. Everything else is just product documentation that’s online to read.
- depr 3y agoI think in this case they _are_ keeping their methods a secret; they don't say how their detection methods work. The BeyondTrust blog post doesn't mention their methods either. In this case that is probably right because knowing what they do to detect intruders would make it easier to stay invisible.
- kaycebasques 3y agoOften I see news like this and expect the stock price to take a hit, and then am surprised to discover that "the market" doesn't care. Not so in this case: OKTA down 12% today Edit: Looks like a fairly strong down day in general so maybe it was just a coincidence. When you look at 1Y of data it's clear that OKTA swings around a lot. There's precedent for moves like this, in other words.
- pwarner 3y agoWhat I read between the lines is that CF is going to launch their own IDP, and it'll probably be better done than Okta.
- lolinder 3y agoWith most security breaches the product is serving some primary purpose other than security: security is an ancillary goal, and security breaches may not immediately cause companies to reevaluate their investment in the product. Okta is different—the problem it solves is squarely in the domain of their customers' infosec teams, and its one job is to be safer than alternative authentication methods for large enterprises. It makes sense that investors would worry about them taking a larger hit than ${random_SaaS_product} would.
- timcavel 3y ago[dead]
- donmcronald 3y ago> It appears that in our case, the threat-actor was able to hijack a session token from a support ticket which was created by a Cloudflare employee. How does a session token end up in a support ticket? Are they putting session tokens in URLs or something?
- JshWright 3y agoIt was in a HAR file
- sophacles 3y agoOkta asks for a HAR file of your okta session. By default this includes okta session tokens. The attacker who compromised oktas support systems (again) then used this session token to access customer's okta accounts as the first step in trying to gain access to the customer's systems.
- tflinton 3y agoIt’s probably worth pointing out that cloudflare uploaded sensitive tokens to their support website and their support website was compromised. Cloudflare probably shouldn’t be posting their tokens anywhere to anyone so it’s hard for me to think they’re some how in a moral high ground.
- lolinder 3y agoFor context, this is apparently a regular part of Okta's customer support process [0][1], not something Cloudflare just decided to do on their own. It's the kind of obscure process that would be hard to catch by enumerating specific rules like "don't upload HAR files to customer support even if they ask for them", and it's a technical enough process that you wouldn't expect any random employee to realize that an HAR file contains the keys to the kingdom (much less why that even matters when you're dealing with customer support for your authentication provider who seems to already have the keys). I think it's pretty fair for Cloudflare to place this in Okta's court. Okta customer support knew what they were asking for and should have had greater controls in place for dealing with those files safely. > Okta explained that when it is troubleshooting issues with customers it will often ask for a recording of a Web browser session (a.k.a. an HTTP Archive or HAR file). These are sensitive files because in this case they include the customer’s cookies and session tokens, which intruders can then use to impersonate valid users. [2] [0] Old content: http://web.archive.org/web/20230207011818/https://help.okta.com/oag/en-us/content/topics/access-gateway/troubleshooting-with-har.htm http://web.archive.org/web/20230207011818/https://help.okta.... [1] New content with updated warning: https://help.okta.com/oag/en-us/content/topics/access-gateway/troubleshooting-with-har.htm https://help.okta.com/oag/en-us/content/topics/access-gatewa... [2] https://krebsonsecurity.com/2023/10/hackers-stole-access-tokens-from-oktas-support-unit/ https://krebsonsecurity.com/2023/10/hackers-stole-access-tok...
- HL33tibCe7 3y agoIn fairness, they posted Okta tokens to Okta support. Cloudflare already trust Okta with their auth anyway. Sure, it’s still not a good idea for them to give Okta their tokens. But the above, combined with the fact that forgetting to redact a HAR is an incredibly easy mistake to make, makes me tend towards giving them a pass on this one.
- HL33tibCe7 3y agoThe really worrying thing about this is that Okta’s announcement fails to mention several factors which paint them in a bad light, like the fact they sat on a report about this compromise for 16 days.
- FridgeSeal 3y agoSurely you’re not suggesting that the company whose CEO denied being compromised for a week or so (before relenting and playing semantics games about whether it was “really” a leak or a hack) _might_ not be real trustworthy about disclosure…
- victor106 3y ago> the threat-actor was able to hijack a session token from a support ticket which was created by a Cloudflare employee. If a threat-actor stole bank my password because I stored it in a file on my laptop how is it my banks fault?
- lamontcg 3y agoIt was a support ticket opened on Okta's systems which got breached and leaked the info which was used to attack Cloudflare systems.
- victor106 3y agoThanks for correcting me. I re-read the article and I stand corrected. Okta is at fault here. That said, security is a really hard thing to get right 100% of the time. And there is no guarantee that another security vendor won’t have other issues. But Okta should’ve been more forthcoming communicating this.
- lamontcg 3y agoOkta has had a bunch of security issues fairly recently and the last one I recall reading was enough for me to not want to have anything to do with the company. This pretty much seals the deal for me. Compared to other businesses I've worked in, they seem to be having too many preventable security issues, and they're in the security business. Cloudflare's incident report here is what I'd consider forgivable and sounds like they take security very seriously, but they can't be perfect. Okta looks pretty bad.
- NicoJuicy 3y agoIt's nuts that Cloudflare is detecting almost every security vulnerability online and also helping resolving it! In this case: > In fact, we contacted Okta about the breach of their systems before they had notified us.
- raoofia778 3y ago[flagged]
- cybersister 3y ago[flagged]
- davinci123 3y agoOkta messed this up big time. Companies get breached all the time but how they handle the breach shows the maturity/mettle of the leadership team. This is the second time their comms strategy failed miserably. They need to immediately re-evaluate their incident response operating structure. Cloudfare has some good suggestions.
- NeedBigHelp 3y agoHow do I report a breach to OKTA, my company is clearly ignoring my efforts to alert them that we’ve been hacked.. I also used the 2FA app because I thought it was the newest thing to do. But, it gave me admin rights, and back door access. Ect. And nobody believes me.
- NeedBigHelp 3y agoHow do I report a breach to OKTA? Our company has been clearly hacked, and they are ignoring me. This happened through OKTA portal-ServiceNow app. I also think my iPhone is screwed because I used my iPhone to do the 2FA. It gave me full admin pass, backdoor access, and customer information. Nobody is listening.
- iceddante 3y agoIf I am reading this correctly, Cloudflare is just as culpable as Okta. It looks like a Cloudflare employee uploaded the har file with the open session.