17 ms·

1Password detects "suspicious activity" in its internal Okta account

• C60H92O6 3y ago
  Every product, and I mean every, will eventually get hacked. Putting all your keys in one basket and handing them over defies common sense, not to mention that it also makes the basket a high profile target.
• _a9 3y ago
  https://blog.1password.com/okta-incident/ https://blog.1password.com/okta-incident/ Original Incident Report: https://blog.1password.com/files/okta-incident/okta-incident-report.pdf https://blog.1password.com/files/okta-incident/okta-incident...
  • jrockway 3y ago
    That is fascinating: > A member of the IT team was engaged with Okta support, and at their request, created a HAR file from the Chrome Dev Tools and uploaded it to the Okta Support Portal. This HAR file contains a record of all traffic between the browser and the Okta servers, including sensitive information such as session cookies. In the early morning hours of Friday, Sept. 29th, an unknown actor used the same Okta session that was used to create the HAR file to access the Okta administrative portal Like your bank tells you, don't give the support person your password.
    • delive 3y ago
      I thought so too, but then was surprised to see: Based on the activity logs provided by Okta for their Support Portal, the HAR file had not been accessed by their support engineer until after the events of the incident
      • StressedDev 3y ago
        That tells me either the attacker either removed the access from the logs OR the attacker figured out how to access the HAR file without having the access recorded in the logs.
        • dannyw 3y ago
          All this says is that the support engineer didn't access the file. It doesn't say the attacker wasn't logged accessing it.
      • fernandotakai 3y ago
        >Oct 21, 2023: Okta confirmed publicly that their internal support systems were compromised. This answers how the HAR file was accessed by the attacker and that the initial compromise was not through the employee’s laptop. how can "the industry" trust okta?!
    • tjoff 3y ago
      > Like your bank tells you, don't give the support person your password. Sure, but was the user aware what the HAR-file actually contained? At the least all active sessions should be cleared after sharing something like that. But that hinges on you knowing about it. Support should also make it mandatory/automatic.
    • thayne 3y ago
      It seems to me like the browser should not include cookies in HAR files by default. Sure, have a way to enable including cookies, but it should be behind a scary warning informing the user that the cookies likely contain sensitive data.
      • hunter2_ 3y ago
        This sounds to me like the correct way to plug this leak. It will benefit additional use cases involving the HAR export beyond individual support portals sanitizing it before storage. The "Copy as cURL" feature has the same issue. As does highlighting and copying request/response headers to the clipboard. However, less sensitive data should ideally remain easily transferred, without much effort on the part of the user.
      • usrbinbash 3y ago
        How about: People shouldn't send around HAR files that contain sensitive information, or at least make sure the information contained is no longer sensitive (eg. by flushing active sessions)? HAR files are a debug tool. If I have to debug a problem with a webservice, I require them to contain all the information that was sent/received by the browser. The browser arbitrarily deciding to delete part of that information, would make it worthless to me as a debug tool.
        • usrusr 3y ago
          Reminds me of that banner Facebook writes to the browser console to warn people of pasting stuff that will hand over their session to third parties. Handing over a HAR is just as bad if there is any form of session involved. Browsers could add even more nag screens between the user and the tools, but those have zero effect once the assumption "I'm talking to a person from the hoster" is established. It's the old "put on a safety vest and a hardhat and you can walk anywhere" hack that only training can protect you from. And even with the best training, you'll never reach 100%. That's why you need many tiers of your operation is as sensitive as selling a trust store. It's well possible that 1Password are still far from being breached thanks to tiers, but it's interesting to see even people working full-time on the conflict between authentication and convenience struggle with that balancing act.
    • lelanthran 3y ago
      > Like your bank tells you, don't give the support person your password. Sure, but they uploaded it to the okta portal, not to any random support person. Most users would expect that people getting files from the company portal would be cleared to see confidential and sensitive stuff. Obviously not passwords, but still ...
      • x0x0 3y ago
        Ideally, okta would clear those sessions when someone uploads a har file. That way, you don't have to trust users did the right thing...
• insanitybit 3y ago
  > The IT team member’s credentials for all systems were rotated, switched to only using a Yubikey for MFA, I hope this spurs them to switch all employees to use Yubikeys for 2FA. Anything less than FIDO2 is really weak. Can someone explain why people still choose Okta? I personally feel way more comfortable using GSuite as an IDP. Okta suffered a pretty terrible breach and, candidly, I hadn't heard great things about their security practices before that.
  • latchkey 3y ago
    > Can someone explain why people still choose Okta? Or 1password for that matter... do you really trust them as well? "we are continuously enhancing our security measures" How about not using Okta to begin with?
    • ghshephard 3y ago
      Who would you recommend as an iDP if not Okta?
      • Nextgrid 3y ago
        At this point any self-hosted option would do just as well. Not perfect, but not only it would require a targeted attack to pwn, but if you do end up pwned at least you weren't paying tens of thousands a year for the privilege.
        • ghshephard 3y ago
          I guess I'm curious as to how someone self-hosting would be able to perform their security better than a Tier 1 iDP. It would be the equivalent of trying to self-host email. Also - never having worked for a company that hosted their own - what's the experience like - with Okta, you can essentially SSO onto hundreds (thousands?) of different cloud service providers. Similiar if you self-host?
          • Nextgrid 3y ago
• taspeotis 3y ago
  Did they detect it again? https://news.ycombinator.com/item?id=37991863 https://news.ycombinator.com/item?id=37991863
• ChrisArchitect 3y ago
  [dupe] https://news.ycombinator.com/item?id=37991863 https://news.ycombinator.com/item?id=37991863
• Thorrez 3y ago
  >1Password now becomes the second known Okta customer to be targeted in a follow-on attack. Is Ars not aware that Cloudflare was also a victim? https://blog.cloudflare.com/how-cloudflare-mitigated-yet-another-okta-compromise/ https://blog.cloudflare.com/how-cloudflare-mitigated-yet-ano...
  • nixgeek 3y ago
    BeyondTrust reported the issue. Cloudflare is the first known Okta customer targeted, and 1Password is the second known. I think that’s what Ars is saying?
    • cdchn 3y ago
      Makes sense - those are the two juiciest customers I bet Okta has, also the best equipped to detect any potential problems vectoring via Okta.
      • Thorrez 3y ago
        All 3 customers detected it quickly on their own. It sounds from their blog posts that BeyondTrust and Cloudflare both had good alerting that immediately alerted them to the problem. It sounds like 1Password's alerting wasn't as good, and they got lucky that the impacted employee saw a suspicious email saying "here's the report you requested" thought "I didn't request that report" and reported the suspicious email to the security response team. https://www.beyondtrust.com/blog/entry/okta-support-unit-breach https://www.beyondtrust.com/blog/entry/okta-support-unit-bre... https://blog.cloudflare.com/how-cloudflare-mitigated-yet-another-okta-compromise/ https://blog.cloudflare.com/how-cloudflare-mitigated-yet-ano... https://blog.1password.com/okta-incident/ https://blog.1password.com/okta-incident/
    • Thorrez 3y ago
      There were 3 known customers targeted. >BeyondTrust reported the issue. All 3 customers discovered the issue before Okta fixed the issue. I think all 3 reported the issue to Okta before Okta fixed the issue, but I'm not sure. 1Password and BeyondTrust both reported the issue to Okta before Okta discovered the issue (although 1Password wasn't sure whether the cause was a compromise of Okta's support or malware on the laptop; 1Password didn't find malware, but still thought there might be some). I'm not sure whether Cloudflare reported the issue to Okta before Okta fixed the issue, but it seems likely due to timing. Cloudflare was compromised Oct 18 and immediately discovered it, and the very next day Okta fixed the issue, whereas previously the issue had been languishing for 19 days. That seems to indicate to me that Cloudflare reported it to Okta and Okta kicked their incident response into high gear due to Cloudflare's report. All 3 customers posted blog posts about the issue after Okta posted a blog post about the issue. The order they were targeted was: 1Password (Sept 29), BeyondTrust (Oct 2), Cloudflare (Oct 18). I think they reported the issues to Okta in that same order (BeyondTrust reported the issue to Okta the same day they were targeted, I'm not sure about 1Password, and as mentioned above, I suspect Cloudflare reported the issue to Okta the same day they were targeted). The order they posted blog posts about the issue was: Okta, BeyondTrust, Cloudflare, 1Password. https://sec.okta.com/harfiles https://sec.okta.com/harfiles https://www.beyondtrust.com/blog/entry/okta-support-unit-breach https://www.beyondtrust.com/blog/entry/okta-support-unit-bre... https://blog.cloudflare.com/how-cloudflare-mitigated-yet-another-okta-compromise/ https://blog.cloudflare.com/how-cloudflare-mitigated-yet-ano... https://blog.1password.com/okta-incident/ https://blog.1password.com/okta-incident/ https://krebsonsecurity.com/2023/10/hackers-stole-access-tokens-from-oktas-support-unit/ https://krebsonsecurity.com/2023/10/hackers-stole-access-tok... Ars never mentions Cloudflare. Ars says >Security firm BeyondTrust said it discovered the intrusion after an attacker used valid authentication cookies in an attempt to access its Okta account. The attacker could perform “a few confined actions,” but ultimately, BeyondTrust access policy controls stopped the activity and blocked all access to the account. 1Password now becomes the second known Okta customer to be targeted in a follow-on attack. That seems to say "BeyondTrust was first and 1Password was second".
• evanjrowley 3y ago
  Why wouldn't 1Password implement their own SSO rather than use Okta?
  • StressedDev 3y ago
    Writing a SSO system is very hard. They have very high up time requirements and very high security requirements. 1Password might select a different vendor or self-host an open source or commercial SSO system. Still, there are no perfect answers, and each choice has tradeoffs. Even worse, the new system is not guaranteed to be any better than the old system. One thing which frustrates me when reading a lot of these posts is a lot of people assume that security is easy, organizations should be perfect, and breaches only occur if an organization or service is terrible. None of these are true.
    • m1keil 3y ago
      True, but in this case we are literally dealing with a company that claims they are really good at data security and it is the reason why the users are paying them. It is one thing to outsource a monitoring solution. But outsourcing your SSO to a third party? They can only guarantee how well 1Password works, they cannot guarantee how well Okta works.
      • pantulis 3y ago
        I am also baffled by this. It's not like 1Password doesn't have the chops to roll their own SSO in a secure way specially given that resorting to use Okta puts you just on top of their attack surface.
    • clwg 3y ago
      You're absolutely correct. Identity and SSO are hard and require a high degree of competency to do well, and they're massively impactful when they go wrong. I think the problem is trust. So the question is, why would 1Password trust Okta?
      • 3y ago
• HtmlProgrammer 3y ago
  Now, will 1Password pull a Lastpass and slowly and silently keep updating their blogpost, each time growing from “a lil bit compromised” to “oopsie we lost all your data through negligence but we care about your privacy we swear”
  • AceJohnny2 3y ago
    In this case, Okta deserves more of your scorn. They're the ones who've been slow to disclose their failures. For example: https://blog.cloudflare.com/how-cloudflare-mitigated-yet-another-okta-compromise/ https://blog.cloudflare.com/how-cloudflare-mitigated-yet-ano...
    • schainks 3y ago
      This. Okta leadership is asleep at the security wheel because printing money is the name of their game.
      • FBISurveillance 3y ago
        They print money at -111M loss per quarter; I wouldn't say they are succeeding at that either, technically speaking! Edit: when time comes rolling their debt at a higher interest rate (post-ZIRP) I expect them to trim down their workforce and start costcutting. They currently have 6k+ employees.
        • ilyt 3y ago
          I think we'd be in far healthier society if corporations not turning profit for half a decade would just die instead of entangling more users into their failure. via investment money...
    • trvz 3y ago
      1Password shouldn't be using Okta in the first place.
  • hn_throwaway_99 3y ago
• azifali 3y ago
  Maybe it is time for a decentralized vault. We've built this for Privileged access - https://authnull.com https://authnull.com. Demo video here: https://authnull.com/videos/2023-07-01-passwordless-auth-demo https://authnull.com/videos/2023-07-01-passwordless-auth-dem...
• varjag 3y ago
  It's remarkable how C memory safety vulnerabilities are the most talked about. But the most impactful ones are due to insight collapse from over-complexity caused by following so-called best design practices.
  • c0balt 3y ago
    What can be measured tends to get optimized. We have numerical, easily gatherable data on software vulnerabilities. However we, imo, lack information on how (and sometimes even when) organizations get data stolen. The reasons behind the latter are most likely complex, as they pertain to humans and as auch it's easy to look for a rescue in technology.
  • _8j50 3y ago
    It's just complexity. Stuff gets missed, stuff gets updated but the cascading effect it will have on other logic gets overlooked,etc... complexity breeds vulnerability. But scdlt, layered security defenses and countermeasures can reduce risk. I like to think our inability to grasp vast levels of complexity is the root cause.
  • CGamesPlay 3y ago
    Organizational bugs can be very impactful to the organization that has them. But the memory safety bugs are typically exploitable via automatic methods and affect all organizations using the software. For example, Heartbleed was so talked about because it was so impactful.
    • varjag 3y ago
      There's plenty of really nasty C specific zero days out there, but the bulk have rather limited applicability: witness the latest curl CVE. Potential compromise of the world's largest password silo however sounds clearly impactful. And whether Heartbleed was worse than log4shell is mostly philosophical.
      • ilyt 3y ago
        > And whether Heartbleed was worse than log4shell is mostly philosophical. Most definitely not. Heartbleed means that as long as your are on the internet they have your certs, full stop. Log4shell required access in both directions (to inject the vulnerable string, then for target machine to load the payload off internet) to do anything. The sheer fact that you think it is 'philosophical' difference points to you not understanding anything about the topic. log4shell could be trivally prevented by common security practices like "not allowing your apps to freely access anything on internet", heartbleed could not.
• usrbinbash 3y ago
  And people keep asking me why I use a self-hosted, self synced, password manager instead of using one of those super-easy, super-helpful online services to do it for me. For the same reason why I don't throw my apartment keys into the local train stations safebox.
  • realusername 3y ago
    Same, I use the pass command line + git (since it's integrated with it) and I never had a single problem until now. Additionally I can even use it in scripts. There's no server breach, no web extension exploit, no server errors making me lose me all my passwords, it just works.
  • fofoz 3y ago
    I'm curious. Which one are you using?
    • usrbinbash 3y ago
      https://www.passwordstore.org/ https://www.passwordstore.org/
      • 8organicbits 3y ago
        I think this works well if you're already invested in GPG for code signing or something and use terminals a lot. It's a little unusual as key management uses asymmetric crypto, so you can add a password without opening your keyring. The passwords are stored in directories and the directory names are the plaintext name of the password (i.e. the site and username). So it doesn't offer privacy. Tampering is possible, you can delete a password without unlocking the keychain by deleting the directory. So it only really protects read access to a password.
        • ilyt 3y ago
          We use similiar gpg-based one in our Configuration Management infrastructure, works really well. The data files are encrypted with keys of the CM main servers and other admins, GIt has history in case something that should not get removed got removed and if you want you can also force that the data will always be signed with certain keys on server side git hooks if you want to have more accountability than just git logs. With little config even git diff/git log work showing unencrypted (if you have right key) content.
• computerfriend 3y ago
  Centralised authentication is a mistake.
• jon-wood 3y ago
  I think something people are missing here is that outsourcing your SSO isn't just about whether it's technically easier to do so, or having the competence to run it. A significant factor is also being able to say on contracts with clients that you use a reputable provider for SSO, who are then on the hook for dealing with all the documentation around precisely how they do that, and how they ensure key material is secured. In the case of 1Password its a bit more interesting, since they presumably already have all that sorted because that is literally the product that they're selling, but in most cases it's easier to say "no one in the organisation has access to keys" rather than "no one has access, except Bob, because he runs the SSO server, but we really trust him you know".
  • codeflo 3y ago
    I agree about the messaging, but in terms of actual security: What about your service provider’s “Bob”?
    • easton 3y ago
      If Azure AD was compromised your clients were all probably compromised too, so in that way you’re probably fine. I’m sure that Microsoft/Okta/Google hold the keys in some kind of HSM or whatever, but there’s probably a Bob there that could get in if he really wanted to. You have to figure out what you’re threat modeling for, most people aren’t worried about the big SSO providers being breached because it would be expensive to work around them completely.
      • imglorp 3y ago
        But AD is compromised and Azure clients are open... https://news.ycombinator.com/item?id=36770235 https://news.ycombinator.com/item?id=36770235
        • ilyt 3y ago
          And you can just wash hands because you used "industry's best practice" and continue the security theatre.
• supermatt 3y ago
  What sort of best practices are there for monitoring apps for "suspicious behaviour"? I understand some basics, like checking request/error rates on service scopes, etc, but how sophisticated do these things get? Is there any "intelligent" tooling that people use for this kind of anomaly/threat detection where I can just point it at an event stream and it will identify anomalous behaviour, or do i need to consider everything up-front and add rules for what i need to monitor?
  • Cthulhu_ 3y ago
    I am an armchair layperson with no experience in this, so take this with a grain of salt. First order of business is an audit trail; every operation done on a user's account or data, every change in the system, should be in an audit log containing timestamps, user, etc. Especially things like updating passwords or account details. Once you have that event stream, you can release your data analysis tooling. A few days / weeks / months of activity gives you a baseline of what's normal, so something that stands out - like idk, mass password or email updates - should trigger alarms. But that's just armchair hypotheses; is there anyone on here that has experience with audit logs and what to do with them?
  • Muromec 3y ago
    That’s a good idea for AI startup — Security ad Service. Just proxy all your requests through a third party service, which will detect anomalies. But if doesn’t, the service is no liable.
  • sleek 3y ago
    https://www.obsidiansecurity.com/ https://www.obsidiansecurity.com/
• kevlened 3y ago
  The ability to reuse these extracted cookies is an exfiltration issue. The solution to exfiltration is ensuring the cookie is only valid from a specific client (aka token binding). Because token binding is just a signature, debugging isn’t crippled, because the content is still readable while the cookies in the HAR files stay safe. We’ll likely see more session implementations using token binding soon. (probably not mTLS due to the UX)
  • sanHACKING78YOU 3y ago
    [flagged]
  • ec109685 3y ago
    How would that work in practice to prevent this hijacking of session cookies from browser based apps?
    • kevlened 3y ago
      These systems work using pub/private keys, signing each outgoing request (much like traditional tls). If you were able to get the cookies, you’d be unable to use them from another browser unless you also extracted the private key.
      • ec109685 3y ago
        Yeah, I get the idea and it seems promising. But how would a company deploy a solution today using this technique?
        • kevlened 3y ago
          While the TLS-based version was removed from Chrome [0], you can implement a less efficient (and slightly less secure) version in js. I created an example [1]. The gist is: 1. Client creates a key pair in js 2. Client sends pub key on initial auth 3. Server validates auth and attaches pub key to cookie 4. Client signs each request payload with a timestamp 5. Server validates signature and timestamp The security issue is in step 1. WebCrypto can generate non-extractable private keys and store them in indexeddb. However, this assumes no malicious code flips the “extractable” flag before the pair is generated. So this strategy is trust-on-first-use. [0] - https://chromestatus.com/feature/5097603234529280 https://chromestatus.com/feature/5097603234529280 [1] - https://github.com/kevlened/prevent-exfiltration/blob/main/index.mjs https://github.com/kevlened/prevent-exfiltration/blob/main/i...
• OSI-Auflauf 3y ago
  [dead]
• tikkun 3y ago
  What's the worst case? My understanding is that everything is encrypted at rest and in transit, therefore even if there's a breach, user passwords would be secure.
  • ezfe 3y ago
    Your understanding is correct, I believe that a compromised software update would have to be issued to skim passwords out of users databases.
    • smileybarry 3y ago
      Essentially, yes. And that means duping a developer into adding your code somehow, since builds are digitally signed with a hardware token.
      • dzikimarian 3y ago
        Or supply chain attack
        • smileybarry 3y ago
          But that requires it to be present before the build is signed, ergo, early in the supply chain. Otherwise, the signature breaks and the build isn’t trusted for the rest of it.
  • zelon88 3y ago
    When you have root access to the environment where the data is encrypted at rest, there is the possibility that you can exploit that environment to either reveal the encryption keys, learn valuable information about the encryption scheme, or directly decrypt the data through internal applications. For example, you have direct console access to a database, and when you view the data it is encrypted. But if you can steal the credentials of an administrator you may be able to login to an admin portal as an employee and make their SaaS decrypt it for you. Once you nail that down you script it with Bash + Curl and dump the database that way. Several hours later you have a freshly decrypted database to play with. Also, when you have root access to the environment where the data is encrypted in transit you can intercept one of the handshakess in the front or back-end to decrypt the data in transit. The idea is to impersonate both sides of the connection, then you can write your own certificates and create your own handshake. For example, maybe there is a web server and a caching server. The caching server sends data to a user outside the network with one certificate, and the web server sends data to the caching server with another certificate. If you perform a man-in-the-middle on the web server -> caching server, then you might be able to rewrite the encryption handshake, depending on the security configuration of the back-end. None of this would throw an error on the users browser.
• declan_roberts 3y ago
  Once upon a time we purchased 1Password for a set price. The software was completely offline, storing the encrypted vault locally or in dropbox. No need to worry about hackers because there was nothing online to steal. But somebody ran the numbers and decided the way forward to increased profitability was to move everyone's data to the cloud and charge a subscription. And here we are, wondering if our data was compromised.
  • barbazoo 3y ago
    > But somebody ran the numbers and decided the way forward to increased profitability was to move everyone's data to the cloud and charge a subscription. And, you know, the huge increase in convenience for all those people not savvy enough to do it themselves. Also as far as we know, how is it different to have your vault in Dropbox? Dropbox could be hacked and that data is famously not encrypted. We don't know of anyone's 1PAss vault actually being breached yet, do we?
    • sugarpile 3y ago
      1. Setup to sync via icloud was very straightforward and 100% fine for "non-tech saavy" 2. Why would it matter if dropbox was hacked? Your vault password was never sent to to dropbox. It was just a dumb store for an encrypted vault. The calculus changes now that the vault is online and stored by the same party you're sending the password to.
      • insanitybit 3y ago
        Your vault password is never sent to 1Password either.
      • smileybarry 3y ago
        > Your vault password was never sent to to dropbox. It was just a dumb store for an encrypted vault. The calculus changes now that the vault is online and stored by the same party you're sending the password to. You never send your password or account key to 1Password. Each side authenticates the other via cryptographic challenges and you receive the same encrypted database that 1P stores, as a dumb file host. They have a whole whitepaper on the security design of 1Password accounts: https://1passwordstatic.com/files/security/1password-white-paper.pdf#Account%20password%20and%20Secret%20Key https://1passwordstatic.com/files/security/1password-white-p... Technically, the earlier OPVault format stored on Dropbox/iCloud/locally was less secure due to generating a key just from your password.
• declan_roberts 3y ago
  I think the fault here is squarely on Okta. They're asking for these plain-text HAR-encoded sessions for troubleshooting but crossing their fingers hoping the end user properly sanitizes it. Why not sanitize the HAR data on upload so that by the time it hits your system and is available to your underpaid and understaffed support techs it's completely sanitized to only the relevant and non-secure portions? HAR is structured data and is very easy to sanitize programatically. It's not rocket science.